Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learning algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

DeepMerge – II. Building robust deep learning algorithms for merging galaxy identification across domains

In astronomy, neural networks are often trained on simulation data with the prospect of being used on telescope observations. Unfortunately, training a model on simulation data and then applying it to instrument data leads to a substantial and potentially even detrimental decrease in model accuracy on the new target dataset. Simulated and instrument data represent different data domains, and for an algorithm to work in both, domain-invariant learning is necessary. Here we employ domain adaptation techniques— Maximum Mean Discrepancy (MMD) as an additional transfer loss and Domain Adversarial Neural Networks (DANNs)— and demonstrate their viability to extract domain-invariant features within the astronomical context of classifying merging and non-merging galaxies. Additionally, we explore the use of Fisher loss and entropy minimization to enforce better in-domain class discriminability. We show that the addition of each domain adaptation technique improves the performance of a classifier when compared to conventional deep learning algorithms. We demonstrate this on two examples: between two Illustris-1 simulated datasets of distant merging galaxies, and between Illustris-1 simulated data of nearby merging galaxies and observed data from the Sloan Digital Sky Survey. The use of domain adaptation techniques in our experiments leads to an increase of target domain classification accuracy of up to ~20%. With further development, these techniques will allow astronomers to successfully implement neural network models trained on simulation data to efficiently detect and study astrophysical objects in current and future large-scale astronomical surveys.

galaxies: interactions↗
NASA NTRS2017

Challenges in the Verification of Reinforcement Learning Algorithms

Machine learning (ML) is increasingly being applied to a wide array of domains from search engines to autonomous vehicles. These algorithms, however, are notoriously complex and hard to verify. This work looks at the assumptions underlying machine learning algorithms as well as some of the challenges in trying to verify ML algorithms. Furthermore, we focus on the specific challenges of verifying reinforcement learning algorithms. These are highlighted using a specific example. Ultimately, we do not offer a solution to the complex problem of ML verification, but point out possible approaches for verification and interesting research opportunities.

Van Wesel, Perry↗
DOE OSTI2025

Materials Learning Algorithms (MALA): Scalable machine learning for electronic structure calculations in large-scale atomistic simulations

We present the Materials Learning Algorithms (MALA) package, a scalable machine learning framework designed to accelerate density functional theory (DFT) calculations suitable for large-scale atomistic simulations. Using local descriptors of the atomic environment, MALA models efficiently predict key electronic observables, including local density of states, electronic density, density of states, and total energy. The package integrates data sampling, model training and scalable inference into a unified library, while ensuring compatibility with standard DFT and molecular dynamics codes. We demonstrate MALA's capabilities with examples including boron clusters, aluminum across its solid-liquid phase boundary, and predicting the electronic structure of a stacking fault in a large beryllium slab. Scaling analyses reveal MALA's computational efficiency and identify bottlenecks for future optimization. With its ability to model electronic structures at scales far beyond standard DFT, MALA is well suited for modeling complex material systems, making it a versatile tool for advanced materials research.

Density functional theory↗
DOE OSTI2022

Three-dimensional realizations of flood flow in large-scale rivers using the neural fuzzy-based machine-learning algorithms

Machine learning methods have been extensively used to study the dynamics of complex fluid flows. One such algorithm, known as adaptive neural fuzzy inference system (ANFIS), can generate data-driven predictions for flow fields, but has not been applied to natural geophysical flows in large-scale rivers. Herein, we demonstrate the potential of ANFIS to produce three-dimensional (3D) realizations of the instantaneous flood flow field in several large-scale, virtual meandering rivers. The 3D dynamics of flood flow in large-scale rivers were obtained using large-eddy simulation (LES). The LES results, i.e., the 3D velocity components, were employed to train the learnable coefficients of an ANFIS. Further, the trained ANFIS, along with a few time-steps of LES results (precursor data) were then used to produce 3D realizations of flood flow fields in large-scale rivers with geometries other than the one the ANFIS was trained with. We also used the trained ANFIS to generate 3D realizations of river flow at a discharge other than that the ANFIS was trained with. The flow field results obtained from ANFIS were validated using separate LES runs to assess the accuracy of the 3D instantaneous realizations of the machine learning algorithm. An error analysis was conducted to quantify the discrepancies among the ANFIS and LES results for various flood flow predictions in large-scale rivers.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1995

Cascade Error Projection: An Efficient Hardware Learning Algorithm

A new learning algorithm termed cascade error projection (CEP) is presented. CEP is an adaption of a constructive architecture from cascade correlation and the dynamical stepsize of A/D conversion from the cascade back propagation algorithm.

learning algorithm pattern recognition cascade err↗
DOE OSTI2022

SMALE: Enhancing Scalability of Machine Learning Algorithms on Extreme-Scale Computing Platforms

Deployment and execution of machine learning tasks on extreme-scale computing platforms face several significant technical challenges: 1) High computing cost incurred by dense networks – The computing workload of deep networks with densely-connected topology increases rapidly with the network size, imposing a non-scalable computing model of extreme-scale computing platforms; 2) Non-optimized workload distribution – Many advanced deep learning algorithms, e.g., sparsification and irregular net-work topology, produce very unbalanced workload distribution on extreme-scale computing platforms. The computation efficiency is greatly hindered by the incurred data and computation redundancies as well as long tails of the node with extensive workload; 3) Constraints in data movement and I/O bottle-neck – Inter-node data movement in extreme-scale computing platforms are associated with high energy and latency costs, and subject to the constraints of I/O bandwidth; and 4) Generalization of algorithm realization and acceleration on computing platforms – The large varieties of machine learning algorithms and structures of extreme-scale computing platforms make the derivation of a generalized algorithm realization and acceleration method very challenging, which, however, is the requirement by domain scientists and interested users. We call the above challenges Smale’s Problems in Machine Learning and Understanding for High-Performance Computing Scientific Discovery. The objective of our three-year research project is to develop a holistic innovation set at structure, assembly, and acceleration layers of machine learning algorithms to address the above challenges in algorithm deployment and execution. Three tasks are particularly performed, including: At the algorithm structure level, we investigate the techniques that can structurally sparsify on the topology of deep networks for computing workload reduction. We also study clustering and pruning techniques that can optimize the workload distributions over the extreme-scale computing platforms; At the algorithm assembly level, we derive a unified learning framework for unsupervised transfer learning and dynamic growing capabilities. Novel training methods are also exploited to enhance the training efficiency of the proposed framework; At the algorithm acceleration level, we will develop a series of techniques that can accelerate the computation of sparse matrix operations, which are one of the core executions in deep learning and optimize memory access of the concerned platforms. Our proposed techniques attack the fundamental problems in machine learning algorithms running on extreme-scale computing platforms by vertically integrating the solutions at three closely entangled layers, paving the long-term scaling path of machine learning applications under DOE context. Three tasks corresponding to the above respective research orientations are performed during the three-year project period with our collaborators at ORNL. The outcome of the proposed project is anticipated to form a holistic solution set of novel algorithms and network topologies, efficient training techniques, and fast acceleration methods to promote the computing scalability of the machine learning applications of particular interest to DOE.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1995

Cascade Error Projection Learning Algorithm

A detailed mathematical analysis is presented for a new learning algorithm termed cascade error projection (CEP) and a general learning frame work. This frame work can be used to obtain the cascade correlation learning algorithm by choosing a particular set of parameters.

learning algorithm neural network cascade error pr↗
DOE OSTI2023

A Hybrid Optimization and Deep Learning Algorithm for Cyber-Resilient DER Control

With the proliferation of distributed energy resources (DERs) in the distribution grid, it is a challenge to effectively control a large number of DERs resilient to the communication and security disruptions, as well as to provide the online grid services, such as voltage regulation and virtual power plant (VPP) dispatch. To this end, a hybrid feedback-based optimization algorithm along with deep learning forecasting technique is proposed to specifically address the cyber-related issues. The online decentralized feedback-based DER optimization control requires timely, accurate voltage measurement from the grid. However, in practice such information may not be received by the control center or even be corrupted. Therefore, the long short-term memory (LSTM) deep learning algorithm is employed to forecast delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder with high penetration of PV systems is used to validate the efficiency of the proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost voltage can effectively improve the performance of the DER control algorithm in the practical cyber-physical architecture; and 2) the LSTM forecasting strategy outperforms other strategies of using previous message and skipping dual parameter update.

cyber-resilient algorithm↗
NASA NTRS1996

Cascade Error Projection: A Learning Algorithm for Hardware Implementation

In this paper, we workout a detailed mathematical analysis for a new learning algorithm termed Cascade Error Projection (CEP) and a general learning frame work. This frame work can be used to obtain the cascade correlation learning algorithm by choosing a particular set of parameters. Furthermore, CEP learning algorithm is operated only on one layer, whereas the other set of weights can be calculated deterministically. In association with the dynamical stepsize change concept to convert the weight update from infinite space into a finite space, the relation between the current stepsize and the previous energy level is also given and the estimation procedure for optimal stepsize is used for validation of our proposed technique. The weight values of zero are used for starting the learning for every layer, and a single hidden unit is applied instead of using a pool of candidate hidden units similar to cascade correlation scheme. Therefore, simplicity in hardware implementation is also obtained. Furthermore, this analysis allows us to select from other methods (such as the conjugate gradient descent or the Newton's second order) one of which will be a good candidate for the learning technique. The choice of learning technique depends on the constraints of the problem (e.g., speed, performance, and hardware implementation); one technique may be more suitable than others. Moreover, for a discrete weight space, the theoretical analysis presents the capability of learning with limited weight quantization. Finally, 5- to 8-bit parity and chaotic time series prediction problems are investigated; the simulation results demonstrate that 4-bit or more weight quantization is sufficient for learning neural network using CEP. In addition, it is demonstrated that this technique is able to compensate for less bit weight resolution by incorporating additional hidden units. However, generation result may suffer somewhat with lower bit weight quantization.

Duong, Tuan A.↗
NASA NTRS2000

Any Two Learning Algorithms Are (Almost) Exactly Identical

This paper shows that if one is provided with a loss function, it can be used in a natural way to specify a distance measure quantifying the similarity of any two supervised learning algorithms, even non-parametric algorithms. Intuitively, this measure gives the fraction of targets and training sets for which the expected performance of the two algorithms differs significantly. Bounds on the value of this distance are calculated for the case of binary outputs and 0-1 loss, indicating that any two learning algorithms are almost exactly identical for such scenarios. As an example, for any two algorithms A and B, even for small input spaces and training sets, for less than 2e(-50) of all targets will the difference between A's and B's generalization performance of exceed 1%. In particular, this is true if B is bagging applied to A, or boosting applied to A. These bounds can be viewed alternatively as telling us, for example, that the simple English phrase 'I expect that algorithm A will generalize from the training set with an accuracy of at least 75% on the rest of the target' conveys 20,000 bytes of information concerning the target. The paper ends by discussing some of the subtleties of extending the distance measure to give a full (non-parametric) differential geometry of the manifold of learning algorithms.

Wolpert, David H.↗
DOE OSTI2022

Using dorsal surface for individual identification of dairy calves through 3D deep learning algorithms

Advances in machine learning techniques have allowed the development of computer vision systems (CVS) that can accurately predict several phenotypes of interest for livestock operations. In this context, 3D images taken from a top-down view are particularly useful for estimating body condition score, growth development, and body biometrics in cattle. Frequently, such CVS rely on identification (ID) systems, such as electronic tags, as a way to match animal ID and the predicted phenotype. However, the same 3D images used to predict body weight and other animal biometrics could be adopted for animal recognition as well. Such alternative would optimize CVS to recognize animal ID and monitor growth development simultaneously while leveraging the same hardware infrastructure. Furthermore, this strategy could be used to recognize animals with similar color patterns. Nonetheless, growing animals are continuously changing body shape, which could limit its use as an invariant feature for pattern recognition. Thus, the objectives of this study were: (1) to compare algorithms for different 3D object representations to identify individual animals; and (2) to evaluate how short-term changes in body shape due to animal growth affect the predictive performance of these algorithms. For objective 1, the algorithms were trained (n = 4,558) and tested (n = 1,139) using images from 38 Holstein calves. For objective 2, we designed three different experiments using images (n = 2,347) from five Holstein calves taken over six weeks during their growing period, always training and testing on different weeks. Each experiment evaluated how changing a different parameter of the image capturing procedure affected the predictive ability of the trained algorithms. In the first experiment, we varied the total number of images per animal in the training set; in the second experiment, we varied the number of weeks while keeping a fixed number of images in the training set; and in the third experiment, we skipped weeks between images in the training and test sets. The F 1 score for objective (1) was up to 0.804 when testing with the last frames of each video, and up to 0.959 when using random frames for testing. For objective (2), the F 1 score was up to 0.947 for the first experiment when using 130 images per animal; up to 0.979 for the second experiment when using all five weeks; and up to 0.917 when not skipping weeks between training and testing. In conclusion, these results show that deep learning algorithms can be used to identify individual animals through their dorsal area 3D surfaces, and, from our experiments using calves in their growing period, that they are robust enough to account for changes in body shape and size, making them a promising tool for animal recognition during growth.

3D neural networks↗
NASA NTRS1995

Cascade Error Projection: A New Learning Algorithm

A new neural network architecture and a hardware implementable learning algorithm is proposed. The algorithm, called cascade error projection (CEP), handles lack of precision and circuit noise better than existing algorithms.

neural network learning algorithm cascade error pr↗
DOE OSTI2025

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. Here, we explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer

We present regression and compression algorithms for lattice QCD data utilizing the efficient binary optimization ability of quantum annealers. In the regression algorithm, we encode the correlation between the input and output variables into a sparse coding machine learning algorithm. The trained correlation pattern is used to predict lattice QCD observables of unseen lattice configurations from other observables measured on the lattice. In the compression algorithm, we define a mapping from lattice QCD data of floating-point numbers to the binary coefficients that closely reconstruct the input data from a set of basis vectors. Since the reconstruction is not exact, the mapping defines a lossy compression, but, a reasonably small number of binary coefficients are able to reconstruct the input vector of lattice QCD data with the reconstruction error much smaller than the statistical fluctuation. In both applications, we use D-Wave quantum annealers to solve the NP-hard binary optimization problems of the machine learning algorithms.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2018

Application of Machine-Learning Algorithms for On-Board Asteroid Shape Model Determination

The Application of Machine-learning Algorithms for On-board Asteroid Shape Model Determination project will develop an innovative system for spacecraft navigation to expand the capability of small spacecraft to meet the critical challenges associated with small-body exploration. Such challenges include accurate navigation in a microgravity environment and precision targeting of particular locations on an asteroid surface for sample collection. This on-board system will cut the computational "umbilical" back to Earth-currently necessary for the generation of a global shape model that requires thousands of images with sufficient resolution and adequate variation of incidence and emission angles, processed manually by a team of experts on Earth for several months. Small satellites have limited bandwidth and are unable to downlink the data volume required for this processing, restricting their ability to perform deep-space asteroid exploration.

Machine learning algorithms↗
DOE OSTI2024

Vehicle Position Detection Based on Machine Learning Algorithms in Dynamic Wireless Charging

Dynamic wireless charging (DWC) has emerged as a viable approach to mitigate range anxiety by ensuring continuous and uninterrupted charging for electric vehicles in motion. DWC systems rely on the length of the transmitter, which can be categorized into long-track transmitters and segmented coil arrays. The segmented coil array, favored for its heightened efficiency and reduced electromagnetic interference, stands out as the preferred option. However, in such DWC systems, the need arises to detect the vehicle’s position, specifically to activate the transmitter coils aligned with the receiver pad and de-energize uncoupled transmitter coils. This paper introduces various machine learning algorithms for precise vehicle position determination, accommodating diverse ground clearances of electric vehicles and various speeds. Through testing eight different machine learning algorithms and comparing the results, the random forest algorithm emerged as superior, displaying the lowest error in predicting the actual position.

47 OTHER INSTRUMENTATION↗
NASA NTRSDate not supplied

Machine Learning Algorithms for Alignment Verification of the Roman Space Telescope

The Nancy Grace Roman Telescope is a NASA observatory designed to unravel the secrets of dark energy and dark matter, search for and image exoplanets, and explore many topics in infrared optics. Scheduled to launch no earlier than October 2026, this 2.4 meter aperture telescope has a field of view 100 times greater than the Hubble Space Telescope. The mission is currently in its construction phase, where the telescope and its two instruments will soon be aligned together to ensure proper pupil matching. To help verify this alignment, multiple point sources above the entrance pupil of the telescope will illuminate the optical path through the telescope-instrument system, and shadows of various obstructions in the system will be analyzed using machine learning algorithms to determine the pupil matching error. This presentation discusses the test approach and the machine learning algorithms employed, as well as our uncertainty predictions based on a modeled Monte-Carlo analysis of the test.

Telescope↗