Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Stable parallel training of Wasserstein conditional generative adversarial neural networks

In this work, we propose a stable, parallel approach to train Wasserstein conditional generative adversarial neural networks (W-CGANs) under the constraint of a fixed computational budget. Differently from previous distributed GANs training techniques, our approach avoids inter-process communications, reduces the risk of mode collapse and enhances scalability by using multiple generators, each one of them concurrently trained on a single data label. The use of the Wasserstein metric also reduces the risk of cycling by stabilizing the training of each generator. We illustrate the approach on the CIFAR10, CIFAR100, and ImageNet1k datasets, three standard benchmark image datasets, maintaining the original resolution of the images for each dataset. Performance is assessed in terms of scalability and final accuracy within a limited fixed computational time and computational resources. To measure accuracy, we use the inception score, the Fréchet inception distance, and image quality. An improvement in inception score and Fréchet inception distance is shown in comparison to previous results obtained by performing the parallel approach on deep convolutional conditional generative adversarial neural networks as well as an improvement of image quality of the new images created by the GANs approach. Weak scaling is attained on both datasets using up to 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Scalable training of trustworthy and energy-efficient predictive graph foundation models for atomistic materials modeling: a case study with HydraGNN

We present our work on developing and training scalable, trustworthy, and energy-efficient predictive graph foundation models (GFMs) using HydraGNN, a multi-headed graph convolutional neural network architecture. HydraGNN expands the boundaries of graph neural network (GNN) computations in both training scale and data diversity. It abstracts over message passing algorithms, allowing both reproduction of and comparison across algorithmic innovations that define nearest-neighbor convolution in GNNs. This work discusses a series of optimizations that have allowed scaling up the GFMs training to tens of thousands of GPUs on datasets consisting of hundreds of millions of graphs. Our GFMs use multitask learning (MTL) to simultaneously learn graph-level and node-level properties of atomistic structures, such as energy and atomic forces. Using over 154 million atomistic structures for training, we illustrate the performance of our approach along with the lessons learned on two state-of-the-art US Department of Energy (US-DOE) supercomputers, namely the Perlmutter petascale system at the National Energy Research Scientific Computing Center and the Frontier exascale system at Oak Ridge Leadership Computing Facility. The HydraGNN architecture enables the GFM to achieve near-linear strong scaling performance using more than 2000 GPUs on Perlmutter and 16,000 GPUs on Frontier.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture

This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.

42 ENGINEERING↗
DOE OSTI2022

Detecting Large Explosions With Machine Learning Models Trained on Synthetic Infrasound Data

Explosions produce low-frequency acoustic (infrasound) waves capable of propagating globally, but the spatio-temporal variability of the atmosphere makes detecting events difficult. Machine learning (ML) is well-suited to identify the subtle and nonlinear patterns in explosion infrasound signals, but a previous lack of ground-truth data inhibited training of generalized models. We introduce a physics-based method that propagates infrasound sources through realistic atmospheres to create 28,000 synthetic events, which are used to train ML classifiers. A simple artificial neural network and modern temporal convolutional network discriminate synthetic events from background noise with >90% accuracy and, more importantly, successfully identify the majority of real-world explosion signals recorded during the Humming Road Runner experiment. ML models trained entirely on physics-based synthetics advance explosion detection capabilities and make ML more viable to related fields lacking training data.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

A Novel Method to Train Classification Models for Structure Detection in In Situ Spacecraft Data

We present a method for creating spacecraft-like data which can be used to train Machine Learning (ML) models to detect and classify structures in in situ spacecraft data. First, we use the Grad-Shafranov equation to numerically solve for several magnetohydrostatic equilibria which are variations on a known analytic equilibrium. These equilibria are then used as the initial conditions for Particle-In-Cell simulations in which the structures of interest are observed and labeled. We then take one-dimensional slices through the simulations to replicate what a spacecraft collecting data from the simulation would observe. This sliced data then can be used as training data for the initial training of ML models intended for use on spacecraft data. We demonstrate the method applied to the problem of detecting small-scale plasmoids in the magnetotail, which is important for understanding complex magnetotail reconnection dynamics. The simple 1D classifier we train is able to detect more than 70% of the plasmoid points in the data set but also produces a large number of false positives. Our further work on this example problem is detailed, and further potential uses of the method are discussed.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Temporal dynamics of the multi-omic response to endurance exercise training

Regular exercise promotes whole-body health and prevents disease, but the underlying molecular mechanisms are incompletely understood. Here, the Molecular Transducers of Physical Activity Consortium profiled the temporal transcriptome, proteome, metabolome, lipidome, phosphoproteome, acetylproteome, ubiquitylproteome, epigenome and immunome in whole blood, plasma and 18 solid tissues in male and female Rattus norvegicus over eight weeks of endurance exercise training. The resulting data compendium encompasses 9,466 assays across 19 tissues, 25 molecular platforms and 4 training time points. Thousands of shared and tissue-specific molecular alterations were identified, with sex differences found in multiple tissues. Temporal multi-omic and multi-tissue analyses revealed expansive biological insights into the adaptive responses to endurance training, including widespread regulation of immune, metabolic, stress response and mitochondrial pathways. Many changes were relevant to human health, including non-alcoholic fatty liver disease, inflammatory bowel disease, cardiovascular health and tissue injury and recovery. The data and analyses presented in this study will serve as valuable resources for understanding and exploring the multi-tissue molecular effects of endurance training and are provided in a public repository (https://motrpac-data.org/).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (β) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Teacher-student training improves the accuracy and efficiency of machine learning interatomic potentials

Machine learning interatomic potentials (MLIPs) are revolutionizing the field of molecular dynamics (MD) simulations. Recent MLIPs have tended towards more complex architectures trained on larger datasets. The resulting increase in computational and memory costs may prohibit the application of these MLIPs to perform large-scale MD simulations. Herein, we present a teacher-student training framework in which the latent knowledge from the teacher (atomic energies) is used to augment the students' training. We show that the light-weight student MLIPs have faster MD speeds at a fraction of the memory footprint compared to the teacher models. Remarkably, the student models can even surpass the accuracy of the teachers, even though both are trained on the same quantum chemistry dataset. Our work highlights a practical method for MLIPs to reduce the resources required for large-scale MD simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2015

Temperature dependent exchange bias training effect in single-crystalline BiFeO{sub 3}/Co bilayers

Single-crystalline BiFeO{sub 3} (BFO)/Co bilayers were prepared by combined pulsed laser deposition and magnetron sputtering on (001) SrTiO{sub 3} substrates. Exchange bias (EB) and accompanying training effect have been studied as a function of temperature (T) between 5 K and 300 K. A non-monotonic exchange field variation with sharp increase below 100 K has been observed. In the meanwhile, strong training effect was recorded when T < 100 K and it weakens monotonically with increasing T up to 300 K. These temperature dependent EB and training effect may be caused by the uncompensated spins in both the interfacial spin-glass (SG) phase at low temperature and the antiferromagnetic BFO layer at higher temperature. The low temperature EB training results can be well fitted by a modified Binek's model considering asymmetric changes of the pinning SG spins at the descending and the ascending branches.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2016

Exchange bias training relaxation in spin glass/ferromagnet bilayers

A canonical spin glass (SG) FeAu layer is fabricated to couple to a soft ferromagnet (FM) FeNi layer. Below the SG freezing temperature, exchange bias (EB) and training are observed. Training in SG/FM bilayers is insensitive to cooling field and may suppress the EB or change the sign of the EB field from negative to positive at specific temperatures, violating from the simple power-law or the single exponential function derived from the antiferromagnet based systems. In view of the SG nature, we employ a double decay model to distinguish the contributions from the SG bulk and the SG/FM interface to training. Dynamical properties during training under different cooling fields and at different temperatures are discussed, and the nonzero shifting coefficient in the time index as a signature of slowing-down decay for SG based systems is interpreted by means of a modified Monte Carlo Metropolis algorithm.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

An Entropy-Maximization Approach to Automated Training Set Generation for Interatomic Potentials

Machine learning-based interatomic potentials are currently garnering a lot of attention as they strive to achieve the accuracy of electronic structure methods at the computational cost of empirical potentials. Given their generic functional forms, the transferability of these potentials is highly dependent on the quality of the training set, the generation of which can be highly labor-intensive. Good training sets should at once contain a very diverse set of configurations while avoiding redundancies that incur cost without providing benefits. We formalize these requirements in a local entropy-maximization framework and propose an automated sampling scheme to sample from this objective function. We show that this approach generates much more diverse training sets than unbiased sampling and is competitive with hand-crafted training sets.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2014

Ti:sapphire/KrF hybrid laser system generating trains of subterawatt subpicosecond UV pulses

The GARPUN-MTW Ti:sapphire/KrF hybrid laser system is used to investigate different multipass schemes for amplifying trains of ultrashort pulses (USPs) of subpicosecond duration. It is shown that, for an USP repetition period of 3 – 5 ns, which exceeds the gain-medium recovery time (∼2 ns), trains are amplified in the same way as single USPs. Due to this, a train can efficiently extract pump energy from the amplifier and sum energies of individual USPs. The energy of a four-USP train, extracted during four passes through the preamplifier and two passes through the final KrF amplifier (4 + 2 scheme), is saturated at a level of 1.6 J and corresponds to maximum USP peak powers of about 0.6 TW. The energy of amplified spontaneous emission (ASE), on the contrary, rapidly increases at a large total gain length L{sub eff} ≈ 6 m and is approximately equal to the USP energy. In the (4 + 1) and (2 + 2) schemes the USP energy decreases only slightly, to E{sub out} = 1.3 and 1.2 J, and the ASE fraction is reduced to about 10% and 3%, respectively. USP self-focusing leads to multiple laser beam filamentation and a 200-fold local increase in the radiation intensity in filaments, to ∼2 × 10{sup 11} W cm{sup -2}, a level at which the nonlinear loss in the output CaF{sub 2} windows of the KrF amplifier, caused by three-photon absorption, nonlinear scattering, and broadening of the radiation spectrum to a value exceeding the gain band of the KrF laser transition, becomes the main factor determining the saturation of the USP output energy. (lasers)

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2015

PROPAGATION AND DISPERSION OF SAUSAGE WAVE TRAINS IN MAGNETIC FLUX TUBES

A localized perturbation of a magnetic flux tube produces wave trains that disperse as they propagate along the tube, where the extent of dispersion depends on the physical properties of the magnetic structure, on the length of the initial excitation, and on its nature (e.g., transverse or axisymmetric). In Oliver et al. we considered a transverse initial perturbation, whereas the temporal evolution of an axisymmetric one is examined here. In both papers we use a method based on Fourier integrals to solve the initial value problem. We find that the propagating wave train undergoes stronger attenuation for longer axisymmetric (or shorter transverse) perturbations, while the internal to external density ratio has a smaller effect on the attenuation. Moreover, for parameter values typical of coronal loops axisymmetric (transverse) wave trains travel at a speed 0.75–1 (1.2) times the Alfvén speed of the magnetic tube. In both cases, the wave train passage at a fixed position of the magnetic tube gives rise to oscillations with periods of the order of seconds, with axisymmetric disturbances causing more oscillations than transverse ones. To test the detectability of propagating transverse or axisymmetric wave packets in magnetic tubes of the solar atmosphere (e.g., coronal loops, spicules, or prominence threads) a forward modeling of the perturbations must be carried out.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Empirically-calibrated H100 node power models for accurate AI training energy estimation

Accurately quantifying the energy use of artificial intelligence (AI) training is critical for infrastructure planning, carbon accounting, and sustainable data center operation, but few studies have directly measured the power consumption of production workloads on contemporary hardware. By combining empirical measurements from Brookhaven National Laboratory during AI training on 8-graphics-processing-unit H100 systems with open-source benchmarking data, we develop statistical models relating computational intensity to node-level power consumption. We measure the gap between manufacturer-rated thermal design power (TDP) and actual power demand during AI training. Our analysis reveals that even computationally intensive workloads operate at only 76% of the 10.2 kW TDP rating. Our architecture-specific model, calibrated to floating-point operations, predicts energy consumption with 11.4% mean absolute percentage error, significantly outperforming TDP-based approaches (27%–37% error). We identified distinct power signatures between transformer and convolutional neural network architectures, with transformers showing characteristic fluctuations that may impact grid stability. These results provide a measurement-grounded basis for improving AI training energy estimates, enabling more reliable infrastructure sizing, cost projections, and environmental impact assessments.

Newkirk, Alex C↗
DOE OSTI2021

Optimizing a magnitude-limited spectroscopic training sample for photometric classification of supernovae

ABSTRACT In preparation for photometric classification of transients from the Legacy Survey of Space and Time (LSST) we run tests with different training data sets. Using estimates of the depth to which the 4-m Multi-Object Spectroscopic Telescope (4MOST) Time Domain Extragalactic Survey (TiDES) can classify transients, we simulate a magnitude-limited sample reaching rAB ≈ 22.5 mag. We run our simulations with the software snmachine, a photometric classification pipeline using machine learning. The machine-learning algorithms struggle to classify supernovae when the training sample is magnitude limited, in contrast to representative training samples. Classification performance noticeably improves when we combine the magnitude-limited training sample with a simulated realistic sample of faint high-redshift supernovae observed from larger spectroscopic facilities; the algorithms’ range of average area under receiver operator characteristic curve (AUC) scores over 10 runs increases from 0.547–0.628 to 0.946–0.969 and purity of the classified sample reaches 95 per cent in all runs for two of the four algorithms. By creating new, artificial light curves using the augmentation software avocado, we achieve a purity in our classified sample of 95 per cent in all 10 runs performed for all machine-learning algorithms considered. We also reach a highest average AUC score of 0.986 with the artificial neural network algorithm. Having ‘true’ faint supernovae to complement our magnitude-limited sample is a crucial requirement in optimization of a 4MOST spectroscopic sample. However, our results are a proof of concept that augmentation is also necessary to achieve the best classification results.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Generation of shock trains in free liquid jets with a nanosecond green laser

Shock wave trains in liquid jets were previously generated only by ablation with femtosecond x-ray lasers. Here we show that shock trains in water microjets can be also generated using nanosecond green laser pulses with 1- to 10-mJ energy. Furthermore, the ablation of 15-, 20-, 30-, and 70-μm water microjets opened a gap in the jets and launched an initial shock wave. Fully developed shock trains were observed in the 30- and 70-μm jets up to 250-ns delays, and these trains were also transmitted inside the nozzles. A few tens of nanoseconds after the pulse, the shock dynamics and its pressure became similar to the ones generated by x-ray lasers, with a more rapid pressure decay in thinner jets. At time delays exceeding 100 ns in the 30-μm jets, the leading shock pressure stabilized to an approximately constant pressure of 40 MPa. The energy density deposited in the jets was estimated at 30 MJ/cm 3 by comparing the jet gaps in the green and x-ray laser experiments, and matched previous estimates for optical ablation in water. Here, the pressure decay in the 30-μm jets was modeled based on the pressure decay observed in x-ray laser experiments.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2025

Microstructural and rheological training and memory of nanocolloidal soft glasses under cyclic shear

An intrinsic feature of disordered and out-of-equilibrium materials, such as glasses, is the dependence of their properties on their history. An important example is rheological memory, in which disordered solids obtain properties based on their deformation history. Here, in this study, we employ x-ray photon correlation spectroscopy with in situ rheometry to characterize memory formation in a nanocolloidal soft glass due to cyclic shear. During a cycle, particles undergo irreversible displacements composed of a combination of shear-induced diffusion and heterogeneous, residual strain fields. At lower shear amplitudes, the displacements resemble a random walk in which the directions in each cycle are independent of those in preceding cycles, while at high amplitude, the irreversible displacements in consecutive cycles become correlated. The magnitudes of the displacements decrease with each cycle before reaching a steady state where the microstructure has been trained to achieve enhanced reversibility even at shear amplitudes well above yielding and despite the presence of thermal fluctuations. At amplitudes below and near yielding, these decreases are monotonic, while well above yielding, they are nonmonotonic, suggesting evidence of shear banding. Accompanying this microstructural training are corresponding decreases in the dissipation during each cycle and the magnitude of the residual stress toward steady-state values. Memory of the training is revealed by measurements in which the amplitude of the shear is changed after steady state is reached. The magnitude of the particle displacements, as well as the dissipation and the change in residual stress, vary nonmonotonically with the new shear amplitude, having minima near the training amplitude, thereby revealing correlated microscopic and macroscopic signatures of memory.

Chen, Yihao [Johns Hopkins Univ., Baltimore, MD (U↗
DOE OSTI2019

Single-Net Continual Learning with Progressive Segmented Training

There is an increasing need of continual learning in dynamic systems, such as the self-driving vehicle, the surveillance drone, and the robotic system. Such a system requires learning from the data stream, training the model to preserve previous information and adapt to a new task, and generating a single-headed vector for future inference. Different from previous approaches with dynamic structures, this work focuses on a single network and model segmentation to prevent catastrophic forgetting. Leveraging the redundant capacity of a single network, model parameters for each task are separated into two groups: one important group which is frozen to preserve current knowledge, and secondary group to be saved (not pruned) for a future learning. A fixed-size memory containing a small amount of previously seen data is further adopted to assist the training. Without additional regularization, the simple yet effective approach of Progressive Segmented Training (PST) successfully incorporates multiple tasks and achieves the state-of-the-art accuracy in the single-head evaluation on CIFAR-10 and CIFAR-100 datasets. Moreover, the segmented training significantly improves computation efficiency in continual learning at the edge.

Du, Xiaocong↗