Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

AADL: Anderson Accelerated Deep Learning

We propose a stable, distributed approach to perform AA that accelerates the convergence rate of stochastic first-order optimizers to train neural networks. Differently from previous works, we do not alter neither the scheme to perform AA nor the loss function minimized during the training. To improve robustness against stagnation, we customize general guidelines that suggest to relax the frequency of AA corrections by performing AA only at the end of an entire training epoch. To improve robustness of AA against the stochastic oscillations of first-order optimizers, we average the gradients computed on consecutive stochastic optimization updates. The improved regularity of the converging sequence and the reduced amplitude of stochastic oscillations across consecutive optimization steps allows AA to efficiently extrapolate an improved converging sequence, thereby overcoming limitations of existing approaches to perform AA on stochastic optimization.

Lupo Pasini, Massimiliano [Oak Ridge National Lab.↗

Stochastic Ensemble Generation for Improved Characterization of Representing Geologic Variability in a Reservoir: IBDP Case Study for SMART Initiative

This document is a poster covering the findings from activities on training data generation, specifically geologic ensemble generation. The generated geologic realizations captured the range of possible permeability distributions of the subsurface at the Illinois Basin - Decatur Project (IBDP) site, based on available well log variabilities. The percentages of reservoirs and baffles in the injection zone and a truncation of baffle permeability led to more variance in the simulations. This will be used to build forward modeling, history matching, and optimization workflows. The geologic realizations were also ranked according to dynamic measures of hydraulic diffusivity, and simulations confirm a greater contrast between the reservoir and the baffles during injection.

stochastic ensemble generation↗

Towards Scaling Law Analysis For Spatiotemporal Weather Data

Compute-optimal scaling laws are relatively well studied for NLP and CV, where objectives are typically single-step and targets are comparatively homogeneous. Weather forecasting is harder to characterize in the same framework: autoregressive rollouts compound errors over long horizons, outputs couple many physical channels with disparate scales and predictability, and globally pooled test metrics can disagree sharply with per-channel, late-lead behavior implied by short-horizon training. We extend neural scaling analysis for autoregressive weather forecasting from single-step training loss to long rollouts and per-channel metrics. We quantify (1) how prediction error is distributed across channels and how its growth rate evolves with forecast horizon, (2) if power law scaling holds for test error, relative to rollout length when error is pooled globally, and (3) how that fit varies jointly with horizon and channel for parameter, data, and compute-based scaling axes. We find strong cross-channel and cross-horizon heterogeneity: pooled scaling can look favorable while many channels degrade at late leads. We discuss implications for weighted objectives, horizon-aware curricula, and resource allocation across outputs.

Kiefer Jr, Alexander [ORNL] (ORCID:000000025398874↗

Predicting Rare Earth Element Potential in Produced and Geothermal Waters of the United States via Emergent Self-Organizing Maps

This work applies emergent self-organizing map (ESOM) techniques, a form of machine learning, in the multidimensional interpretation and prediction of rare earth element (REE) abundance in produced and geothermal waters in the United States. Visualization of the variables in the ESOM trained using the input data shows that each REE, with the exception of Eu, follows the same distribution patterns and that no single parameter appears to control their distribution. Cross-validation, using a random subsample of the starting data and only using major ions, shows that predictions are generally accurate to within an order of magnitude. Using the same approach, an abridged version of the U.S. Geological Survey Produced Waters Database, Version 2.3 (which includes both data from produced and geothermal waters) was mapped to the ESOM and predicted values were generated for samples that contained enough variables to be effectively mapped. Results show that in general, produced and geothermal waters are predicted to be enriched in REEs by an order of magnitude or more relative to seawater, with maximum predicted enrichments in excess of 1000-fold. Cartographic mapping of the resulting predictions indicates that maximum REE concentrations exceed values in seawater across the majority of geologic basins investigated and that REEs are typically spatially co-associated. The factors causing this co-association were not determined from ESOM analysis, but based on the information currently available, REE content in produced and geothermal waters is not directly controlled by lithology, reservoir temperature, or salinity.

Engle, Mark A. (ORCID:0000000152587374)↗

Brief communication: Monitoring snow depth using small, cheap, and easy-to-deploy snow–ground interface temperature sensors

Abstract. Temporally continuous snow depth estimates are vital for understanding changing snow patterns and impacts on permafrost in the Arctic. We trained a random forest machine learning model to predict snow depth from variability in snow–ground interface temperature. The model performed well on Alaska's Seward Peninsula where it was trained and at Arctic evaluation sites (RMSE ≤ 0.15 m). It performed poorly at temperate sites with deeper snowpacks, partially due to training data limitations. Small temperature sensors are cheap and easy to deploy, so this technique enables spatially distributed and temporally continuous snowpack monitoring at high latitudes to an extent previously infeasible.

54 ENVIRONMENTAL SCIENCES↗

Information systems and services, user services

The following topics were discussed: (1) data availability and distribution, (2) complete processing systems, (3) subsystems, (4) applications, (5) research for future technology, and (6) education, training opportunities, and materials. Evidence was given that remote sensing technology is being increasingly utilized. Therefore, it was concluded that a second stage of remote sensing technology should be developed.

Landgrebe, D. A.↗

Measurements of pressures on the wing of an aircraft model during steady rotation

An investigation has been conducted in the Spin Tunnel Facility at the NASA Langley Research Center to measure the pressures on the wing surfaces of a model of a Basic Training Aircraft during steady rotation. The tests were made to determine the nature of the wing pressure distribution during rotations typical of spin entry and steady spin. Comparisons are made between the forces and moments obtained from integrating the pressure field with those measured directly during rotary balance force tests. The results are also compared with estimates determined from a simple numerical model of the wing aerodynamic forces.

Martin, Colin A.↗

Surveillance system and method having an adaptive sequential probability fault detection test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Herzog, James P.↗

Surveillance system and method having an adaptive sequential probability fault detection test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Bickford, Randall L.↗

Surveillance System and Method having an Adaptive Sequential Probability Fault Detection Test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Bickford, Randall L.↗

NASA LaRC Contribution to the High Angle Working Group of the Third Aeroelastic Prediction Workshop: BSCW Shock Buffet

FUN3D Core Capabilities - Established as a research code in late 1980s; now supports numerous internal and external efforts across the speed range - Solves 2D/3D steady and unsteady Euler and RANS equations on node-based mixed element grids for compressible and incompressible flows - General dynamic mesh capability: any combination of rigid / overset / morphing grids, including 6-DOF effects - Aeroelastic modeling using mode shapes, full FEM, etc. - Constrained / multipoint adjoint-based design and mesh adaptation - Distributed development team using agile/extreme software practices including 24/7 regression, performance testing - Capabilities fully integrated, online documentation, training videos, tutorials

Pawel Chwalowski↗

Cyberguardians and STEM Warriors (Final Technical Report (FTR))

In the past decade, solar power, with and without energy storage has become the fastest growing source of energy generation in the world. In the U.S., solar employment more than doubled from 105,145 jobs in 2011 to 255,037 jobs in 2021, four times faster than the U.S. job growth rate overall. These factors, combined with technology advancements, creates a skills gap that puts tremendous stress on society to deliver the workers to fill the open job requisitions. The Cyberguardians and STEM Warriors project (Cyberguardians) was designed to address the trained-worker shortage in the energy industry in three ways: 1) by developing educational curriculum that addresses DER technology changes; 2) by delivering curriculum to prospective workers, including military veterans and their families, via universities, community colleges, and vocational training outlets; and 3) introducing individuals who have completed training to employers that can hire them. Cyberguardians exceeded its curriculum goals by producing 27 academic units of university-accredited material (12 total courses) covering energy fundamentals, smart inverters, Distributed Energy Resource (DER) data communication, cybersecurity, standardization, certification, data analytics, and IEEE 1547 standard topics. The North American Board of Certified Energy Practitioners (NABCEP) also accredited the material for use in their credential program. Seven instructors were recruited and trained, and six academic institutions (University of California San Diego, State University of New York, North Carolina State University, Harper Community College, Green Village Academy, and the SunSpec Alliance) were enlisted, meeting program goals. All course material was published under the Creative Commons license and made available royalty free, thus providing a long-lasting public benefit. The program’s outreach program vastly exceeded program goals and incorporated the efforts of 13 outreach partners (11 of which are veteran focused), an advisory board representing 15 companies, webinars and 10’s of thousands of email messages sent to prospective students and hiring managers. Despite these efforts, the global pandemic depressed anticipated program participation by about a third. Still, a total of 396 students enrolled and 289 completed the courses and were accredited. The job applicant task achieved similar results (111 realized vs a 174 goal) but reported job placement was weaker at (9 realized vs. a 51 goal). The Cyberguardians program fills a critical void for cost-effective, royalty-free curriculum and training pertaining to DER technologies and cybersecurity that prospective energy workers must possess to be effective in the 21 st century. On this basis alone, the investment of taxpayer funds will pay dividends for years to come.

14 SOLAR ENERGY↗

Adaptive language model training for molecular design

Abstract The vast size of chemical space necessitates computational approaches to automate and accelerate the design of molecular sequences to guide experimental efforts for drug discovery. Genetic algorithms provide a useful framework to incrementally generate molecules by applying mutations to known chemical structures. Recently, masked language models have been applied to automate the mutation process by leveraging large compound libraries to learn commonly occurring chemical sequences (i.e., using tokenization) and predict rearrangements (i.e., using mask prediction). Here, we consider how language models can be adapted to improve molecule generation for different optimization tasks. We use two different generation strategies for comparison, fixed and adaptive. The fixed strategy uses a pre-trained model to generate mutations; the adaptive strategy trains the language model on each new generation of molecules selected for target properties during optimization. Our results show that the adaptive strategy allows the language model to more closely fit the distribution of molecules in the population. Therefore, for enhanced fitness optimization, we suggest the use of the fixed strategy during an initial phase followed by the use of the adaptive strategy. We demonstrate the impact of adaptive training by searching for molecules that optimize both heuristic metrics, drug-likeness and synthesizability, as well as predicted protein binding affinity from a surrogate model. Our results show that the adaptive strategy provides a significant improvement in fitness optimization compared to the fixed pre-trained model, empowering the application of language models to molecular design tasks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Distributed Quantum Learning with co-Management in a Multi-tenant Quantum System

The rapid advancement of quantum computing has pushed classical designs into the quantum domain, breaking physical boundaries for computing-intensive and data-hungry applications with the hope that some systems may provide a quantum speedup. For example, variational quantum algorithms have been proposed for quantum neural networks to train deep learning models on qubits, achieving promising results. Existing quantum learning architectures and systems rely on single, monolithic quantum machines with abundant and stable resources, such as qubits. However, fabricating a large, monolithic quantum device is considerably more challenging than producing an array of smaller devices. In this paper, we investigate a distributed quantum system that combines multiple quantum machines into a unified system. We propose DQuLearn, which divides a quantum learning task into multiple subtasks. Each subtask can be executed distributively on individual quantum machines, with the results looping back to classical machines for subsequent training iterations. Additionally, our system supports multiple concurrent clients and dynamically manages their circuits according to the runtime status of quantum workers. Through extensive experiments, we demonstrate that DQuLearn achieves similar accuracies with significant runtime reduction, by up to 68.7% and an increase per-second circuit processing speed, by up to 3.99 times, in a 4-worker multi-tenant setting.

quantum computing↗

Fault-Tolerant Deep Learning Cache with Hash Ring for Load Balancing in HPC Systems

Large-scale DL on HPC systems like Frontier and Summit uses distributed node-local caching to address scalability and performance challenges. However, as these systems grow more complex, the risk of node failures increases, and current caching approaches lack fault tolerance, jeopardizing large-scale training jobs. We analyzed six months of SLURM job logs from Frontier and found that over 30% of jobs failed after an average of 75 minutes. To address this, we propose fault-tolerance strategies that recache data lost from failed nodes using a hash ring technique for balanced data recaching in the distributed node-local caching, reducing reliance on the PFS. Our extensive evaluations on Frontier showed that the hash ring-based recaching approach reduced training time by approximately 25% compared to the approach that redirects I/O to the PFS after node failures and demonstrated effective load balancing of training data across nodes.

Lee, Seoyeong↗

Automated Classification of Plasma Regions Using 3D Particle Energy Distributions

We investigate the properties of the ion sky maps produced by the Dual Ion Spectrometers (DIS) from the Fast Plasma Investigation (FPI). We have trained a convolutional neural network classifier to predict four regions crossed by the Magnetospheric Multiscale Mission (MMS) on the dayside magnetosphere: solar wind, ion foreshock, magnetosheath, and magnetopause using solely DIS spectrograms. The accuracy of the classifier is urn:x-wiley:21699380:media:jgra56742:jgra56742-math-0001%. We use the classifier to detect mixed plasma regions, in particular to find the bow shock regions. A similar approach can be used to identify the magnetopause crossings and reveal regions prone to magnetic reconnection. Data processing through the trained classifier is fast and efficient and thus can be used for classification for the whole MMS database.

79 ASTRONOMY AND ASTROPHYSICS↗

Demonstrate new plasticity models for doped UO 2 that capture dislocation mechanisms

In light water reactors, fuel vendors are investigating the use of dopants to modify the properties of UO 2 pellets, with the goal of improving pellet-cladding mechanical interactions during operation. Dopants are expected to ‘soften’ the pellets; that is, the doped pellets have higher plastic deformation than conventional UO 2 . This leads to a reduction in the severity of mechanical pellet-cladding interactions, helping to reduce the hoop strain on the cladding. By minimizing the strain exerted by the pellet on the cladding, it is anticipated that cladding performance under accident conditions can be enhanced (i.e., lowering the risk of burst during a LOCA). Dopants such as chromium (Cr) promote grain growth during pellet fabrication, leading to larger grains; therefore, understanding the link between chemistry, microstructure and mechanical deformation (enhanced creep rates) behavior of UO 2 is critical to helping operators further substantiate the benefits of doping UO 2 . Historically, the nuclear energy industry has relied on empirical models to make assessments of performance. Compared to empirical models, mechanistic physics-based models provide benefits, such as, fewer data points for validation and better extrapolation where experimental data is scarce or non-existent. In this report, Bayesian inference techniques have been applied to a previously developed lower length-scale-informed diffusional creep model. The objective is to i) infer lower-length-scale parameter distributions from available experiment and then ii) determine the uncertainties in the measurable quantity (in this case creep rates) after propagating the inferred lower length scale parameter uncertainties. The approach requires many evaluations of the model, which becomes computationally insurmountable; therefore, a neural-network model is trained to data obtained by sampling the full model over the most important parameters. This neural-network is then used in the Bayesian inference approach to determine probability distributions in the parameter values that represent the uncertainty in the model given what is known from the experiments (posterior). A significant reduction compared to conservative initial (prior) uncertainties is achieved through inference against the experimental data, demonstrating the efficacy of this approach. Furthermore, by accounting for uncertainties in the experimental conditions and sample non-stoichiometry, it is possible to resolve apparent discrepancies in experimental measurements within a self-consistent grain boundary (Coble) creep model that is sensitive to chemistry. This work has been written up and submitted to Nuclear Technology for a special issue on accelerated fuel qualification (AFQ). This uncertainty quantification (UQ) work not only improves the diffusional model, while accounting for uncertainty, but also establishes a framework which can readily be applied to the mechanistic models of dislocation deformation developed in this study. The most likely values from the Bayesian analysis are incorporated into our UO 2 diffusional creep model and a lower length scale-informed irradiation UO 2 creep mechanistic model to generate a dataset. This dataset has been provided to our INL collaborators for training an artificial neural network surrogate model, which will be implemented in the BISON fuel performance code to assess how the results differ from those currently obtained using a fully empirical model and that of using the nominal (uncalibrated) atomic scale parameters in our mechanistic model. Plastic deformation (creep and glide) in UO 2 is a complex phenomenon, governed by multiple underlying processes such as local defect concentrations, applied stresses, and microstructural characteristics. Consequently, there is a need for a meso-scale model with polycrystalline resolution capable of extrapolating to large grain sizes applicable to doped UO 2 , where data is limited and the model can help bridge the knowledge gap. By integrating atomistic data into the polycrystal LApx code, it becomes possible to predict dislocation climb and glide plasticity that simple analytical models cannot accurately represent. The application of atomic-scale data within LApx demonstrated the importance of climb and glide mechanisms in reproducing high-stress UO 2 behavior. Behaviors such as this are crucial to capture and implement in BISON, as parts of the fuel pellet can reach temperatures where glide can occur before pellet cracking. This model which captures dislocation based mechanisms for UO 2 is then used to stand up the doped model accounting for larger grain sizes. It was found that larger grain sizes can lead to enhanced deformation rates in the glide regime, and therefore can help with the pellet cladding mechanical interaction. Therefore if the fuel pellet reaches conditions (stress/temperature) where glide is active, the enhanced creep rates for larger grains in the glide regime (doped UO 2 ) can help with pellet cladding mechanical interactions. Plastic deformation in UO 2 involves multiple mechanisms, including diffusional creep, dislocation climb, and glide. This milestone contains two parts: (1) UQ of a pre-existing lower length scale informed mechanistic diffusional creep model, and (2) development of a new LApx based model for dislocation-mediated creep mechanisms in UO 2 , with application to large-grain doped UO 2 .

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Debiasing with Diffusion: Probabilistic Reconstruction of Dark Matter Fields from Galaxies with CAMELS

Abstract Galaxies are biased tracers of the underlying cosmic web, which is dominated by dark matter (DM) components that cannot be directly observed. Galaxy formation simulations can be used to study the relationship between DM density fields and galaxy distributions. However, this relationship can be sensitive to assumptions in cosmology and astrophysical processes embedded in galaxy formation models, which remain uncertain in many aspects. In this work, we develop a diffusion generative model to reconstruct DM fields from galaxies. The diffusion model is trained on the CAMELS simulation suite that contains thousands of state-of-the-art galaxy formation simulations with varying cosmological parameters and subgrid astrophysics. We demonstrate that the diffusion model can predict the unbiased posterior distribution of the underlying DM fields from the given stellar density fields while being able to marginalize over uncertainties in cosmological and astrophysical models. Interestingly, the model generalizes to simulation volumes ≈500 times larger than those it was trained on and across different galaxy formation models. The code for reproducing these results can be found athttps://github.com/victoriaono/variational-diffusion-cdm✎.

Astronomy & Astrophysics↗