SEARCH · Engineering Papers

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

All sources NASA NTRS DOE OSTI

DOE OSTI2022

SMALE: Enhancing Scalability of Machine Learning Algorithms on Extreme-Scale Computing Platforms

Deployment and execution of machine learning tasks on extreme-scale computing platforms face several significant technical challenges: 1) High computing cost incurred by dense networks – The computing workload of deep networks with densely-connected topology increases rapidly with the network size, imposing a non-scalable computing model of extreme-scale computing platforms; 2) Non-optimized workload distribution – Many advanced deep learning algorithms, e.g., sparsification and irregular net-work topology, produce very unbalanced workload distribution on extreme-scale computing platforms. The computation efficiency is greatly hindered by the incurred data and computation redundancies as well as long tails of the node with extensive workload; 3) Constraints in data movement and I/O bottle-neck – Inter-node data movement in extreme-scale computing platforms are associated with high energy and latency costs, and subject to the constraints of I/O bandwidth; and 4) Generalization of algorithm realization and acceleration on computing platforms – The large varieties of machine learning algorithms and structures of extreme-scale computing platforms make the derivation of a generalized algorithm realization and acceleration method very challenging, which, however, is the requirement by domain scientists and interested users. We call the above challenges Smale’s Problems in Machine Learning and Understanding for High-Performance Computing Scientific Discovery. The objective of our three-year research project is to develop a holistic innovation set at structure, assembly, and acceleration layers of machine learning algorithms to address the above challenges in algorithm deployment and execution. Three tasks are particularly performed, including: At the algorithm structure level, we investigate the techniques that can structurally sparsify on the topology of deep networks for computing workload reduction. We also study clustering and pruning techniques that can optimize the workload distributions over the extreme-scale computing platforms; At the algorithm assembly level, we derive a unified learning framework for unsupervised transfer learning and dynamic growing capabilities. Novel training methods are also exploited to enhance the training efficiency of the proposed framework; At the algorithm acceleration level, we will develop a series of techniques that can accelerate the computation of sparse matrix operations, which are one of the core executions in deep learning and optimize memory access of the concerned platforms. Our proposed techniques attack the fundamental problems in machine learning algorithms running on extreme-scale computing platforms by vertically integrating the solutions at three closely entangled layers, paving the long-term scaling path of machine learning applications under DOE context. Three tasks corresponding to the above respective research orientations are performed during the three-year project period with our collaborators at ORNL. The outcome of the proposed project is anticipated to form a holistic solution set of novel algorithms and network topologies, efficient training techniques, and fast acceleration methods to promote the computing scalability of the machine learning applications of particular interest to DOE.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Representing Complex Systems as Graphs for Debugging and Predictive Maintenance-Preliminary Thoughts

Representing complex systems as graphs enables use of mathematical tools to identify faults or predict failures. Graph nodes correspond to individual modules or subsystems, and edges link coupled system parts. ‘Probes’ measure the node outputs, monitoring the system health for unexpected behavior. Assuming one cannot probe every point, within a system, the fault correlates to a region—not necessarily the specific location. Bayesian networks trained to understand fault patterns can accurately identify the source. The diagnostic tool described aides debugging by pinpointing system failure causes. For predictive maintenance, probe data develop probability distribution functions describing subsystem mean time to failure. Unit lifetime can be estimated through these probability distributions. Two approaches include using Bayesian classifiers to infer the system failure source and developing maintenance schedules by treating systems as collections of random variables. When failure behavior does not follow a closed form function, use of similarity models is proposed.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Raman Microscopy Analysis of Wyoming CarbonSAFE Pilot Well Thin Sections for Mineralogy and Organic Matter Characterization

Scanning confocal Raman microspectroscopy (RMS) was used to analyze thin sections for inorganic mineral content and for evaluation of organic material (OM). Thin sections were selected from a stratigraphic test well core in the Powder River Basin, Wyoming under the Wyoming CarbonSAFE project. The well was analyzed with a variety of rock and fluid characterization techniques to determine the feasibility of a commercial-scale CO2 storage site. RMS complements other analyses, including traditional petrography, SEM, porosity and permeability, and XRD For this aspect of the study, RMS is especially important in the evaluation of OM in sealing lithologies. At prospective geologic CO2 storage sites, it is imperative to assess the unconventional oil and gas potential of seals to ensure that any future development would not compromise the integrity of the seals. Whole-slide mineralogical surveys were performed on thin sections from various shale and sandstone formations. Surveys were analyzed with Direct Classical Least Squares to identify and quantify minerals. The location and concentration of minerals was color-coded and overlaid on optical images for visualization of the distribution of minerals. Dense hyperspectral Raman mapping of OM was performed on five thin sections. Eleven spectral parameters diagnostic of organic type and thermal maturity were used to train a Partial Least Squares (PLS) calibration against a set of artificially matured samples spanning the pre- to mid-oil window. The PLS was applied to the study set and a post-mature set. Additionally, the PLS was applied to each point in hyperspectral maps for visualization of trends in maturity across sample sets and discrimination of organic matter types within a given map. In inorganic surveys on thin sections, a total of 14 unique inorganic minerals were identified in Raman spectra including quartz, dolomite, calcite, hematite and anhydrite. Shale thin sections tended to be dominated by organic material. OM was often observed mixed with inorganic minerals. Sand- and mudstones were dominated by inorganic minerals. The PLS extrapolated the post-mature set to reflectances >1.2%. The study set ranged from very immature to postmature in the median of map fit-peak parameters. However, point maturity maps indicate that matrix OM in all study samples is immature and that discreet organic particles selected for mapping, which may be inertinites, bias medians towards more-mature. The work demonstrates the capabilities of RMS to perform both whole-slide mineralogy and OM analysis with applications to formation evaluation in oil & gas, carbon sequestration and mining. Here, analysis of sealing formations in the well indicates high levels of immature organic matter that would not be a viable target for future oil production that could compromise the CO2 storage site. The work brings together diverse disciplines from geology and petrography to analytical chemistry, big data and microscopy.

Myers, Grant↗

DOE OSTI2026

Resource-Adaptive Federated Text Generation with Differential Privacy

In cross-silo federated learning (FL), sensitive text datasets remain confined to local organizations due to privacy regulations, making repeated training for each downstream task both communication-intensive and privacy-demanding. A promising alternative is to generate differentially private (DP) synthetic datasets that approximate the global distribution and can be reused across tasks. However, pretrained large language models (LLMs) often fail under domain shift, and federated finetuning is hindered by computational heterogeneity: only resource-rich clients can update the model, while weaker clients are excluded, amplifying data skew and the adverse effects of DP noise. We propose a flexible participation framework that adapts to client capacities. Strong clients perform DP federated finetuning, while weak clients contribute through a lightweight DP voting mechanism that refines synthetic text. To ensure the synthetic data mirrors the global dataset, we apply control codes (e.g., labels, topics, metadata) that represent each client’s data proportions and constrain voting to semantically coherent subsets. This two-phase approach requires only a single round of communication for weak clients and integrates contributions from all participants. Experiments show that our framework improves distribution alignment and downstream robustness under DP and heterogeneity.

Wang, Jiayi [ORNL]↗

DOE OSTI2020

Semi-supervised Machine Learning Enables the Robust Detection of Multireference Character at Low Cost

Multireference (MR) diagnostics are common tools for identifying strongly correlated electronic structure that makes single-reference (SR) methods (e.g., density functional theory or DFT) insufficient for accurate property prediction. However, MR diagnostics typically require computationally demanding correlated wave function theory (WFT) calculations, and diagnostics often disagree or fail to predict MR effects on properties. To overcome these challenges, we introduce a semi-supervised machine learning (ML) approach with virtual adversarial training (VAT) of an MR classifier using 15 WFT and DFT MR diagnostics as inputs. In semi-supervised learning, only the most extreme SR or MR points are labeled, and the remaining point labels are learned. The resulting VAT model outperforms the alternatives, as quantified by the distinct property distributions of SR- and MR-classified molecules. Additionally, to reduce the cost of generating inputs to the VAT model, we leverage the VAT model’s robustness to noisy inputs by replacing WFT MR diagnostics with regression predictions in an MR decision engine workflow that preserves excellent performance. We demonstrate the transferability of our approach to larger molecules and those with distinct chemical composition from the training set. This MR decision engine demonstrates promise as a low-cost, high-accuracy approach to the automatic detection of strong correlation for predictive high-throughput screening.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Development and evaluation of a list mode neutron coincidence collar for spatial response measurements of fresh fuel assemblies

A traditional safeguards neutron coincidence counting system, the Neutron Coincidence Collar, has been modified to incorporate individual preamplifiers on each of its 18 3He tubes in active interrogation mode. When used with list mode data acquisition (LMDA) and analysis, a signal from each 3He tube can be recorded and analyzed to allow a spatial response measurement to be performed on an item based on count rate and 3He tube location within the system. The ultimate goal of this project is to demonstrate the capabilities of a list mode response matrix for the nondestructive assay of fresh nuclear fuel assemblies. To enable partial defect detection of fuel pin locations and absences within an assembly, the project aims to extend well-established correlated neutron analysis techniques on a preexisting Neutron Coincidence Collar by extracting a greater number of useful signatures from the system than are currently generated. LMDA, combined with the addition of multiple preamplifiers, facilitates this capability by increasing the number of simultaneous signals that can be measured; this allows an in-depth analysis of neutron coincidence events to determine a fissioning item’s location based on the measured doubles count rate in various channel logic coincidence combinations. All of this can be done from a single measurement pulse train in offline analysis, which is the major benefit provided by LMDA. Through various stages of development and testing, a Mirion Technologies model JCC-71 Neutron Coincidence Collar has been successfully retrofit with modern electronics designed in-house at Oak Ridge National Laboratory, matching preexisting JAB-01 electronics performance, while maintaining the original system footprint. This paper presents these various stages of development and experimental evaluation of the proof of concept system.

Moore, Angela S.↗

DOE OSTI2021

Permutationally Invariant Polynomial Expansions with Unrestricted Complexity

A general strategy is presented for constructing and validating permutationally invariant polynomial (PIP) expansions for chemical systems of any stoichiometry. Demonstrations are made for three categories of gas-phase dynamics and kinetics: collisional energy-transfer trajectories for predicting pressure-dependent kinetics, three-body collisions for describing transient van der Waals adducts relevant to atmospheric chemistry, and nonthermal reactivity via quasiclassical trajectories. In total, 30 systems are considered with up to 15 atoms and 39 degrees of freedom. Permutational invariance is enforced in PIP expansions with as many as 13 million terms and 13 permutationally distinct atom types by taking advantage of petascale computational resources. The quality of the PIP expansions is demonstrated through the systematic convergence of in-sample and out-of-sample errors with respect to both the number of training data and the order of the expansion, and these errors are shown to predict errors in the dynamics for both reactive and nonreactive applications. Here, the parallelized code distributed as part of this work enables the automation of PIP generation for complex systems with multiple channels and flexible user-defined symmetry constraints and for automatically removing unphysical unconnected terms from the basis set expansions, all of which are required for simulating complex reactive systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Thermal Experiments for Fractured Rock Characterization: Theoretical Analysis and Inverse Modeling

Abstract Field‐scale properties of fractured rocks play a crucial role in many subsurface applications, yet methodologies for identification of the statistical parameters of a discrete fracture network (DFN) are scarce. We present an inversion technique to infer two such parameters, fracture density and fractal dimension, from cross‐borehole thermal experiments data. It is based on a particle‐based heat‐transfer model, whose evaluation is accelerated with a deep neural network (DNN) surrogate that is integrated into a grid search. The DNN is trained on a small number of the heat‐transfer model runs and predicts the cumulative density function of the thermal field. The latter is used to compute fine posterior distributions of the (to be estimated) parameters. Our synthetic experiments reveal that fracture density is well constrained by data, while fractal dimension is harder to determine. Adding nonuniform prior information related to the DFN connectivity improves the inference of this parameter.

Zhou, Zitong↗

DOE OSTI2024

Innovative dead-time correction and background subtraction for neutron multiplicity measurements using neural networks

Abstract The number of neutrons emitted from a nuclear reaction plays a crucial role in various fields, including nuclear theory, nuclear nonproliferation, nuclear energy and nuclear criticality safety. Accurate determination of neutron multiplicities requires the application of several corrections, with dead-time correction and background subtraction being particularly significant. These corrections become more challenging for neutron detectors with time-dependent neutron capture. In this work, we perform a comprehensive study of three existing methods used for dead-time correction and background subtraction in neutron detectors with time-dependent neutron capture. The methods were tested for dead-times in the range from 0 to 1 μs using a Monte Carlo model simulating the dead-time and background effects in the standard neutron multiplicity probability distribution of $$^{252}$$ 252 Cf. The previous methods showed larger than desired uncertainty or systematic trade off. Those uncertainties prompted the development of a novel approach using neural networks trained with data from Monte Carlo simulations. The Neural Network method enabled the correction of neutron multiplicity probabilities more accurately than the other methods with fractional errors smaller than 3% for multiplicities around the peak of $$^{252}$$ 252 Cf. A similar approach using neural networks could be applied to problems where the system being studied can be accurately simulated without having an accurate analytical description available. The neural network method presented in this paper can be easily expanded if multiplicities greater than 10 are expected.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2026

Boosted decision tree reweighting of simulated neutrino interactions for O ( 1 ) GeV neutrino cross-section measurements

This paper illustrates a generic method for multidimensional reweighting of O ( 1 ) GeV neutrino interaction Monte Carlo samples. The reweighting is based on a boosted decision tree algorithm trained on high-dimensional space in detector final-state observables. This enables one generator’s events to be reweighted so that its reconstructed particle content and kinematics distributions, as well as detector efficiency, match those of a target model. The approach establishes an efficient way to reuse legacy Monte Carlo data, avoiding regeneration. As an example, we test its use in a measurement of transverse kinematic imbalance of the μ - and proton in charged-current quasielastic like ν μ events from the MINERvA experiment.

Lin, Z. [Rochester U.] (ORCID:0009000188903698)↗

DOE OSTI2020

A self-supervised workflow for particle picking in cryo-EM

High-resolution single-particle cryo-EM data analysis relies on accurate particle picking. To facilitate the particle picking process, a self-supervised workflow has been developed. This includes an iterative strategy, which uses a 2D class average to improve training particles, and a progressively improved convolutional neural network for particle picking. To automate the selection of particles, a threshold is defined (%/Res) using the ratio of percentage class distribution and resolution as a cutoff. This workflow has been tested using six publicly available data sets with different particle sizes and shapes, and can automatically pick particles with minimal user input. The picked particles support high-resolution reconstructions at 3.0 Å or better. This workflow is a step towards automated single-particle cryo-EM data analysis at the stage of particle picking. It may be used in conjunction with commonly used single-particle analysis packages such as Relion , cryoSPARC , cisTEM , SPHIRE and EMAN2 .

2D class averages↗

DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗

DOE OSTI2024

PV-Finder: ML Based Algorithm for Primary Vertex Identification

he CMS detector at the High-Luminosity Large Hadron Collider (HL-LHC) will operate in challenging conditions with expected pile-up of up to 200 collisions per bunch crossing, necessitating the development of a more resilient primary vertex (PV) reconstruction method to ensure the integrity of data analysis and the efficiency of the CMS triggering system. This contribution describes preliminary studies on a new ML based PV-Finder method for PV identification. The method is based on a model trained using Kernel Density Estimations (KDEs) derived from the positions of reconstructed tracks at the beamline, incorporating uncertainties from track parameters. It also utilizes target histograms, modeled as Gaussian distributions centered on the actual ground truth values of specific primary vertices.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

When Spectral Modeling Meets Convolutional Networks: A Method for Discovering Reionization-era Lensed Quasars in Multiband Imaging Data

Over the last two decades, around 300 quasars have been discovered at z ≳ 6, yet only one has been identified as being strongly gravitationally lensed. We explore a new approach—enlarging the permitted spectral parameter space, while introducing a new spatial geometry veto criterion—which is implemented via image-based deep learning. We first apply this approach to a systematic search for reionization-era lensed quasars, using data from the Dark Energy Survey, the Visible and Infrared Survey Telescope for Astronomy Hemisphere Survey, and the Wide-field Infrared Survey Explorer. Our search method consists of two main parts: (i) the preselection of the candidates, based on their spectral energy distributions (SEDs), using catalog-level photometry; and (ii) relative probability calculations of the candidates being a lens or some contaminant, utilizing a convolutional neural network (CNN) classification. The training data sets are constructed by painting deflected point-source lights over actual galaxy images, to generate realistic galaxy–quasar lens models, optimized to find systems with small image separations, i.e., Einstein radii of θ E ≤ 1''. Visual inspection is then performed for sources with CNN scores of P lens > 0.1, which leads us to obtain 36 newly selected lens candidates, which are awaiting spectroscopic confirmation. These findings show that automated SED modeling and deep learning pipelines, supported by modest human input, are a promising route for detecting strong lenses from large catalogs, which can overcome the veto limitations of primarily dropout-based SED selection approaches.

High-redshift galaxies↗

DOE OSTI2022

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

As power systems are undergoing a significant transformation with more uncertainties, less inertia and closer to operation limits, there is increasing risk of large outages. Thus, there is an imperative need to enhance grid emergency control to maintain system reliability and security. Towards this end, great progress has been made in developing deep reinforcement learning (DRL) based grid control solutions in recent years. However, existing DRL-based solutions have two main limitations: 1) they cannot handle well with a wide range of grid operation conditions, system parameters, and contingencies; 2) they generally lack the ability to fast adapt to new grid operation conditions, system parameters, and contingencies, limiting their applicability for real-world applications. Here, in this paper, we mitigate these limitations by developing a novel deep meta-reinforcement learning (DMRL) algorithm. The DMRL combines the meta strategy optimization together with DRL, and trains policies modulated by a latent space that can quickly adapt to new scenarios. We test the developed DMRL algorithm on the IEEE 300-bus system. We demonstrate fast adaptation of the meta-trained DRL polices with latent variables to new operating conditions and scenarios using the proposed method, which achieves superior performance compared to the state-of-the-art DRL and model predictive control (MPC) methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

A Subcritical Assembly for Training and Education Use at the Oak Ridge National Laboratory

A near-final design has been completed for a new subcritical assembly at Oak Ridge National Laboratory (ORNL). This subcritical assembly will support the Nuclear Criticality Safety Program (NCSP) training and education program to augment current subcritical assembly capabilities at Sandia National Laboratories (SNL) and at the National Criticality Experiments Research Center (NCERC). Other collaborations have been proposed with the Sunrise Consortium and others. This proposed subcritical assembly uses legacy AGN-201M research reactor fuel plates that are available from the Y-12 National Security Complex. This subcritical assembly contains approximately 620 grams of 235 U as UO 2 particulates distributed homogeneously in radiation-stabilized polyethylene. The fuel plates will have a graphite neutron reflector to obtain a core multiplication, M, from 10 to 50, corresponding to a k eff of 0.9 to 0.95, respectively. The subcritical assembly will be able to support at least four experiments for the training courses: (1) the addition of fissile material to increase neutron multiplication (mass), (2) a core separation experiment (interaction), (3) the effect of adding moderators to the core (moderation), and (4) the effect of adding neutron absorbers to the core (poison/absorption). The proposed ORNL subcritical assembly will provide additional student bandwidth to support NCSP training courses and to provide for some backup capacity if nuclear facility operations are disrupted at SNL or NCERC due to some unforeseen issue. The subcritical assembly will be designed to be an inherently safe subcritical assembly—subcritical under all normal and abnormal conditions—and it will provide the capability to conduct hands-on training to support NCSP and general nuclear criticality safety (NCS) staff training and qualification goals.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

A Subcritical Assembly for Training and Education Use at the Oak Ridge National Laboratory [Abstract]

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2020

Hybrid geological modeling: Combining machine learning and multiple-point statistics

Accurately modeling and constructing a geologically realistic subsurface model remains an outstanding problem as the morphology controls the flow behaviors. Particularly, one of the pattern-based methods, namely cross-correlation based simulation, has been proved to be an effective way to reconstruct a realistic model, at both small and large scales. However, conditioning to point data in the large-scale problems is still a crucial issue in these algorithms, since there is always a trade-off between the quality of the realizations and the degree of point data reproduction. Specifically, it is not practical to build a training image (TI) which includes all the possibilities and variabilities. Therefore, finding a pattern that can represent the point data and, at the same time, preserving the connectivities is difficult. This leads to producing highly-connected realizations with a significant mismatch or poor models with a reasonable degree of point data reproduction. To accurately reproduce the densely distributed hard data, pixel-based methods can also produce some unrealistic artifacts around the hard data. In this paper, to overcome this challenge, however, we use pattern-based methods as they often produce more disconnected geobodies when dealing with dense hard data, and proposed a hybrid algorithm using the pattern-based methods and convolutional neural network (CNN). The trained CNN model is utilized to improve the quality of conditioning to point data for the original realizations generated by the pattern-based algorithm. As such, the mismatch locations are identified, and the same regions are used in the training of CNN to mimic the procedure through which a missing region can be filled. To evaluate the performance of the proposed hybrid algorithm, it is tested on cases with different dimensions and different numbers of facies. Then, the newly improved realizations are compared with the initial realizations generated by the pattern-based algorithm. The comparison is also conducted by the flow simulation test. And it indicates that the proposed hybrid algorithm can better reproduce the point data, while the connectivities are better preserved.

58 GEOSCIENCES↗