Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
NASA NTRS1976

Joint aircraft loading/structure response statistics of time to service crack initiation

A reliability analysis for predicting the statistical distribution of time to fatigue crack initiation for aircraft structures in service is presented. The present analysis utilizes the statistical data of the specimen fatigue tests, the full-scale structure tests, and the statistical dispersion of aircraft service loads. The statistical distribution of the time to fatigue crack initiation of the full-scale structure under laboratory loading spectrum is assumed to be Weibull. The service loads for gust turbulences are modeled as Poisson processes for transport-type aircraft, while the maneuver loads are modeled as compound Poisson processes for fighter and training aircraft. It is found that the statistical distribution of time to fatigue crack initiation for aircraft structures in service is not Weibull and that the prediction on the basis of the Weibull distribution is unconservative, in particular in the early service time.

Yang, J.-N.↗
DOE OSTI2021

Operator learning for predicting multiscale bubble growth dynamics

We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

ThunderSecure: deploying real-time intrusion detection for 100G research networks by leveraging stream-based features and one-class classification network

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks (40/100G) across distributed sites for processing, storage, and analysis. Like general purpose networks, research networks experience intrusions. However, monitoring anomalies in such high-speed network traffics is challenging given current cyber-infrastructure. Moreover, traditional network intrusion detection systems (NIDS) are signature based. However, anomaly patterns are difficult to define and that rulesets are often not updated frequently enough to reflect the changes of attack behaviors. We present ThunderSecure, a high-throughput, unsupervised learning-based intrusions detection system for 100G research networks. ThunderSecure implements an efficient packet processing and detection pipeline using multi-cores and GPUs. It extracts statistical and temporal features from real-time network data streams and feeds them to a one-class anomaly detection network. A baseline of normal distribution will be created based on the training observation. Testing traffic deviated from the learned profile will be marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets mirrored from two 100G network connections at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. Results show that ThunderSecure can recognize science data traffic captured long after the training and made nearly certain detection on the segment of the streams where anomalous flows were injected.

100G research network↗
DOE OSTI2024

SDYN-GANs: Adversarial learning methods for multistep generative models for general order stochastic dynamics

We introduce adversarial learning methods for data-driven generative modeling of dynamics of nth-order stochastic systems. Our approach builds on Generative Adversarial Networks (GANs) with generative model classes based on stable m-step stochastic numerical integrators. From observations of trajectory samples, we introduce methods for learning long-time predictors and stable representations of the dynamics. Our approaches use discriminators based on Maximum Mean Discrepancy (MMD), training protocols using both conditional and marginal distributions, and methods for learning dynamic responses over different time-scales. We show how our approaches can be used for modeling physical systems to learn force-laws, damping coefficients, and noise-related parameters. Our adversarial learning approaches provide methods for obtaining stable generative models for dynamic tasks including long-time prediction and developing simulations for stochastic systems.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2025

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗
DOE OSTI2022

Catlas: an automated framework for catalyst discovery demonstrated for direct syngas conversion

Catalyst discovery is paramount to support access to energy and key chemical feedstocks in a post fossil fuel era. Exhaustive computational searches of large material design spaces using ab initio methods like density functional theory (DFT) are infeasible. We seek to explore large design spaces at relatively low computational cost by leveraging large, generalized, graph-based machine learning (ML) models, which are pretrained and therefore require no upfront data collection or training. We present Catlas, a framework that distributes and automates the generation of adsorbate-surface configurations and ML inference of DFT energies to achieve this goal. Catlas is open source, making ML assisted catalyst screenings easy and available to all. To demonstrate its efficacy, we use Catlas to explore catalyst candidates for the direct conversion of syngas to multi-carbon oxygenates. For this case study, we explore 947 stable/metastable binary, transition metal intermetallics as possible catalyst candidates. On this subset of materials, we are able to predict the adsorption energy of key descriptors, *CO and *OH, with good accuracy (0.16, 0.14 eV MAE, respectively). Using the projected selectivity towards C 2+ oxygenates from an existing microkinetic model, we identified 144 candidate materials. For 10 promising candidates, DFT calculations reveal a good correlation with our assessment using ML. Finally, among the top elemental combinations were Pt–Ti, Pd–V, Ni–Nb, and Ti–Zn, all of which appear unexplored experimentally.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Optimisation of the Kaplan hydropower system via PID 2 and digital twin

Here, this paper proposes a proportional–integral-double–derivative (PID 2 ) optimisation method for the Kaplan hydropower system by building a digital twin. The study first uses one multilayer perceptron (MLP) to model the hydroturbine dynamic and then adopts three connected MLPs to model the generator dynamic, both in an open-loop fashion. Inspired by stochastic distribution control (SDC) theory, we regard the training of the turbine's neural network model as a process control problem, and we propose minimising entropy loss to update the network parameters. The next step is to build the digital twin by connecting the neural network models with a PID 2 controller and a lead-lag exciter and run the whole model in a closed-loop fashion. After that, a binary search approach is applied to optimise the PID 2 parameters based on the obtained digital twin model. The simulation results show that the proposed method can reduce the mean square tracking error by more than 90%. Furthermore, the method is extended to jointly optimise the PID 2 controller and excitation system gains through multiobjective optimisation, leveraging Pareto frontier analysis to balance active power and voltage tracking performance. Simulation results confirm the effectiveness of the proposed method, achieving a 83.46% reduction in relative mean square error of active power, a 47.13% reduction in terminal voltage tracking error, and an 82.78% improvement in the overall scalarized objective.

Hydropower system↗
DOE OSTI2025

Dynamical Sketching for Enhanced Communication Efficiency in Federated Learning

Federated learning (FL) has revolutionized distributed machine learning by enabling collaborative model training without sharing local data. However, communication efficiency and privacy guarantees remain significant challenges. This paper introduces a dynamic sketching mechanism in FL, optimizing the trade-off between communication efficiency and model accuracy. By dynamically selecting the sketch matrix size, our approach adapts to the evolving characteristics of the data and the model, ensuring optimal performance across diverse scenarios. We leverage Bayesian optimization to systematically tune the sketch parameters, achieving an effective balance between resource efficiency and model performance. Experimental results on the MNIST dataset using a convolutional neural network (CNN) architecture validate the proposed method's efficiency and scalability. Our dynamic sketching approach significantly outperforms fixed-size sketching techniques, achieving higher compression ratios (up to 62x) and providing better privacy guarantees while maintaining high model accuracy. These findings highlight the robustness and versatility of our approach and make it a valuable solution for privacy-preserving, communication-efficient federated learning.

Afrose, Sharmin [ORNL]↗
DOE OSTI2021

Preventing Failures By Dataset Shift Detection in Safety-Critical Graph Applications

Dataset shift refers to the problem where the input data distribution may change over time (e.g., between training and test stages). Since this can be a critical bottleneck in several safety-critical applications such as healthcare, drug-discovery, etc., dataset shift detection has become an important research issue in machine learning. Though several existing efforts have focused on image/video data, applications with graph-structured data have not received sufficient attention. Therefore, in this paper, we investigate the problem of detecting shifts in graph structured data through the lens of statistical hypothesis testing. Specifically, we propose a practical two-sample test based approach for shift detection in large-scale graph structured data. Our approach is very flexible in that it is suitable for both undirected and directed graphs, and eliminates the need for equal sample sizes. Using empirical studies, we demonstrate the effectiveness of the proposed test in detecting dataset shifts. We also corroborate these findings using real-world datasets, characterized by directed graphs and a large number of nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Propagating Uncertainties in the SALT3 Model-training Process to Cosmological Constraints

Type Ia supernovae (SNe Ia) are standardizable candles that must be modeled empirically to yield cosmological constraints. To understand the robustness of this modeling to variations in the model-training procedure, we build an end-to-end pipeline to test the recently developed SALT3 model. We explore the consequences of removing pre-2000s low-z or poorly calibrated U-band data, adjusting the amount and fidelity of SN Ia spectra, and using a model-independent framework to simulate the training data. We find that the SALT3 model surfaces are improved by having additional spectra and U-band data, and can be shifted by ~5% if host-galaxy contamination is not sufficiently removed from SN spectra. We find that resulting measurements of w are consistent to within 2.5% for all of the training variants explored in this work, with the largest shifts coming from variants that add color-dependent calibration offsets or host-galaxy contamination to the training spectra and those that remove pre-2000s low-z data. These results demonstrate that the SALT3 model-training procedure is largely robust to reasonable variations in the training data, but that additional attention must be paid to the treatment of spectroscopic data in the training process. We also find that the training procedure is sensitive to the color distributions of the input data—the resulting w measurement can be biased by ~2% if the color distribution is not sufficiently wide. Future low-z data, particularly u-band observations and high signal-to-noise ratio SN Ia spectra, will help to significantly improve SN Ia modeling in the coming years.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1979

Landsat-D thematic mapper simulation in an urban area using aircraft multispectral scanner data

A simulation of imagery from the Landsat-D thematic mapper was conducted in order to determine its usefulness for urban land-use classification. Aircraft 24-channel multispectral scanner imagery of the Los Angeles area at 7.5-m resolution was processed digitally by means of matrix averaging and image smoothing techniques to simulate the 30-m resolution of the thematic mapper. Mean and standard deviation statistics of training sites for resolutions of 7.5, 15, 30 and 60 m were used to generate final classification maps. Plots of relative standard deviation showed that for larger training sites, as the resolution decreased, the distribution range of density values also decreased, while plots of relative classification accuracies showed that as resolution decreased, classification accuracies for three levels of standard deviation increased. A point of diminishing returns was indicated, however, confirming the utility of the resolution intended for Landsat-D.

Clark, J.↗
NASA NTRS1938

Investigations on the downwash behind a tapered wing with fuselage and propeller

The new downwash measurements behind a tapered wing with parallel center section described in the present report can be brought into good agreement with theoretical calculations if made on the basis of not-rolled-up vortex sheet and allowance is made for the lowering of the sheet. The test values are about 1 degree higher than the "upper limit" established for it, as against approximately 0.5 degrees in the earlier tests behind a rectangular and elliptical wing. The measurements on lateral axes, especially if lying below the wing on a level with the vortex train, disclosed in accord with the lift distribution, a marked change in angle over the span of the tail in contrast to the rectangular and elliptical wing.

Muttray, H↗
NASA NTRS2011

NASA A-Train and Terra Observations of the 2010 Russian Wildfires

Wildfires raged throughout western Russia and parts of Eastern Europe during a persistent heat wave in the summer of 2010. Anomalously high surface temperatures (35 - 41 C) and low relative humidity (9 - 25 %) from mid- June to mid-August 2010 shown by analysis of radiosonde data from multiple sites in western Russia were ideal conditions for the wildfires to thrive. Measurements of outgoing longwave radiation (OLR) from the Atmospheric Infrared Sounder (AIRS) over western Russian indicate persistent subsidence during the heat wave. Daily three-day back-trajectories initiated over Moscow reveal a persistent anticyclonic circulation for 18 days in August, coincident with the most intense period of fire activity observed by Moderate Resolution Imaging Spectroradiometer (MODIS). This unfortunate meteorological coincidence allowed transport of polluted air from the region of intense fires to Moscow and the surrounding area. We demonstrate that the 2010 Russian wildfires are unique in the record of observations obtained by remote-sensing instruments on-board NASA satellites: Aura and Aqua (part of the A-Train Constellation) and Terra. Analysis of the distribution of MODIS fire products and aerosol optical thickness (AOT), UV aerosol index (AI) and single-scattering albedo (SSA) from Aura's Ozone Monitoring Instrument (OMI), and total column carbon monoxide (CO) from Aqua s Atmospheric Infrared Sounder (AIRS) show that the region in the center of western Russia surrounding Moscow (52-58 deg N, 33 -43 deg E) is most severely impacted by wildfire emissions. Over this area, AIRS CO, OMI AI, and MODIS AOT are significantly enhanced relative to the historical satellite record during the first 18 days in August when the anti-cyclonic circulation persisted. By mid-August, the anti-cyclonic circulation was replaced with westerly transport over Moscow and vicinity. The heat wave

Witte, J. C.↗
DOE OSTI2025

A Risk-Informed Approach to Trustworthiness Assessment in Digital Twins-Based Autonomous Control

In autonomous control systems, digital twins (DTs) are used to perform diagnostic and prognostic functions. The trustworthiness of these DTs is dependent on quality and coverage of the training data, model accuracy and integrity of sensor data. This work introduces a methodology to determine the trustworthiness of a DT system given faulty sensor data using a risk informed approach. Bayesian Belief Networks (BBNs) are used to propagate uncertainties and determine the probability of trustable recommendations. The decision to trust the control action provided by the DT is based on the DT output, expert opinion, and severity of problems. The performance of DTs is reliant on the data they are trained on. When they encounter out of distribution data, the trustworthiness of the recommendations decreases. To address this issue, we include an expert component that provides input on sensor degradation. For this, we utilize a generative artificial intelligence (AI) model, such as Generative Pretrained Transformer (GPT). The GPT functions as an expert with broad knowledge. The GPT is fine-tuned to understand and discriminate sensor degradation scenarios using manufactured data. This methodology is demonstrated through a case study on a Nearly Autonomous Management and Control System (NAMAC) during a steady state scenario. Various sensor degradation types with different severity levels are considered. Degraded sensor data is processed by the DT system and the fine-tuned GPT. Finally, using the BBN, we combine the GPT information and the DT output with its sources of uncertainty. This provides an output regarding the trustworthiness of the DT recommendation.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1988

An expert system for a distributed real-time trainer

The problem addressed by this expert system concerns the expansion of capability of a Real Time Trainer for the Spacelab flight crew. As requirements for more models or fidelity are placed upon the system, expansion is necessary. The simulator can be expanded using a larger processor or by going to a distributed system and expand by adding additional processors. The distributed system is preferable because it is more economical and can be expanded in a more incremental manner. An expert system was developed to evaluate modeling and timing capability within a real time training simulator. The expert system is based upon a distributed configuration. Components of the modeled system are control tasks, network tasks, emulator tasks, processors, displays, and a network. The distributed module expert system (DMES) allows the configuring of processors, tasks, display use, keyboard use, and selection of alternate methods to update the data buffer. Modules can be defined with execution occurring in a specific processor on a network. The system consists of a knowledge front end editor to interactively generate or update the knowledge base, an inference engine, a display module, and a recording module.

Purinton, Steven C.↗
DOE OSTI2021

Reinforcement Learning for Distribution Grid Optimization (PyCIGAR) v0.1

PyCIGAR is a python software package that merges off-the-shelf reinforcement learning libraries (RLLib and Ray) with electric power distribution system simulation tools (OpenDSS and a custom power flow solver built by LBL). PyCIGAR enables the training of neural networks to optimize the behavior of different components in the electric distribution grid, such as control systems in photovoltaic rooftop solar inverters and electric battery storage systems. The software package has been used to train neural networks to update settings in photovoltaic rooftop solar inverter control systems to mitigate cyber attacks on other solar photovoltaic rooftop devices.

Arnold, Daniel↗
DOE OSTI2022

Large Scale Caching and Streaming of Training Data for Online Deep Learning

The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.

data pipelines↗
DOE OSTI2022

A Machine‐Learning‐Assisted Stochastic Cloud Population Model as a Parameterization of Cumulus Convection

Abstract A machine‐learning‐assisted stochastic cloud population model is coupled with the Advanced Research Weather Research and Forecasting (WRF) model to represent fluctuations in the cloud‐base mass flux associated with the life cycles and interactions among cumulus convection cells. In this cloud population model, the size distribution and the associated cloud‐base mass flux of the convective cells are related to their previous state and to the change in the total convective area via a transition function. The convective area tendency in turn is assumed to depend on the cloud‐base mass flux that is resolved by the host WRF model. The transition function is represented by a single hidden‐layer neural network trained by the evolution of convective cell size distributions in a 1‐km grid‐spacing WRF simulation run over the Australian Monsoon region. At every grid point of the host model, the cloud population model predicts the cell size and cloud‐base mass flux distributions from which a random sample of cells is fed to an entraining parcel model that calculates precipitation as well as the associated liquid water potential temperature and total moisture tendencies. These tendencies are averaged over the cells and provided to the host model. Several regional simulations are performed over tropical and midlatitude domains to test this as a potential approach to scale‐aware parameterization. It is shown that such an approach could be a new promising path to simulating realistic precipitation statistics and propagation of precipitation associated with the Madden‐Julian Oscillation while maintaining realistic depictions of the diurnal cycle over both land and ocean.

54 ENVIRONMENTAL SCIENCES↗