Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Training Time”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Uncertainty quantification for neural network potential foundation models

Abstract For neural network potentials (NNPs) to gain widespread use, researchers must be able to trust model outputs. However, the blackbox nature of neural networks and their inherent stochasticity are often deterrents, especially for foundation models trained over broad swaths of chemical space. Uncertainty information provided at the time of prediction can help reduce aversion to NNPs. In this work, we detail two uncertainty quantification (UQ) methods. Readout ensembling, by finetuning the readout layers of an ensemble of foundation models, provides information about model uncertainty, while quantile regression, by replacing point predictions with distributional predictions, provides information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to the foundation model and a series of finetuned models. The uncertainties produced by the readout ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Training Ultrasound Image Classification Deep-Learning Algorithms for Pneumothorax Detection Using a Synthetic Tissue Phantom Apparatus

Ultrasound (US) imaging is a critical tool in emergency and military medicine because of its portability and immediate nature. However, proper image interpretation requires skill, limiting its utility in remote applications for conditions such as pneumothorax (PTX) which requires rapid intervention. Artificial intelligence has the potential to automate ultrasound image analysis for various pathophysiological conditions. Training models require large data sets and a means of troubleshooting in real-time for ultrasound integration deployment, and they also require large animal models or clinical testing. Here, we detail the development of a dynamic synthetic tissue phantom model for PTX and its use in training image classification algorithms. The model comprises a synthetic gelatin phantom cast in a custom 3D-printed rib mold and a lung mimicking phantom. When compared to PTX images acquired in swine, images from the phantom were similar in both PTX negative and positive mimicking scenarios. We then used a deep learning image classification algorithm, which we previously developed for shrapnel detection, to accurately predict the presence of PTX in swine images by only training on phantom image sets, highlighting the utility for a tissue phantom for AI applications.

Boice, Emily N. (ORCID:0000000171802842)↗
DOE OSTI2025

CRCNS22 Learning Rules in the Hippocampus and their Mapping to Neuromorphic Systems (Final Technical Report)

Large scale biologically-realistic computational models are key to investigating the interplay between structure and function in nervous systems, thus paving the way to new clinical methods and neuro-inspired computing solutions. This project focuses on the hippocampus, in particular the CA3-CA1 regions, due to their role in associative learning and memory, pattern separation and completion, and spatial navigation. Investigations into the neuronal organization and learning rule(s) of this circuit can shed light into how declarative memories are formed, stored, recalled and forgotten and inform computational, experimental and clinical neuroscience work. Our project aims at developing a novel data-driven methodology supported by a broad heterogeneous base of neuroscience experimental knowledge and inspired from advances in computer science and engineering. Specifically, this work will benchmark existing and new learning rules within a full-scale spiking neural network simulation of the CA3-CA1 region. The model will be based on an open-source repository, called the Hippocampome, which contains neuronal morphologies, firing patterns, synapse probabilities, and most other required parameters for all known neuron types in the rodent hippocampal formation. The model will be first trained in a supervised fashion for associative memory tasks using backpropagation through time traditionally used in computer science, enhanced with a new technique called the surrogate gradient method. This optimization method will be used to obtain a global loss minimization, but it is not biologically inspired as it assumes the use of data not locally available to the synapses. However, we propose its use as a benchmarking tool, to compare the training performance of local biologically plausible and hardware-mappable learning rules at scale. New rules or combinations will be proposed and tested as needed, based on the obtained results. Progress in this area will also drive the development of novel hardware-mappable algorithms for continual lifelong learning and categorization of new events from few presented examples. This project goes beyond the existing state-of-the-art by looking at large scale realistic neuronal circuits as networks trainable via global optimization methods such as surrogate gradient descent. The objective function of the brain that supports learning is largely unknown, but it is likely that it operates through local learning rules. Studying network trajectories around local minima as proposed in this work represents a useful strategy for understanding whether a network is training by using a specific (set of) learning rule(s). Starting from a completely untrained network is a challenging test since it is difficult to determine how the learning rule affects the trajectory of the network. This interdisciplinary project will help understand what rule governs learning in these regions or if multiple learning rules are involved. The work will develop a robust methodology to measure if the network is converging to the target solution, oscillating around it, or diverging away.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Automatic identification and quantification of dense microcracks in high-performance fiber-reinforced cementitious composites through deep learning-based computer vision

Highlights: • A method is presented to detect, locate, quantify, and visualize dense microcracks in HPFRCC. • Quantification of dense microcracks is realized using deep learning method for the first time. • The presented method uses deep learning models that are trained using a realistic dataset size. • The presented method has a high computation efficiency for identifying and quantifying cracks. • The presented method provides crack width with errors up to 50 μm and a R{sup 2} value of 0.984. High-performance fiber-reinforced cementitious composites (HPFRCCs) feature high mechanical strengths, crack resistance, and durability. Under excessive loading, HPFRCCs demonstrate dense microcracks that are difficult to identify using existing methods. This study presents a computer vision method for identification, quantification, and visualization of microcracks in HPFRCCs based on deep learning. The presented method integrates multiple deep learning models and computer vision techniques in a hierarchical architecture. The crack pattern (e.g., number, width, and spacing of cracks) are automatically determined from pictures without human intervention. This study shows that the presented method achieves an accuracy of 0.992 for crack detection and an accuracy finer than 50 μm (R{sup 2} > 0.984) for quantification of crack width when deep learning models are trained using only 200 pictures of HPFRCCs and 200 pictures of conventional concrete with incorporation of data augmentation. The presented method is expected to be also applicable to other materials featuring complex cracks.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Subseasonal Great Plains Rainfall via Remote Extratropical Teleconnections: Regional Application of Theory‐Guided Causal Networks

Abstract Long‐range U.S. summer rainfall prediction skill is low. Monsoon variability, especially over the West North Pacific Monsoon (WNPM) and/or East Asian Monsoon (EAM) region, can influence U.S. Great Plains hydroclimate variability via a forced Rossby wave response. Here, we explored subseasonal monsoon variability as a source of predictability for Great Plains rainfall. The boreal summer intraseasonal oscillation (BSISO) is related to Great Plains convection and Great Plains low‐level jet (LLJ) anomalies as well as a cross‐Pacific wave train. Using a causal effect network, we found that the time between BSISO‐related geopotential height anomalies and Great Plains rainfall anomalies is about 2 weeks; therefore, BSISO convection may be a valuable forecast of opportunity for subseasonal prediction of Great Plains convection anomalies. More specifically, causal link patterns/maps revealed that the above‐normal weekly EAM rainfall, rather than WNPM rainfall or general geopotential height activity over the East Asia, was causally linked to Great Plains LLJ strengthening and active Great Plains convection the following week.

Meteorology & Atmospheric Sciences↗
DOE OSTI2025

A Comparative Study of Physics‐Informed and Data‐Driven Neural Networks for Compound Flood Simulation at River‐Ocean Interfaces: A Case Study of Hurricane Irene

Simulating compound flooding (CF) at the river-ocean interface within large-scale Earth System Models (ESMs) presents significant challenges due to complex interactions between river discharge, storm surge, and tides. This study assesses the comparative advantages of physics-informed and data-driven machine learning (ML) approaches for enhancing local ESM performance. We systematically compare data-driven neural network models (i.e., CNNs, U-Net, Long Short-Term Memory (LSTM), Gated Recurrent Unit), and physics-informed neural network (PINN) models, including vanilla PINN and a finite-difference-based PINN (FD-PINN). Specifically, FD-PINN is introduced to enhance computational efficiency, accelerating vanilla PINNs by ∼6.5 times while improving accuracy. To enhance data-driven model training, a new data-generation approach is developed to sample historical fluvial and coastal flood events, which ensures a robust data set for extreme event prediction. The models are evaluated using a realistic one-dimensional river domain extracted from an ESM's river mesh and the Hurricane Irene event as an independent test case. Results show that FD-PINN achieves accurate predictions with significantly reduced computational costs relative to vanilla PINNs. Among data-driven models, the best overall performance is achieved by a CNN-LSTM hybrid, which balances accuracy and efficiency. While a fully connected CNN (CNN-FC) provides the best accuracy, it incurs high computational cost. Architectures lacking strong temporal modeling tend to underperform on unseen events. These findings highlight the importance of sequence-aware designs for robust generalization. This study reveals the trade-offs between physics-informed and data-driven models and proposes an adaptive hybrid framework for integrating ML into ESMs to enhance local flood simulations.

Earth Systems Modeling↗
DOE OSTI2024

ATAT: Astronomical Transformer for time series and Tabular data

Context. The advent of next-generation survey instruments, such as theVera C. RubinObservatory and its Legacy Survey of Space and Time (LSST), is opening a window for new research in time-domain astronomy. The Extended LSST Astronomical Time-Series Classification Challenge (ELAsTiCC) was created to test the capacity of brokers to deal with a simulated LSST stream. Aims. Our aim is to develop a next-generation model for the classification of variable astronomical objects. We describe ATAT, the Astronomical Transformer for time series And Tabular data, a classification model conceived by the ALeRCE alert broker to classify light curves from next-generation alert streams. ATAT was tested in production during the first round of the ELAsTiCC campaigns. Methods. ATAT consists of two transformer models that encode light curves and features using novel time modulation and quantile feature tokenizer mechanisms, respectively. ATAT was trained on different combinations of light curves, metadata, and features calculated over the light curves. We compare ATAT against the current ALeRCE classifier, a balanced hierarchical random forest (BHRF) trained on human-engineered features derived from light curves and metadata. Results. When trained on light curves and metadata, ATAT achieves a macro F1 score of 82.9 ± 0.4 in 20 classes, outperforming the BHRF model trained on 429 features, which achieves a macro F1 score of 79.4 ± 0.1. Conclusions. The use of transformer multimodal architectures, combining light curves and tabular data, opens new possibilities for classifying alerts from a new generation of large etendue telescopes, such as theVera C. RubinObservatory, in real-world brokering scenarios.

Astronomy & Astrophysics↗
DOE OSTI2023

Data-driven modeling and control of an X-ray bimorph adaptive mirror

Adaptive X-ray mirrors are being adopted on high-coherent-flux synchrotron and X-ray free-electron laser beamlines where dynamic phase control and aberration compensation are necessary to preserve wavefront quality from source to sample, yet challenging to achieve. Additional difficulties arise from the inability to continuously probe the wavefront in this context, which demands methods of control that require little to no feedback. In this work, a data-driven approach to the control of adaptive X-ray optics with piezo-bimorph actuators is demonstrated. This approach approximates the non-linear system dynamics with a discrete-time model using random mirror shapes and interferometric measurements as training data. For mirrors of this type, prior states and voltage inputs affect the shape-change trajectory, and therefore must be included in the model. Without the need for assumed physical models of the mirror's behavior, the generality of the neural network structure accommodates drift, creep and hysteresis, and enables a control algorithm that achieves shape control and stability below 2 nm RMS. Using a prototype mirror and ex situ metrology, it is shown that the accuracy of our trained model enables open-loop shape control across a diverse set of states and that the control algorithm achieves shape error magnitudes that fall within diffraction-limited performance.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Detecting Anomalies in Time Series Using Kernel Density Approaches

This paper introduces a novel anomaly detection approach tailored for time series data with exclusive reliance on normal events during training. Our key innovation lies in the application of kernel-density estimation (KDE) to scrutinize reconstruction errors, providing an empirically derived probability distribution for normal events post-reconstruction. This non-parametric density estimation technique offers a nuanced understanding of anomaly detection, differentiating it from prevalent threshold-based mechanisms in existing methodologies. In post-training, events are encoded, decoded, and evaluated against the estimated density, providing a comprehensive notion of normality. In addition, we propose a data augmentation strategy involving variational autoencoder-generated events and a smoothing step for enhanced model robustness. The significance of our autoencoder-based approach is evident in its capacity to learn normal representation without prior anomaly knowledge. Through the KDE step on reconstruction errors, our method addresses the versatility of anomalies, departing from assumptions tied to larger reconstruction errors for anomalous events. Our proposed likelihood measure then distinguishes normal from anomalous events, providing a concise yet comprehensive anomaly detection solution. The extensive experimental results support the feasibility of our proposed method, yielding significantly improved classification performance by nearly 10% on the UCR benchmark data.

Frehner, Robin↗
DOE OSTI2022

Characterization of Temperature Heterogeneity in Utility-Scale Power Plant Boilers by Spatially Distributed Ultrasonic Measurements

In extreme environments, even hardened insertion sensors fail quickly. For such environments, we have developed an ultrasonic (US) method for measuring the spatial distribution of temperatures in solid materials and, specifically, across containments of extreme processes. By deploying US sensors in multiple locations, spatial heterogeneity in temperatures inside harsh environments may be characterized. In this note, we update our progress on testing the developed approach in an industrial setting and its performance in describing the temperature distribution inside a 500 MW (electrical) coalfired utility boiler of an electrical power generation plant. We use waveguides (WG) structured to contain echogenic features that produce a train of echoes in response to an excitation pulse. The time of flight between echoes encodes the information on the temperature distribution in the corresponding segment of the WG, which we reconstruct using its parametrization. Five waveguides were welded to the boiler’s heat exchange surface (waterwall) and different locations at the same boiler elevation and produced reliable US waveforms 18 months after the installation. For several weeks during the latest trial, we performed US measurements in five locations spanning half of the boiler’s 14-meter width. The measurements were acquired during regular operation, including load cycling to adjust for demand and intermittent contributions of renewable power sources, and during boiler shutdown for emergency maintenance and the subsequent restart. The interpretation of US waveforms by signal processing resulted in an accurate estimation of temperature distribution along the waveguides. We captured daily cyclical variations in demand-following load. Measurements performed in multiple locations revealed unexpected temperature variations across the boiler. We conclude that an array of developed sensors can provide responsive and spatial temperature measurements while maintaining functionality despite prolonged exposure to an extreme environment.

Walton, Kenneth↗
DOE OSTI2024

Deep Learning-Based Failure Prognostic Model for PV Inverter Using Field Measurements

Here, this study presents a novel approach for the precise monitoring and prognosis of photovoltaic (PV) inverter status, which is crucial for the proactive maintenance of PV systems. It addresses the gaps in traditional model-based methods, which tend to neglect the overall reliability of inverters, and the limitations of data-driven approaches that largely depend on simulated data. This research presents a robust solution applicable to real-world scenarios. The proposed data-driven model for PV inverter failure prognosis employs actual inverter measurements, integrating various operational and weather-related factors based on domain knowledge. This approach effectively represents inverter stressors and operational status. Utilizing an Enhanced Siamese Convolutional Neural Network (ESCNN), the model merges operational data with domain knowledge features, redefining the prognosis challenge as a classification task. Furthermore, the paper discusses an ESCNN-based real-time inverter failure monitoring method developed on the well-trained model. The proposed models are rigorously trained and tested with real inverter data and a novel filtering method is included to address accidental failures in practical scenarios. The results validate the model's efficacy, and the directions for future research are also outlined.

42 ENGINEERING↗
DOE OSTI2024

Power System Frequency Dynamics Modeling, State Estimation, and Control using Neural Ordinary Differential Equations (NODEs) and Soft Actor-Critic (SAC) Machine Learning Approaches

With the global energy transition of the electric power system, grid control, supervision, and protection is becoming more challenging. With the increasing integration of renewable energy sources (RES), the system dynamics are changing, causing traditional power system dynamic modeling with swing equation-based modeling approaches to fail. Additionally, the converter-dominated power grid is decreasing the system inertia, making the power system more fragile to the frequency swings. This paper first investigates and compares the application of a model-based Kalman filter state estimation approach with (i) a model-free machine learning approach --- neural ordinary differential equations (NODEs) --- and (ii) a data-driven system identification (SysId) approach to model and infer critical state values of the power system frequency dynamics. Then a model predictive control (MPC) framework is compared to a model-free Soft Actor-Critic (SAC) reinforcement learning (RL) control algorithm in providing efficient fast frequency response (FFR) to the power system frequency dynamics. The approaches are compared in terms of their performance goals as well as their per-timestep computational efficiency. Furthermore, the comparative study for state estimation shows that for the model-free requirement, both NODEs and SysId can provide accurate state estimates; however, with increasing model complexity, NODEs can be a better choice for model identification. Similarly, the results from the FFR comparative study show that the SAC RL-based FFR, once trained, outperforms MPC with better control signals and faster computation time, making the SAC RL-based FFR better option for providing FFR to the power system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)

Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.

Eseye, Abinet Tesfaye↗
DOE OSTI2023

Seasonal and Geographical Variations in Fundamental Weather Patterns during Extreme Precipitation as Identified from Omega Equation Forcing

Abstract We investigate the large-scale weather patterns during extreme precipitation (PEx) events over the conterminous United States (CONUS) by applying a version of the quasigeostrophic (QG) omega equation. This work aims to develop a climatology of the weather patterns most related to PEx events during current climate. Extreme events are examined for each of seven regions defined by consistent annual cycles of precipitation and spanning the CONUS. For the CONUS we train several self-organizing maps (SOM) on a pressure–time series of vertical velocity from each of the advective forcing terms in the QG omega equation for each extreme event. The unsupervised learning of the SOM allows us to identify the most descriptive set of nine patterns in vertical velocity associated with precipitation extremes. This method finds multiple frontal- and cyclone-driven patterns while grouping primarily convective events into one pattern. Frontal events include a synoptic pattern consistent with West Coast atmospheric river events as well as pattern groups linked to developing and to mature (“occluded”) frontal cyclones. The primary patterns found during PEx events vary seasonally and geographically. Frontal cyclone patterns are most common during PEx events during summer in the part of the Great Plains and during winter for the Northeast, Southeast, Pacific Northwest, and Southwest. Convection is the most common pattern during summer in all regions. Except in the Southeast, the annual cycles of monthly number of PEx events and average precipitation match well, partially validating our choice of regions to aggregate PEx events.

Meteorology & Atmospheric Sciences↗
DOE OSTI2024

Spectral Data Fusion From Handheld Laser-Induced Breakdown Spectroscopy (LIBS) and X-ray Fluorescence (XRF) Analyzers for Improved Detection of Cerium in a Simulated Dispersal Accident

Here, this work implements a mid-level data fusion methodology on spectral data from handheld X-ray fluorescence and laser-induced breakdown spectroscopy analyzers to quantify plutonium surrogate (CeO 2 ) contamination in soil samples for the first time. Spectral data from each analyzer were used independently to train supervised machine learning regressions to predict Ce concentration. Fused features from both data sets were then used to train the same models, comparing prediction performance by evaluating model precision and sensitivity. Fusing principal component scores from the two sensors yielded an order of magnitude improvement in precision and sensitivity of predictions made with an artificial neural network, compared to predictions made by models trained on independent sensor data. As a result, a boosted ensemble trained on the fused spectral features yielded an ideal predictor with root-mean-squared error on the order of 10 –6 and calculated limit of detection order 10 –5 wt %.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Bayesian quantum state reconstruction with a learning-based tuned prior

We demonstrate machine-learning-enhanced Bayesian quantum state tomography on near-term intermediate-scale quantum hardware. Our approach to selecting prior distributions leverages pre-trained neural networks incorporating measurement data and enables improved inference times over standard prior distributions.

Regmi, Sangita↗
DOE OSTI2021

LANL Charging Practices Course Number 18930

Because numerous federal agencies and outside organizations provide funding for hundreds of projects and programs, Los Alamos National Laboratory's system for charging for time and effort, and the purchase of materials, supplies, and services, is complex. As a LANL employee, you are responsible, both legally and ethically, for knowing and consistently following charging practice rules, complying with Laboratory financial policies, and understanding the consequences of improper charging. This course is designed to increase your understanding of an employee's responsibilities regarding charging practices at the Laboratory and help you charge time and labor and non-labor costs accurately. The training is required of all Laboratory workers except craft labor, who have separate training that is specific to how their effort is charged and collected. This training should take less than 20 minutes to complete.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

High Energy Physics Network Requirements Review (Final Report, July-October 2020)

Throughout 2020, ESnet and the Office of High Energy Physics (HEP) of the DOE SC organized an ESnet requirements review of HEP-supported activities. Preparation for this event included identification of key stakeholders: program and facility management, research groups, technology providers, and a number of external observers. These individuals were asked to prepare formal case study documents about their relationship to the HEP program to build a complete understanding of the current, near-term, and long-term status, expectations, and processes that will support the science going forward. A series of pre-planning meetings better prepared case study authors for this task, along with guidance on how the review would proceed in a virtual fashion.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗