Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Training Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

MACHINE LEARNING BASED CHEMICAL EXPLOSIVE MODE ANALYSIS (ML-CEMA)

The software consists of a Machine Learning based Chemical Explosive Mode Analysis (ML-CEMA) tool for advanced computational flame diagnostics. CEMA, originally based on eigen-analysis of the local thermochemical system, is capable of identifying reaction fronts and limit phenomena such as auto-ignition and extinction in practical combustion systems, such as internal combustion engines and gas turbine combustors. However, the original CEMA is computationally expensive for large reaction mechanisms that are typically needed to describe fuel chemistry of practical large-hydrocarbon fuels. This novel ML-CEMA tool employs a ML technique to accelerate the eigen-analysis of the basic CEMA approach by orders of magnitude, thus making it suitable for practical fuels. In ML-CEMA,zero-dimensional (0D) reactors and one-dimensional (1D) premixed flames are first used to generate a large number of data points for neural network based ML training. The trained ML model is then used to perform CEMA prediction. This ML-CEMA tool has been demonstrated in canonical 0D and 1D configurations as well as highly-transient three-dimensional spray flames exhibiting multi-mode turbulent combustion, showing promising results. ML-CEMA, as a standalone tool, can be used for computationally-efficient diagnostics of massive datasets generated from both experiments and simulations. For example, based on spatially resolved measurements of a small set of reactive scalars(such as temperature, hydroxyl radical and formaldehyde), ML-CEMA can effectively identify flame fronts and rare events. ML-CEMA also provides a robust online or offline flame feature detection tool. When used for on-the-fly simulations, ML-CEMA further enables zone-adaptive combustion modeling, in which the predicted eigenvalue is used as a robust mode indicator for judicious assignment of locally-valid combustion models. This ML-CEMA based zone-adaptive model can lead to substantial computational cost savings when used for large-scale simulations of multi-mode combustion systems. Third Party Code Web Page to Download Code Web Page Location of Third Party License

Xu, Chao↗
DOE OSTI2022

Efficient Clustering of Software Vulnerabilities using Self Organizing Map (SOM)

The common vulnerabilities and exposures (CVE) database was created with a mission to ``identify, define, and catalog publicly disclosed cybersecurity vulnerabilities''. This rich body of information can be used to enable rapid and efficient response to secure and defend cyber operations and protect critical cyber infrastructure. The main goal of this paper is to develop a visual analytics tool to enable deep analysis of CVEs using unsupervised clustering techniques. We enhance our analysis by first mapping CVEs to hierarchical-classes in Common Weakness Enumeration (CWE) using information in the National Vulnerability Database (NVD). Both the mapping and the numerical representation of CVEs are enabled by V2W-BERT, which uses natural language processing of the extensive information in NVD to generate a large tabular database of 137,226 CVE entries from 1999 to 2020, where each CVE is represented by a vector of 768 numerical features. The vectorized data is processed by Self-Organizing Maps (SOM), which is an unsupervised machine learning technique for dimensionality reduction, visual representation and clustering. Using a Torus map of 6417 units, we achieve ~10-fold data compression of ~140k CVEs using SOM. The trained map is further clustered using standard K-means clustering into 138 clusters of CVEs. We conducted a brief investigation of the rich mapping of CVEs to best-matching-units to K-means clusters, as well as CVEs to CWEs. For example, this novel mapping provided insight into the role of CWE-59 and CWE-264 in several CVEs that is otherwise hard to explore in the original data. We conclude that our this novel approach will not only enable deep analysis of the complex relationships between CVEs and CWEs, but also a mechanism to quickly respond to and design mitigation actions for rapidly evolving vulnerabilities that have not been mapped to existing CWEs.

Panchal, Khyati↗
DOE OSTI2023

Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems

We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems (BIPs) with models governed by nonlinear parametric partial differential equations (PDEs). Neural operators have gained significant attention in recent years for their ability to approximate the parameter-to-solution maps defined by PDEs using as training data solutions of PDEs at a limited number of parameter samples. The computational cost of BIPs can be drastically reduced if the large number of PDE solves required for posterior characterization are replaced with evaluations of trained neural operators. However, reducing error in the resulting BIP solutions via reducing the approximation error of the neural operators in training can be challenging and unreliable. We provide an a priori error bound result that implies certain BIPs can be ill-conditioned to the approximation error of neural operators, thus leading to inaccessible accuracy requirements in training. To reliably deploy neural operators in BIPs, we consider a strategy for enhancing the performance of neural operators: correcting the prediction of a trained neural operator by solving a linear variational problem based on the PDE residual. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error, all while retaining substantial computational speedups of posterior sampling when models are governed by highly nonlinear PDEs. The strategy is applied to two numerical examples of BIPs based on a nonlinear reaction–diffusion problem and deformation of hyperelastic materials. We demonstrate that posterior representations of the two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Discovering Ca II absorption lines with a neural network

Quasar absorption line analysis is critical for studying gas and dust components and their physical and chemical properties as well as the evolution and formation of galaxies in the early universe. Calcium II (Ca II ) absorbers, which are one of the dustiest absorbers and are located at lower redshifts than most other absorbers, are especially valuable when studying physical processes and conditions in recent galaxies. However, the number of known quasar Ca II absorbers is relatively low due to the difficulty of detecting them with traditional methods. In this work, we developed an accurate and quick approach to search for Ca II absorption lines using deep learning. In our deep learning model, a convolutional neural network, tuned using simulated data, is used for the classification task. The simulated training data are generated by inserting artificial Ca II absorption lines into original quasar spectra from the Sloan Digital Sky Survey (SDSS), while an existing Ca II catalogue is adopted as the test set. The resulting model achieves an accuracy of 96 per cent on the real data in the test set. Our solution runs thousands of times faster than traditional methods, taking a fraction of a second to analyse thousands of quasars, while traditional methods may take days to weeks. The trained neural network is applied to quasar spectra from SDSS’s DR7 and DR12 and discovered 399 new quasar Ca II absorbers. In addition, we confirmed 409 known quasar Ca II absorbers identified previously by other research groups through traditional methods.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Enhanced accuracy through ensembling of randomly initialized auto-regressive models for dynamical systems

Computational mechanics simulations using traditional finite element methods (FEM) require prohibitively expensive computational resources for real-time engineering applications, design optimization, and digital twin implementations. While machine learning (ML) surrogate models offer significant computational speedups, autoregressive ML models for time-dependent mechanical systems suffer from error accumulation that compromises long-term prediction reliability - a critical concern for engineering applications where accuracy over extended time horizons is essential for safety and performance assessments. Here, we propose a deep ensemble framework specifically designed to address this challenge in computational mechanics applications, where multiple ML surrogate models with random weight initializations are trained in parallel and their predictions aggregated during inference. This approach leverages statistical diversity to maximize information gain from a fixed set of training data and to mitigate error propagation, while maintaining the computational efficiency that makes ML surrogates attractive for engineering practice. We validate the framework on three representative problems spanning critical areas of computational mechanics: stress field evolution in heterogeneous microstructures under complex loading (relevant to advanced materials design and composite analysis), planetary-scale shallow water dynamics (applicable to environmental and geotechnical engineering), and Gray-Scott reaction-diffusion systems (relevant to mass transport and chemical process engineering). Across all test cases, the ensemble approach demonstrates consistent error reduction of 15-33% compared to individual models. The codes for this work are available on GitHub (https://github.com/Graham-Brady-Research-Group/AutoregressiveEnsemble_SpatioTemporal_Evolution).

autoregressive prediction↗
DOE OSTI2023

Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning

We identify effective stochastic differential equations (SDEs) for coarse observables of fine-grained particle- or agent-based simulations; these SDEs then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDEs through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler–Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes. They also lend themselves naturally to “physics-informed” gray-box identification when approximate coarse models, such as mean field equations, are available. Existing numerical integration schemes for Langevin-type equations and for stochastic partial differential equations can also be used for training; we demonstrate this on a stochastically forced oscillator and the stochastic wave equation. Our approach does not require long trajectories, works on scattered snapshot data, and is designed to naturally handle different time steps per snapshot. We consider both the case where the coarse collective observables are known in advance, as well as the case where they must be found in a data-driven manner.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

maslov-group/FUN-PROSE

mRNA levels of all genes in a genome is a critical piece of information defining the overall state of the cell in a given environmental condition. Being able to reconstruct such condition-specific expression in fungal genomes is particularly important to metabolically engineer these organisms to produce desired chemicals in industrially scalable conditions. Most previous deep learning approaches focused on predicting the average expression levels of a gene based on its promoter sequence, ignoring its variation across different conditions. Here we present FUN-PROSE—a deep learning model trained to predict differential expression of individual genes across various conditions using their promoter sequences and expression levels of all transcription factors. We train and test our model on three fungal species and get the correlation between predicted and observed condition-specific gene expression as high as 0.85. We then interpret our model to extract promoter sequence motifs responsible for variable expression of individual genes. We also carried out input feature importance analysis to connect individual transcription factors to their gene targets. A sizeable fraction of both sequence motifs and TF-gene interactions learned by our model agree with previously known biological information, while the rest corresponds to either novel biological facts or indirect correlations.

Liu, Simon↗
DOE OSTI2023

FUN-PROSE: A deep learning approach to predict condition-specific gene expression in fungi

mRNA levels of all genes in a genome is a critical piece of information defining the overall state of the cell in a given environmental condition. Being able to reconstruct such condition-specific expression in fungal genomes is particularly important to metabolically engineer these organisms to produce desired chemicals in industrially scalable conditions. Most previous deep learning approaches focused on predicting the average expression levels of a gene based on its promoter sequence, ignoring its variation across different conditions. Here we present FUN-PROSE—a deep learning model trained to predict differential expression of individual genes across various conditions using their promoter sequences and expression levels of all transcription factors. We train and test our model on three fungal species and get the correlation between predicted and observed condition-specific gene expression as high as 0.85. We then interpret our model to extract promoter sequence motifs responsible for variable expression of individual genes. We also carried out input feature importance analysis to connect individual transcription factors to their gene targets. A sizeable fraction of both sequence motifs and TF-gene interactions learned by our model agree with previously known biological information, while the rest corresponds to either novel biological facts or indirect correlations.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Foundations of an Electric Mobility Strategy for the City of Mexicali

The Foundations of an Electric Mobility Strategy for the city of Mexicali aligns with numerous energy, environmental, and transport plans and will help Mexicali meet multiple related goals. Mexicali’s energy mix, with 28% renewables, already enables plugin electric vehicles (PEVs) to reduce the mass of greenhouse gases (GHGs) per km driven 2/3 below that of their conventional counterparts. This GHG benefit will increase should Mexicali take steps to further increase their share of renewables in their electricity supply. Beyond increasing renewables, Mexicali could possibly deploy PEVs so that electric load is added in the right location (depending on further analysis of substations and feeders) and at the right time (between 21:00 and 11:00) in order to minimize grid upgrade costs. There are a handful of charge timing control mechanisms –at various stages of development– that Mexicali could implement. Transport electrification can facilitate mass transit by powering buses, trains, and small vehicles that get people from their homes or work to the transit stations and vice versa. Mexicali could utilize fleets as early PEV adopters in order to gain acceptance and add electric vehicle supply equipment (EVSE). Recommended prioritization of different types of fleets are suggested in this report: transit buses, school buses, airport ground support equipment (GSE), refuse trucks, taxis, shuttle buses, campus vehicles, delivery trucks, utility trucks, and finally semitrailers. There are a handful of policy options that Mexicali could use to incentivize fleets to purchase PEVs, including mandates, economic incentives, energy performance contracts, waivers to access restrictions, electricity discounts, and EVSE requirements in building codes. Mexicali’s taxi fleet was an early adopter of PEVs and had experienced some challenges—mostly related to the insufficient range of the taxis due to hot weather. In this report, we strategize ways to extend the range of the current electric taxis, including ways to make charging more convenient to the drivers, and more suggestions for appropriate vehicles to purchase in the future. This report also includes the groundwork of geotracking Mexicali’s taxi fleet so that more detailed recommendations can be made in the future. Once fleets have increased PEV acceptance and EVSE installations, the market will be ready to expand to private vehicle owners. In order to do this, more EVSE needs to be installed in the right locations. This mobility strategy lays out general local areas where EVSE could be well utilized, based on traffic patterns, land use, and demographic data. Mexicali could approach businesses within these areas that would likely make suitable hosts, based on how well they can profit from the additional business that EVSE would bring. Mexicali could then adopt a series of purchase incentives (including sales tax waivers or access to high-occupancy vehicles [HOV] lanes) that would encourage private vehicle owners to purchase PEVs. Purchase incentives run the risk of creating equity issues, which can be countered by promoting electrification in mass transit, creating more HOV lanes that have PEV exemptions, and installing EVSEs in underserved communities. Private PEV ownership will require a set of experts that Mexicali can help train, including PEV repair technicians, EVSE installation electricians, and first responders.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Puerto Rico Grid Resilience and Transitions to 100% Renewable Energy Study (PR100): Six-Month Progress Update [Slides]

Puerto Rico has committed to meeting its electricity needs with 100% renewable energy by 2050, along with realizing interim goals of 40% by 2025, 60% by 2040, the phase-out of coal-fired generation by 2028, and a 30% improvement in energy efficiency by 2040, as established in Puerto Rico Energy Public Policy Act (Act 17). Since hurricanes Irma and Maria in September 2017, DOE and its national laboratories have provided Puerto Rico energy system stakeholders with tools, training, and modeling support to enable planning and operation of the electric power grid with more resilience against further disruptions. On February 2, 2022, DOE, FEMA, and six national laboratories launched the two-year Puerto Rico Grid Resilience and Transitions to 100% Renewable Energy Study (PR100) to conduct comprehensive analysis of stakeholder-driven pathways to Puerto Rico's energy future. The robust and objective energy analysis entails five activities, with an emphasis on power system reliability, resilience, and generation planning. This presentation provides a six-month progress update on the study, including presentation of four initial scenario frameworks defined with extensive stakeholder input.

14 SOLAR ENERGY↗
DOE OSTI2022

Puerto Rico Grid Resilience and Transitions to 100% Renewable Energy Study (PR100) (Six-Month Progress Update) [Slides]

Puerto Rico has committed to meeting its electricity needs with 100% renewable energy by 2050, along with realizing interim goals of 40% by 2025, 60% by 2040, the phase-out of coal-fired generation by 2028, and a 30% improvement in energy efficiency by 2040, as established in Puerto Rico Energy Public Policy Act (Act 17). Since hurricanes Irma and Maria in September 2017, DOE and its national laboratories have provided Puerto Rico energy system stakeholders with tools, training, and modeling support to enable planning and operation of the electric power grid with more resilience against further disruptions. On February 2, 2022, DOE, FEMA, and six national laboratories launched the two-year Puerto Rico Grid Resilience and Transitions to 100% Renewable Energy Study (PR100) to conduct comprehensive analysis of stakeholder-driven pathways to Puerto Rico's energy future. The robust and objective energy analysis entails five activities, with an emphasis on power system reliability, resilience, and generation planning. This presentation was delivered in a public webinar on July 21, 2022, providing a high level summary of the progress in the first six months of the study, including presentation of four initial scenario definitions based on extensive stakeholder input.

14 SOLAR ENERGY↗
DOE OSTI2024

Inventory of Clean Energy Education and Workforce Development Programs in Allegheny County, PA

Building on other analysis work completed through the Communities LEAP pilot, this document is intended to provide an overview of: 1) what clean energy and related education and workforce development (EWD) resources already exist in and around the Hill District, 2) gaps in EWD resources, and 3) opportunities to expand clean energy education, training, and job opportunities for Hill District residents.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Classifying thermodynamic cloud phase using machine learning models

Vertically resolved thermodynamic cloud-phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Thermodynamic Cloud Phase (THERMOCLDPHASE) value-added product (VAP) uses a multi-sensor approach to classify the thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave-radiometer-derived liquid water path, and radiosonde temperature measurements. The measured pixels are classified as ice, snow, mixed phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multi-layer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with 1 year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1 score, and mean intersection over union (IOU). Analysis of ML confidence scores shows that ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential data streams for ML thermodynamic cloud-phase predictions. Lidar measurements exhibit lower feature importance due to rapid signal attenuation caused by the frequent presence of persistent low-level clouds at the NSA site. The ML models' generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. The models demonstrated similar performance to that observed at the NSA site. Finally, we evaluate the ML models' response to simulated instrument outages and signal degradation and show that a CNN U-Net model trained with input channel dropouts performs better when input fields are missing.

ARM Aerial Facility↗
DOE OSTI2024

The use of digital thread for reconstruction of local fiber orientation in a compression molded pin bracket via deep learning

A deep convolutional neural network (DCNN) was used for microstructure reconstruction using artificial intelligence (MR-AI) by predicting local average fiber orientation distributions (FOD) in a 3D prepreg platelet molded composite (PPMC) pin bracket. To train the MR-AI model, surface strain fields from residual stresses simulated in PPMC plates were used as the input to the DCNN. A training dataset included PPMC plates with various degrees of global fiber alignment, based on the information obtained from high-fidelity flow simulation of a pin bracket. Further, the MR-AI model was then deployed to analyze FOD in the 3D pin bracket by conducting thermo-elastic residual stress analysis. Initially, the MR-AI model was established entirely on the synthetic simulation data. Then, a μCT scan of a physically molded pin bracket was used to create a finite element model that provided data for additional validation of the DCNN model. For the μCT scan finite element pin bracket the MR-AI model predicted the distribution of fiber orientation tensor components with MAE of 0.10 indicating a global prediction error of 10%. For the flow simulated pin bracket, the MR-AI model predicted the distribution of fiber orientation tensor components with a global prediction error of 11%. The MR-AI model showed the ability to predict regions of varying alignment in the base and flange of the pin bracket. The proposed MR-AI methodology allows for rapid prediction of FOD in geometrically complex parts and offers a promising path to detecting unique fiber orientation states in molded components.

42 ENGINEERING↗
DOE OSTI2026

Improving neutrino oscillation measurements through event classification

Precise neutrino energy reconstruction is essential for next-generation long-baseline oscillation experiments, yet current methods remain limited by large uncertainties in neutrino-nucleus interaction modeling. Even so, it is well established that different interaction channels produce systematically varying amounts of missing energy and therefore yield different reconstruction performance–information that standard calorimetric approaches do not exploit. We introduce a strategy that incorporates this structure by classifying events according to their underlying interaction type prior to energy reconstruction. Using supervised machine-learning techniques trained on labeled generator events, we leverage intrinsic kinematic differences among quasielastic scattering, meson-exchange current, resonance production, and deep-inelastic scattering processes. A cross-generator testing framework demonstrates that this classification approach is robust to microphysics mismodeling and, when applied to a simulated DUNE 𝜈 𝜇 disappearance analysis, yields improved accuracy and sensitivity at the 10%–20% level. These results highlight a practical path toward reducing reconstruction-driven systematics in future oscillation measurements.

Ellis, Sebastian A. R. [King's College, London (Un↗
DOE OSTI2021

Multiscale Data-Driven Seismic Full-Waveform Inversion With Field Data Study

Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a nonconvex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCNs). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structures to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. Finally, these numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time and yield more accurate subsurface velocity models in comparison with conventional FWI.

58 GEOSCIENCES↗
DOE OSTI2022

Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements

An emerging use-case of machine learning (ML) is to train a model on a high-performance system and deploy the trained model on energy-constrained embedded systems. Neuromorphic hardware platforms, which operate on principles of the biological brain, can significantly lower the energy overhead of a machine learning inference task, making these platforms an attractive solution for embedded ML systems. In this paper, we present a design-technology tradeoff analysis to implement such inference tasks on the processing elements (PEs) of a Non-Volatile Memory (NVM)-based neuromorphic hardware. Through detailed circuit-level simulations at scaled process technology nodes, we show the negative impact of technology scaling on the information-processing latency, which impacts the quality-of-service (QoS) of an embedded ML system. At a finer granularity, the latency inside a PE depends on 1) the delay introduced by parasitic components on its current paths, and 2) the varying delay to sense different resistance states of its NVM cells. Based on these two observations, we make the following three contributions. First, on the technology front, we propose an optimization scheme where the NVM resistance state that takes the longest time to sense is set on current paths having the least delay, and vice versa, reducing the average PE latency, which improves the QoS. Second, on the architecture front, we introduce isolation transistors within each PE to partition it into regions that can be individually power-gated, reducing both latency and energy. Finally, on the system-software front, we propose a mechanism to leverage the proposed technological and architectural enhancements when implementing a machine-learning inference task on neuromorphic PEs of the hardware. Evaluations with a recent neuromorphic hardware architecture show that our proposed design-technology co-optimization approach improves both performance and energy efficiency of machine-learning inference tasks without incurring high cost-per-bit.

42 ENGINEERING↗