Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Learning curves for drug response prediction in cancer cell lines

Motivated by the size and availability of cell line drug sensitivity data, researchers have been developing machine learning (ML) models for predicting drug response to advance cancer treatment. As drug sensitivity studies continue generating drug response data, a common question is whether the generalization performance of existing prediction models can be further improved with more training data. We utilize empirical learning curves for evaluating and comparing the data scaling properties of two neural networks (NNs) and two gradient boosting decision tree (GBDT) models trained on four cell line drug screening datasets. The learning curves are accurately fitted to a power law model, providing a framework for assessing the data scaling behavior of these models. The curves demonstrate that no single model dominates in terms of prediction performance across all datasets and training sizes, thus suggesting that the actual shape of these curves depends on the unique pair of an ML model and a dataset. The multi-input NN (mNN), in which gene expressions of cancer cells and molecular drug descriptors are input into separate subnetworks, outperforms a single-input NN (sNN), where the cell and drug features are concatenated for the input layer. In contrast, a GBDT with hyperparameter tuning exhibits superior performance as compared with both NNs at the lower range of training set sizes for two of the tested datasets, whereas the mNN consistently performs better at the higher range of training sizes. Moreover, the trajectory of the curves suggests that increasing the sample size is expected to further improve prediction scores of both NNs. These observations demonstrate the benefit of using learning curves to evaluate prediction models, providing a broader perspective on the overall data scaling characteristics. A fitted power law learning curve provides a forward-looking metric for analyzing prediction performance and can serve as a co-design tool to guide experimental biologists and computational scientists in the design of future experiments in prospective research studies.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2004

Examples of Current and Future Uses of Neural-Net Image Processing for Aerospace Applications

Feed forward artificial neural networks are very convenient for performing correlated interpolation of pairs of complex noisy data sets as well as detecting small changes in image data. Image-to-image, image-to-variable and image-to-index applications have been tested at Glenn. Early demonstration applications are summarized including image-directed alignment of optics, tomography, flow-visualization control of wind-tunnel operations and structural-model-trained neural networks. A practical application is reviewed that employs neural-net detection of structural damage from interference fringe patterns. Both sensor-based and optics-only calibration procedures are available for this technique. These accomplishments have generated the knowledge necessary to suggest some other applications for NASA and Government programs. A tomography application is discussed to support Glenn's Icing Research tomography effort. The self-regularizing capability of a neural net is shown to predict the expected performance of the tomography geometry and to augment fast data processing. Other potential applications involve the quantum technologies. It may be possible to use a neural net as an image-to-image controller of an optical tweezers being used for diagnostics of isolated nano structures. The image-to-image transformation properties also offer the potential for simulating quantum computing. Computer resources are detailed for implementing the black box calibration features of the neural nets.

Decker, Arthur J.↗
NASA NTRS2018

Automatic Speech Recognition for Launch Control Center Communication Using Recurrent Neural Networks with Data Augmentation and Custom Language Model

Transcribing voice communications in NASA’s launch control center is important for information utilization. However, automatic speech recognition in this environment is particularly challenging due to the lack of training data, unfamiliar words in acronyms, multiple different speakers and accents, and conversational characteristics of speaking. We used bidirectional deep recurrent neural networks to train and test speech recognition performance. We showed that data augmentation and custom language models can improve speech recognition accuracy. Transcribing communications from the launch control center will help the machine analyze information and accelerate knowledge generation.

Chow, Edward↗
DOE OSTI2024

Statistical data analysis of x-ray spectroscopy data enabled by neural network accelerated Bayesian inference

Bayesian inference applied to x-ray spectroscopy data analysis enables uncertainty quantification necessary to rigorously test theoretical models. However, when comparing to data, detailed atomic physics and radiation transfer calculations of x-ray emission from non-uniform plasma conditions are typically too slow to be performed in line with statistical sampling methods, such as Markov Chain Monte Carlo sampling. Furthermore, differences in transition energies and x-ray opacities often make direct comparisons between simulated and measured spectra unreliable. Here, we present a spectral decomposition method that allows for corrections to line positions and bound–bound opacities to best fit experimental data, with the goal of providing quantitative feedback to improve the underlying theoretical models and guide future experiments. In this work, we use a neural network (NN) surrogate model to replace spectral calculations of isobaric hot-spots created in Kr-doped implosions at the National Ignition Facility. The NN was trained on calculations of x-ray spectra using an isobaric hot-spot model post-processed with Cretin, a multi-species atomic kinetics and radiation code. The speedup provided by the NN model to generate x-ray emission spectra enables statistical analysis of parameterized models with sufficient detail to accurately represent the physical system and extract the plasma parameters of interest.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

FAIR Surrogate Benchmarks Supporting AI and Simulation Research (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia (UVA). SBI repositories include data, code, and all relevant collateral artifacts that the science and engineering community need to use and reuse these data sets and surrogates. SBI repositories generate active research from both the participants in SBI and the broad community of AI and domain scientists. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and captures them as surrogate benchmarks with a rich set of metadata covering: Data; Model; Metrics specification; Machine specification; and Science, Speed, and Power Results. We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non-Surrogate benchmarks that have many common features and similar issues as regards FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, Benchmarks have datasets, models, and metadata and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Neural Network-Based Electric Vehicle Range Prediction for Smart Charging Optimization

Range prediction is a standard feature in most modern road vehicles, allowing drivers to make informed decisions about when to refuel. Most vehicles make range predictions through data- or model-driven means, monitoring the average fuel consumption rate or using a tuned vehicle model to predict fuel consumption. The uncertainty of future driving conditions makes the range prediction problem challenging, particularly for less pervasive battery electric vehicles (BEV). Most contemporary machine learning-based methods attempt to forecast the battery SOC discharge profile to predict vehicle range. In this work, we propose a novel approach using two recurrent neural networks (RNNs) to predict the remaining range of BEVs and the minimum charge required to safely complete a trip. Each RNN has two outputs that can be used for statistical analysis to account for uncertainties; the first loss function leads to mean and variance estimation (MVE), while the second results in bounded interval estimation (BIE). These outputs of the proposed RNNs are then used to predict the probability of a vehicle completing a given trip without charging, or if charging is needed, the remaining range and minimum charging required to finish the trip with high probability. Training data was generated using a low-order physics model to estimate vehicle energy consumption from historical drive cycle data collected from medium-duty last-mile delivery vehicles. Here, the proposed method demonstrated high accuracy in the presence of day-to-day route variability, with the root-mean-square error (RMSE) below 6% for both RNN models.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Finding Real Uncertainties From Physical Simulations

Modeling strong gravitational lenses is computationally expensive for the complex data from modern and next-generation cosmic surveys. Deep learning has emerged as a promising approach for finding lenses and predicting lensing parameters, such as the Einstein radius. Mean-variance Estimators (MVEs) are a common approach for obtaining aleatoric (data) uncertainties from a neural network prediction. However, neural networks have not been demonstrated to perform well on out-of-domain target data successfully - e.g., when trained on simulated data and applied to real, observational data. In this work, we perform the first study of the efficacy of MVEs in combination with unsupervised domain adaptation (UDA) on strong lensing data. The source domain data is noiseless, and the target domain data has noise mimicking modern cosmology surveys. We find that adding UDA to MVE increases the accuracy on the target data by a factor of about two over an MVE model without UDA. Including UDA also permits much more well-calibrated aleatoric uncertainty predictions. Advancements in this approach may enable future applications of MVE models to real observational data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Method to simultaneously facilitate all jet physics tasks

Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that specially constructed machine learning models trained for a specific jet classification task can improve the accuracy, precision, or speed of all other jet physics tasks. This is demonstrated by training on a particular multiclass generation and classification task and then using the learned representation for different generation and classification tasks, for datasets with a different (full) detector simulation, for jets from a different collision system ($pp$ versus $ep$), for generative models, for likelihood ratio estimation, and for anomaly detection. We consider our omnilearn approach thus as a jet-physics foundation model. It is made publicly available for use in any area where state-of-the-art precision is required for analyses involving jets and their substructure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Converting tabular data into images for deep learning with convolutional neural networks

Abstract Convolutional neural networks (CNNs) have been successfully used in many applications where important information about data is embedded in the order of features, such as speech and imaging. However, most tabular data do not assume a spatial relationship between features, and thus are unsuitable for modeling using CNNs. To meet this challenge, we develop a novel algorithm, image generator for tabular data (IGTD), to transform tabular data into images by assigning features to pixel positions so that similar features are close to each other in the image. The algorithm searches for an optimized assignment by minimizing the difference between the ranking of distances between features and the ranking of distances between their assigned pixels in the image. We apply IGTD to transform gene expression profiles of cancer cell lines (CCLs) and molecular descriptors of drugs into their respective image representations. Compared with existing transformation methods, IGTD generates compact image representations with better preservation of feature neighborhood structure. Evaluated on benchmark drug screening datasets, CNNs trained on IGTD image representations of CCLs and drugs exhibit a better performance of predicting anti-cancer drug response than both CNNs trained on alternative image representations and prediction models trained on the original tabular data.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

NEUROSPF: A Tool For the Symbolic Analysis of Neural Networks

This paper presents NEUROSPF, a tool for the symbolic analysis of neural networks. Given a trained neural network model, the tool extracts the architecture and model parameters and translates them into a Java representation that is amenable for analysis using the Symbolic PathFinder symbolic execution tool. Notably, NEUROSPF encodes specialized peer classes for parsing the model’s parameters, thereby enabling efficient analysis. With NEUROSPF the user has the flexibility to specify either the inputs or the network internal parameters as symbolic, promoting the application of program analysis and testing approaches from software engineering to the field of machine learning. For instance, NEUROSPF can be used for coverage-based testing and test generation, finding adversarial examples and also constraint-based repair of neural networks, thus improving the reliability of neural networks and of the applications that use them.

neural networks↗
DOE OSTI2020

SO(3)-invariance of informed-graph-based deep neural network for anisotropic elastoplastic materials

This work examines the frame-invariance (and the lack thereof) exhibited in simulated anisotropic elasto-plastic responses generated from supervised machine learning of classical multi-layer and informed-graph-based neural networks, and proposes different remedies to fix this drawback. The inherent hierarchical relations among physical quantities and state variables in an elasto-plasticity model are first represented as informed, directed graphs, where three variations of the graph are tested. While feed-forward neural networks are used to train path-independent constitutive relations (e.g., elasticity), recurrent neural networks are used to replicate responses that depends on the deformation history, i.e. or path dependent. In dealing with the objectivity deficiency, we use the spectral form to represent tensors and, subsequently, three metrics, the Euclidean distance between the Euler Angles, the distance from the identity matrix, and geodesic on the unit sphere in Lie algebra, can be employed to constitute objective functions for the supervised machine learning. In this, the aim is to minimize the measured distance between the true and the predicted 3D rotation entities. Following this, we conduct numerical experiments on how these metrics, which are theoretically equivalent, may lead to differences in the efficiency of the supervised machine learning as well as the accuracy and robustness of the resultant models. Neural network models trained with tensors represented in component form for a given Cartesian coordinate system are used as a benchmark. Our numerical tests show that, even given the same amount of information and data, the quality of the anisotropic elasto-plasticity model is highly sensitive to the way tensors are represented and measured. The results reveal that using a loss function based on geodesic on the unit sphere in Lie algebra together with an informed, directed graph yield significantly more accurate rotation prediction than the other tested approaches.

42 ENGINEERING↗
DOE OSTI2022

Short-term solar radiation forecast using total sky imager via transfer learning

Ground-based sky cameras, which capture hemispherical images, have been extensively used for localized monitoring of clouds. This paper proposes a short-term forecasting approach based on transfer learning using Total Sky-Imager (TSI) images of the Southern Great Plains (SGP) site obtained from the Atmospheric Radiation Measurement (ARM) dataset. An accurate estimation of solar irradiance using TSI is key for short-term solar energy generation forecasting and optimal energy consumption planning. We make use of deep neural network architectures such as AlexNet and ResNet-101 to extract the underlying deep convolution features from TSI images and then train using an ensemble learning approach to model and forecast solar radiation. We demonstrate the performance of the proposed approach by showcasing the best and worst cases. Thus, the transfer learning approach significantly reduces the time and resources required for modeling solar radiation. We outperform with reference to another state-of-art technique for solar modeling using TSI images at different forecast lead times.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Deep-learning-enhanced assessment of wellbore barrier effectiveness in geologic storage systems with intermediate aquifers

For geologic systems where carbon dioxide (CO 2 ) is injected underground, existing wells represent potential pathways for fluid migration. Here, this study introduces a novel deep learning model to quantify the likelihood and potential magnitude of fluid migration through wellbores at sites with intermediate aquifers or thief zones between the injection units and underground drinking water sources. Synthetic datasets, generated using reservoir simulations, captured a wide range of subsurface conditions, well attributes, operational parameters, and fluid migration scenarios. Among the regression models developed to predict brine and CO 2 leakage rates and CO 2 saturations along leaky wellbores, convolutional neural network (CNN) outperformed both Light Gradient Boosting Machine and deep neural network. Additionally, a CNN-based classification model was created to predict whether brine and CO 2 would leak along a wellbore, further improving performance over regression alone. The best models were integrated into the National Risk Assessment Partnership Open-source Integrated Assessment Model for rapid, stochastic assessment of storage system containment and leakage risks. A case study demonstrated the model’s ability to simulate fluid migration through existing wells with multiple intermediate aquifers. This computationally efficient wellbore model offers value in support of site performance evaluation and risk-informed decision making by stakeholders.

CO2 leakage↗
DOE OSTI2022

Improving Predictions Under Uncertainty of Material Plasma Device Operations

Understanding the properties of materials when exposed to various plasma temperatures and fluxes is essential to the building and operating of fusion reactors. The Material Plasma Exposure eXperiment (MPEX) is an instrument currently being developed by the Department of Energy (DOE) for this purpose. MPEX is expected to come online in stages over the next five years. Proto-MPEX, the predecessor to MPEX, operated from 2014 to 2021, and was designed to understand the generation of plasma temperatures and fluxes at orders of magnitude below what will be obtained by MPEX. This work uses the recently developed stochastic neural network (SNN), a machine learning technique capable of operating under uncertainty to provide a surrogate model for the Proto-MPEX device. We demonstrate that SNN outperforms Bayesian neural network (BNN), a standard in the field of machine learning with uncertainty. The development of a robust surrogate of the Proto-MPEX will aid in the commissioning and operation of the MPEX device.

Archibald, Rick↗
DOE OSTI2020

A Deep Generative Model for Non-Intrusive Identification of EV Charging Profiles

The proliferation of electric vehicles (EVs) brings environmental benefits and technical challenges to power grids. An identification algorithm which can accurately extract individual EV charging profiles out of widely available smart meter measurements has attracted great interests. This paper proposes a non-intrusive identification framework for EV charging profile extraction, which is driven by deep generative models (DGM). First, the proposed DGM is designed as a representation layer embedded into the Markov process and used to model the joint probability distribution of available time-series data. A novel contribution is to approximate posterior distributions by neural networks whose parameters are obtained by variational inference and supervised learning. Second, the EV charging status is inferred from the DGM via dynamic programming. Lastly, the desired EV charging profile can be reconstructed by the rated power of EV models and inferred status. Compared with the benchmark Hidden Markov Models, the proposed framework can better handle noise in data with less computational complexity and better overall accuracy performances with smaller recall. The proposed framework is validated by numerical experiments on the Pecan Street dataset.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2026

Iterative ML and Experiments for Emerging VOCs

SAND2026-17074O Iterative ML and Experiments for Emerging VOCs is a tool that analyzes and predicts the behaviors of SARS-CoV-2 variants. It processes experimental data on ACE2 (the receptor for the SARS-CoV-2 virus that allows it to infect the cell) and antibody binding using machine learning models, including neural networks, to forecast ACE2 interactions and variant expression. The tool employs transfer learning and global epistasis modeling, integrating public datasets with proprietary data to enhance prediction accuracy. Additionally, it fits concentration-response curves to determine dissociation constants and generates visualizations to support research findings, thereby aiding in the identification of new antibodies for emerging variants of concern. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Sheffield, Thomas [Sandia National Lab. (SNL-NM), ↗
DOE OSTI2020

Improving High-Energy Particle Detectors with Machine Learning

Microseconds after the Big Bang, the universe existed in a state called the quark-gluon plasma (QGP). To experimentally study its properties, the QGP is recreated in high-energy nuclear collisions at the LHC, and the particles produced from the QGP are reconstructed from their energy deposition in the ATLAS calorimeter. This requires both classifying the particles and calibrating their deposited energy. The objective of this project is to improve the reconstruction by using machine learning techniques, where the energy depositions of clusters of cells, formed by ATLAS topo-clustering methods, are treated as three-dimensional images when inputted to neural networks. This approach significantly improves the calibration of deposited energies when cross-validating while training, and models trained on idealized data predict the calibrated energies of particles in more complex data sets well. Additionally, implementation of a data generator using uproot allows the program to load input data into memory as needed while training or predicting, significantly reducing the amount of memory used. The data generator also allows for use of multiprocessing to speed up training and evaluating. This work illustrates that using machine learning methods for both classification and calibration has the potential to significantly improve particle reconstruction.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗