Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Hypothesis testing via AI: Generating physically interpretable models of scientific data with machine learning (Full Technical Report)

Deep learning has demonstrated an exceptional ability to solve complex tasks (an engineering success); however, it has done so at the expense of the ability to generate new knowledge (a scientific failure). We propose an alternative framework—entitled Deep Symbolic Regression (DSR)—in which artificial neural networks (NNs) rapidly generate hypotheses about physical relationships among inputs. This framework bypasses the need to interpret an NN altogether, while still leveraging the representational power of deep learning. The resulting models are tractable mathematical expressions, which are inherently and readily human interpretable and can provide insights into underlying physical phenomena. Further, we fold this methodology into the scientific process by allowing the scientist to directly integrate a priori knowledge and beliefs to accelerate learning. We demonstrate this methodology on symbolic regression—the problem of rediscovering underlying expressions describing a dataset—and achieve state-of-the-art performance across a wide variety of symbolic regression problems. Further, we generalize our DSR framework to apply to the more general class of symbolic optimization problems, in which one seeks to optimize a sequence of symbols or “tokens” under a black-box reward function. Examples of other symbolic optimization problems include neural architecture search and computational antibody design. Our generalized tool, Deep Symbolic Optimization (DSO), has been demonstrated on the task of learning symbolic control policies for reinforcement learning environments, and has been adopted as an enabling capability for computational antibody design.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A deep generative model for deciphering cellular dynamics and in silico drug discovery in complex diseases

Human diseases are characterized by intricate cellular dynamics. Single-cell transcriptomics provides critical insights, yet a persistent gap remains in computational tools for detailed disease progression analysis and targeted in silico drug interventions. Here we introduce UNAGI, a deep generative neural network tailored to analyse time-series single-cell transcriptomic data. This tool captures the complex cellular dynamics underlying disease progression, enhancing drug perturbation modelling and screening. When applied to a dataset from patients with idiopathic pulmonary fibrosis, UNAGI learns disease-informed cell embeddings that sharpen our understanding of disease progression, leading to the identification of potential therapeutic drug candidates. Validation using proteomics reveals the accuracy of UNAGI’s cellular dynamics analysis, and the use of the fibrotic cocktail-treated human precision-cut lung slices confirms UNAGI’s predictions that nifedipine, an antihypertensive drug, may have anti-fibrotic effects on human tissues. UNAGI’s versatility extends to other diseases, including COVID, demonstrating adaptability and confirming its broader applicability in decoding complex cellular dynamics beyond idiopathic pulmonary fibrosis, amplifying its use in the quest for therapeutic solutions across diverse pathological landscapes.

Neural Network↗
DOE OSTI2020

Big Sky Regional Carbon Sequestration Partnership (Phase III Final Scientific/Technical Report)

The Big Sky Carbon Sequestration Partnership (BSCSP) pursued a Phase III demonstration project at Kevin Dome in north central Montana. Kevin Dome covers approximately 700 square miles and is a naturally occurring CO 2 reservoir that is flanked by oil and gas fields. The carbon dioxide (CO 2 ) is in the upper Devonian Duperow (carbonate) formation and does not reach the spill point of the dome; therefore, the dome has potential as a CO 2 sequestration reservoir, a CO 2 supply, or as both if anthropogenic sources and enhanced oil recovery (EOR) operations are associated with the dome. Kevin Dome could potentially act as a buffer to continue accepting anthropogenic CO 2 when EOR flooding operations are interrupted or completed. The project objective was to produce one million tonnes of CO 2 from the gas cap of Kevin Dome, pipe it laterally, inject, and re-store it in the brine leg of the same formation to test the hub / buffer storage concept. This was to be accomplished by drilling up to five production wells, building a short pipeline and compression facilities, and drilling an injection well and several monitoring wells. BSCSP commenced outreach and site characterization activities including acquiring baseline data for near-surface insurance monitoring, acquiring 3-dimensional, 9-component surface seismic over the project area, drilling two test wells (one in the production area and one in the injection area), coring key intervals, and performing comprehensive logging. Well tests of those wells revealed two barriers to the project. The production (Danielson 33-17) well was perforated in multiple zones but failed to produce any significant CO 2 . This was despite being drilled in the near vicinity of a historic well that had produced 3700 MCF per day in a drill stem test. Modeling indicated that this was likely due to a phase change during production causing a temperature drop resulting in hydrate and/or water ice formation that clogged the formation. Tests of the injection zone (Wallewein 22-1) well indicated total dissolved solids (TDS) slightly below the EPA required 10,000 parts per million (ppm) for a Class VI underground injection control permit. While the project was initiated before Class VI rules were promulgated, and this was an experimental project (seemingly qualified for a Class V permit), the Environmental Protection Agency (EPA) indicated that injection would require a Class VI permit. The low salinity result was unexpected as contours plotted based on regional formation water quality data indicated an expected TDS above 20,000 ppm, and wells between the recharge zone and the Wallewein well tested above 10,000 ppm. Faced with the inability to obtain an injection permit, the demonstration project could not proceed. However, the project had generated valuable samples and data on a large natural analog including 32 sq. mi. of 3-D, 9-C seismic, 430 ft. of carbonate core covering seven different depositional environments taken from areas with, and without the presence of CO 2 , 30 ft. of core of two caprocks, a tight carbonate and an anhydrite, a full set of modern logs on both wells, and well tests. DOE decided to re-scope the project around completing studies utilizing this data. This report covers both the initial scope and the re-scope (Task / Section numbers preceded with an R). While the report covers a wide range of project activities, highlights of this work include: Development of a geostatic model using neural nets to match well logs to facies and using multi-waveform seismic to inform reservoir heterogeneity; Unique mechanical testing of permeability – stress relationship in two caprock materials; Development of full waveform inversion to generate a high resolution velocity model; Model development for dual permeability (fracture and matrix) systems to better account for matrix-matrix interactions; Joint seismic wave inversion (including the first quadr-joint inversion) exhibiting better imaging of a challenging reservoir zone in stiff rock; Core flow and core flood results on a reactive carbonate; and Innovative laboratory measurements of seismic response of fractured core as a function of fluid fill.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

wa-hls4ml and lui-gnn: A benchmark and GNN-based surrogate model for hls4ml resource and latency estimation

As machine learning (ML) increasingly serves as a tool for addressing real-time challenges in scientific applications, the development of advanced tooling has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as model synthesis, are now becoming limiting factors in the rapid iteration of designs. To reduce these emerging constraints, multiple efforts are being launched toward designing an ML-based surrogate model that estimates resource usage of synthesized accelerator architectures. This model would reduce the design iteration time, especially when designing within a set of given hardware constraints. This approach shows considerable potential, but as it stands, the effort is early and would benefit from coordination and standardization to assist future work as it emerges. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of more than 100,000 fully connected neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. In addition to the resource utilization and latency data provided, the dataset includes generated artifacts and log files for many of the synthesized neural networks, in order to support future research in ML-based code generation. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, as well as the average performance across a subset of the dataset. We measure the performance of a given predictor model through multiple metrics, including $R^2$ score and SMAPE on regression tasks, as well as inference time to further characterize the estimator under test. Additionally, we introduce the latency/utilization inference graph neural network (lui-gnn), a surrogate model that uses a graph neural network to represent input architectures in the form of a directed graph. This graph representation allows for a diverse set of model architectures to all be effectively handled by a surrogate model. We present the architecture and performance of the model, as evaluated by the new proposed benchmark, including SMAPE, $R^2$ score, and inference times, and find that lui-gnn generally predicts latency and utilization for the 75\% quantile within several percent of the synthesized resources on the synthetic test dataset, indicating that this approach of estimating resource and latency via a surrogate models has promise and warrants further research.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Neural Networks-Based Inverter Control: Modeling and Adaptive Optimization for Smart Distribution Networks

The optimal voltage control of inverter-based resources, especially under the high penetration of solar photovoltaics, is critical to the stability of the distribution power system. However, the computational complexity as well as the coordinated operation performance of the voltage control optimization in the distribution power system limits the real-time applications. To mitigate this issue, a model-free based adaptive optimal control scheme for the smart inverter is proposed to maximize the active power generation, minimize the power loss, and maintain the bus voltages in smart distribution networks. An inverter-based optimization model for coordinated operation is first established, considering the uncertainties of renewable power generation. Subsequently, by collecting the data and control strategies, the neural networks (NNs) based algorithm is proposed to efficiently predict the best possible control strategy. The main objective of this scheme is to accurately predict candidate optimal solutions with near-negligible feasibility and optimization gaps, with the advantage of avoiding complicated iteration-based numerical algorithms. Thereafter, the co-simulation among OpenDSS, MATLAB, and Python is set up to fully take advantage of the three individual software. Experiments are conducted based on different control parameter characteristics and structures of NNs. Finally, the results reveal that an average mean squared error of 0.013 and 1 ms response time are achieved, which is lower than some state-of-the-art methods.

42 ENGINEERING↗
DOE OSTI2022

Adsorbate chemical environment-based machine learning framework for heterogeneous catalysis

Abstract Heterogeneous catalytic reactions are influenced by a subtle interplay of atomic-scale factors, ranging from the catalysts’ local morphology to the presence of high adsorbate coverages. Describing such phenomena via computational models requires generation and analysis of a large space of atomic configurations. To address this challenge, we present Adsorbate Chemical Environment-based Graph Convolution Neural Network (ACE-GCN), a screening workflow that accounts for atomistic configurations comprising diverse adsorbates, binding locations, coordination environments, and substrate morphologies. Using this workflow, we develop catalyst surface models for two illustrative systems: (i) NO adsorbed on a Pt 3 Sn(111) alloy surface, of interest for nitrate electroreduction processes, where high adsorbate coverages combined with low symmetry of the alloy substrate produce a large configurational space, and (ii) OH* adsorbed on a stepped Pt(221) facet, of relevance to the Oxygen Reduction Reaction, where configurational complexity results from the presence of irregular crystal surfaces, high adsorbate coverages, and directionally-dependent adsorbate-adsorbate interactions. In both cases, the ACE-GCN model, trained on a fraction (~10%) of the total DFT-relaxed configurations, successfully describes trends in the relative stabilities of unrelaxed atomic configurations sampled from a large configurational space. This approach is expected to accelerate development of rigorous descriptions of catalyst surfaces under in-situ conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

What’s the Difference? The Potential for Convolutional Neural Networks for Transient Detection without Template Subtraction

Abstract We present a study of the potential for convolutional neural networks (CNNs) to enable separation of astrophysical transients from image artifacts, a task known as “real–bogus” classification, without requiring a template-subtracted (or difference) image, which requires a computationally expensive process to generate, involving image matching on small spatial scales in large volumes of data. Using data from the Dark Energy Survey, we explore the use of CNNs to (1) automate the real–bogus classification and (2) reduce the computational costs of transient discovery. We compare the efficiency of two CNNs with similar architectures, one that uses “image triplets” (templates, search, and difference image) and one that takes as input the template and search only. We measure the decrease in efficiency associated with the loss of information in input, finding that the testing accuracy is reduced from ∼96% to ∼91.1%. We further investigate how the latter model learns the required information from the template and search by exploring the saliency maps. Our work (1) confirms that CNNs are excellent models for real–bogus classification that rely exclusively on the imaging data and require no feature engineering task and (2) demonstrates that high-accuracy (>90%) models can be built without the need to construct difference images, but some accuracy is lost. Because, once trained, neural networks can generate predictions at minimal computational costs, we argue that future implementations of this methodology could dramatically reduce the computational costs in the detection of transients in synoptic surveys like Rubin Observatory's Legacy Survey of Space and Time by bypassing the difference image analysis entirely.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Accelerating Hydraulic Fracture Imaging by Deep Transfer Learning

Deep transfer learning has a great success story in computer vision (CV), natural language processing, and many other fields. In this communication, we are going to push forward the deep transfer learning to the hydraulic fracture imaging problem by proposing a two-step approach: 1) train a convolutional neural network (CNN) to reconstruct target geometries by a relatively large amount of approximated field patterns generated from a simplified model and 2) fine-tune the top layers of transferred CNN by a small amount of true field patterns generated through a full model. The advantages include the rapid generation of large amount of data through the simplified model and the high reconstruction accuracy through a careful design of the deep transfer learning. Here, the CNN trained through the deep transfer learning can accurately reconstruct the lateral extent and direction of fractures with unseen conductivity and white Gaussian noise, showing a notable acceleration/accuracy enhancement over the previous CNN trained by mixing a small number of true data into the approximated data as data augmentation.

02 PETROLEUM↗
DOE OSTI2024

Understanding Generative AI Content with Embedding Models

The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.

Vargas, Max↗
DOE OSTI2019

Data and Models: A Framework for Advancing AI in Science

On June 5, 2019, the Office of Science (SC) organized a one-day roundtable to focus on enhancing access to high-quality and fully traceable research data, models, and computing resources to increase the value of such resources for artificial intelligence (AI) research and development and the SC mission.1 In this report, we consider AI to be inclusive of, for example, machine learning (ML), deep learning (DL), neural networks (NN), computer vision, and natural language processing (NLP). We consider “data for AI” to mean the digital artifacts used to generate AI models and/or employed in combination with AI models during inference. In part, this roundtable was motivated by the recognition that a large portion of science data currently are not well suited for AI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

CFD-trained ANN Model for Approximating Near-occupant Condition in Real-time Simulations

The main drawback of Computational Fluid Dynamics (CFD) simulations has been the time and resource consuming nature which is not suitable for real-time applications. In this work, we first generated numerous CFD models of a given indoor space to obtain airspeed, temperature, and mean radiant temperature near an occupant as training data. Several artificial neural networks (ANN) models were trained using this CFD simulated data to approximate near real-time environmental conditions for a given occupant. This trained ANN model approach is a part of a real-time simulation of building operations using a combination of software and real hardware (HVAC equipment) approaches. The preliminary results suggest that the CFD- generated training data and the trained ANN model can accurately approximate such conditions in a real-time application, a method that has great potential in building simulation and building digital twin areas of research.

Zhang, Yun↗
DOE OSTI2025

A generative artificial intelligence framework for long-time plasma turbulence simulations

Generative deep learning techniques are employed in a novel framework for the construction of surrogate models capturing the spatiotemporal dynamics of 2D plasma turbulence. The proposed Generative Artificial Intelligence Turbulence (GAIT) framework enables the acceleration of turbulence simulations for long-time transport studies. GAIT leverages a convolutional variational auto-encoder and a recurrent neural network to generate new turbulence data from existing simulations, extending the time horizon of transport studies with minimal computational cost. The application of the GAIT framework to plasma turbulence using the Hasegawa–Wakatani (HW) model is presented, evaluating its performance via various analyses. Very good agreement is found between the GAIT and the HW models in the spatiotemporal Fourier and Proper Orthogonal Decomposition spectra, the flow topology characterized by the Okubo–Weiss parameter, and the time autocorrelation function of turbulent fluctuations. Excellent agreement has also been obtained in the probability distribution function of particle displacements and the effective turbulent diffusivity. In-depth analyses of the latent space of turbulent states, choice of hyperparameters and alternative deep learning models for the time prediction are presented. Our results highlight the potential of Artificial Intelligence-based surrogate models to overcome the computational challenges in turbulence simulation, which can be extended to other situations such as geophysical fluid dynamics.

Artificial intelligence↗
DOE OSTI2024

Accelerated CO2 Storage Optimization Using Multi-Resolution Fourier Neural Operator at the Illinois Basin Decatur Project (IBDP)

This paper presents a deep learning-based approach for optimizing CO2 injection in carbon capture and storage (CCS) operations. We developed a multi-resolution machine learning model to significantly reduce data generation costs. Utilizing this proxy model, we implemented a multi-objective genetic algorithm to optimize well control during the CO2 injection process. The proposed approach was applied to the Illinois Basin Decatur Project (IBDP), successfully optimizing the CO2 injection schedule based on three key objectives: maximizing the amount of CO2 stored, maximizing sweep efficiency, and minimizing pressure increase. The use of the proxy model accelerated the optimization workflow by two orders of magnitude, while the cost of data generation for the proxy model was reduced by 90% by utilizing a coarse-scale model.

accelerated CO2 storage optimization↗
DOE OSTI2025

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗
DOE OSTI2023

CCSI Toolset 3.19 Release

CCSI Toolset 3.19 Release Highlights A gradient generation tool was developed to support GENN models in FOQUS. Certain machine learning tools train gradient-enhanced neural network (GENN) models which can be more accurate for complex datasets given a priori knowledge of model derivatives. However, the derivatives must be known beforehand and are not often available for process data. This tool automatically predicts the gradients for a training dataset in a form usable by common GENN trainers, such as Surrogate Modeling Toolbox. Support was added for Surrogate Modeling Toolbox GENN models in FOQUS, including updates to the run methods, node properties, test framework, documentation and optional dependencies list. Users can train/save Surrogate Modeling Toolbox gradient-enhanced neural network (GENN) models with custom objects and produce .pkl files compatible with the Machine Learning/Artificial Intelligence Plugin in FOQUS. A simpler implementation of the ordering algorithm in the Sequential Design of Experiments (SDOE) module was included. The SDOE examples documentation was updated. The Optimality-Based Design of Experiments was updated to improve the error handling when the results are None.

AS↗
DOE OSTI2026

Optimized Gear Selection to Maximize Energy Savings in Electric Traction Drives for Medium and Heavy Duty Vehicles

Multi‑gear transmission systems are commonly used in electric traction drives for medium and heavy‑duty vehicles, while most passenger‑vehicle electric drivetrains rely on a single fixed ratio to reduce cost, weight, and complexity. Using multiple gear ratios can enable downsizing of the motor and inverter while still meeting performance requirements. Additionally, appropriately chosen ratios allow the motor to operate more frequently in high‑efficiency regions, improving overall energy usage and reducing operating costs over the drive cycle. This paper presents a systematic approach for selecting optimal gear ratios for electric drive systems. A neural‑network model is first developed to represent motor losses across the full torque–speed range using data generated from finite element analysis. This model enables fast, accurate evaluation of motor efficiency under varying operating conditions. A genetic‑algorithm‑based optimization framework is then applied to identify gear ratios that maximize energy cost savings over the drive cycle, with the resulting optimal ratios stored for real‑time implementation.

Gadiyar, Nishanth [ORNL] (ORCID:0000000348267524)↗
DOE OSTI2024

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗
DOE OSTI2023

NeRVI: Compressive neural representation of visualization images for communicating volume visualization results

We present NeRVI, a new deep-learning approach that compresses a large collection of visualization images generated from time-varying data for communicating volume visualization results. Based on an image-based implicit neural representation, our approach represents tens of thousands of high-resolution rendering images parametrized by different parameters via a hybrid model of multilayer perceptrons and convolutional neural networks. Here, our model predicts images and corresponding masks, and the masks are utilized for loss computation and network training to capture fine structural details and small components. In conjunction with model quantization and weight encoding, NeRVI yields highly compact compressive neural representations while preserving the image fidelity well. We demonstrate the effectiveness of NeRVI with isosurface rendering and direct volume rendering images generated from multiple data sets and compare NeRVI with other state-of-the-art deep learning-based (InSituNet, SIREN, NeRF, and NeRV) methods. Quantitative and qualitative results show that NeRVI provides an alternative solution that augments domain scientists' ability to manage, represent, and communicate scientific visualization output.

97 MATHEMATICS AND COMPUTING↗