SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Recovering the phase and amplitude of X-ray FEL pulses using neural networks and differentiable models

Dynamics experiments are an important use-case for X-ray free-electron lasers (XFELs), but time-domain measurements of the X-ray pulses themselves remain a challenge. Shot-by-shot X-ray diagnostics could enable a new class of simpler and potentially higher-resolution pump-probe experiments. Here, we report training neural networks to combine low-resolution measurements in both the time and frequency domains to recover X-ray pulses at high-resolution. Critically, we also recover the phase, opening the door to coherent-control experiments with XFELs. The model-based generative neural-network architecture can be trained directly on unlabeled experimental data and is fast enough for real-time analysis on the new generation of MHz XFELs.

42 ENGINEERING↗

DOE OSTI2024

Surfactant-Specific AI-Driven Molecular Design: Integrating Generative Models, Predictive Modeling, and Reinforcement Learning for Tailored Surfactant Synthesis

Molecular design is a critical aspect of various scientific and industrial fields, where the properties of molecules hold significant importance. In this study, a 3-fold methodology design is presented that leverages the power of generative artificial intelligence (AI), predictive modeling, and reinforcement learning to create tailored molecules with desired properties. This model synergistically combines deep learning techniques with Self-Referencing Embedded Strings (SELFIES) molecular representation to build a generative model that generates valid molecules and a graphical neural network model that accurately forecasts molecular properties. The Variational Autoencoder (VAE) coupled with reinforcement learning helps refine molecule generation based on targeted attributes. Data from an experimental study involving surfactants were used to test the framework. A validation of the structural integrity of the molecules generated was conducted, and Tanimoto similarities were used to quantify the similarity and diversity between the original and generated molecular structures. Also, saliency maps for the generated surfactants were produced to identify the features explaining the property values. Lastly, molecular dynamics simulations were used to validate the stability of the generated molecules. The results showed that the proposed framework can effectively produce valid molecules within the set property threshold value.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Generation and evaluation of synthetic patient data

Background: Machine learning (ML) has made a significant impact in medicine and cancer research; however, its impact in these areas has been undeniably slower and more limited than in other application domains. A major reason for this has been the lack of availability of patient data to the broader ML research community, in large part due to patient privacy protection concerns. High-quality, realistic, synthetic datasets can be leveraged to accelerate methodological developments in medicine. By and large, medical data is high dimensional and often categorical. These characteristics pose multiple modeling challenges. Methods: In this paper, we evaluate three classes of synthetic data generation approaches; probabilistic models, classification-based imputation models, and generative adversarial neural networks. Metrics for evaluating the quality of the generated synthetic datasets are presented and discussed. Results: While the results and discussions are broadly applicable to medical data, for demonstration purposes we generate synthetic datasets for cancer based on the publicly available cancer registry data from the Surveillance Epidemiology and End Results (SEER) program. Specifically, our cohort consists of breast, respiratory, and non-solid cancer cases diagnosed between 2010 and 2015, which includes over 360,000 individual cases. Conclusions: We discuss the trade-offs of the different methods and metrics, providing guidance on considerations for the generation and usage of medical synthetic data.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

LENS: Learning Enabled Network Synthesis

RTRC and UMD have developed novel machine learning based methods under the ARPA-E DIFFERENTIATE program for rapid acceleration of hypothesis generation in complex architecture design spaces involving both discrete choices of component inclusion and interconnection and continuous parametric decisions. The project named Learning Enabled Network Synthesis (LENS) further demonstrated the developed methods on challenging electrical power converter design problems by identifying the most suitable circuit topologies and simultaneously selecting the most appropriate components to achieve optimized design of power converter with improved performances. We demonstrated that LENS could enable exploration of very large design space of circuit topologies and components by addressing the limitations of conventional design process in non-linear, high switching speed, multi-dimensional power converter design and optimization. The key innovation developed in LENS is the seamless integration of statistical learning and logical reasoning techniques and building on the individual strengths of these techniques for rapid hypothesis discovery. The main component of LENS comprises of: 1) Graph Reasoning Engine (GRE) to enforce composition rules that rapidly reject all discrete architectures that are composed incorrectly and generates an adaptive database of feasible designs which can be used by ML modules, 2) Graph Generative Learning module which is a deep neural network based generative model for graph architectures which can enable design space exploration beyond the dataset generated by the GRE, 3) Graph Reduced Order Model (ROM) for graph domains for accelerating computation of output metrics, and 4) Active learning and Rule Discovery module for sample efficient learning and extracting logical rules from the learned ML models which will be integrated in the GRE to enhance the filtering effectiveness. LENS approach can be applied to any design domains where designs can be represented as multi-attribute graphs. The LENS team integrated the various technical innovations listed above into an optimization pipeline and exercised the optimization pipeline on the converter design problem. The LENS project demonstrated that the developed AI/ML technologies can be used to generate novel converter circuits >45x faster than experts on chosen use-cases. This can enable faster design space exploration and identification of new designs which are not considered by experts due to the increasing design space complexity. This has significant potential impact on the public and energy needs of the country. It is currently estimated that 30% of all electrical powers generated passes through power converters. The future estimate is that 80% of all power generated would be passing through converters. LENS fills a critical gap in this space since by accelerating the design process the designers would be able to generate more efficient converters which can lead to significant energy savings for the country.

42 ENGINEERING↗

DOE OSTI2022

Dynamic Parameter Estimation with Physics-based Neural Ordinary Differential Equations

Accurate estimation of dynamic parameters of gen-erators is crucial to building a reliable model for dynamical studies and reliable operation of the power system. This paper develops a physics-based neural ordinary differential equations (ODE) approach to learn the parameters of generator dynamic model using phasor measurement units (PMU) data. We design a physics-based neural network to represent the swing equations of the power system dynamics. A loss function is defined as the difference between dynamic simulation results from the physics-based neural networks and pseudo PMU measurements. The parameters of generator dynamic model are iteratively updated using the neural ODEs and the adjoint method. By exploiting the mini-batch scheme in neural ODE training, the parameter estimation performance is significantly improved. Numerical study results on a 3-machine 9-bus system show that the proposed algorithm outperforms state-of-the-art baseline method in both computation time and dynamic parameter estimation accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1999

Neural Network and Regression Approximations Used in Aircraft Design

NASA Lewis Research Center's CometBoards Test Bed was used to create regression and neural network models for a High-Speed Civil Transport (HSCT) aircraft. Both approximation models that replaced the actual analysis tool predicted the aircraft response in a trivial computational effort. The models allow engineers to quickly study the effects of design variables on constraint and objective values for a given aircraft configuration. For example, an engineer can change the engine size by 1000 pounds of thrust and quickly see how this change affects all the output values without rerunning the entire simulation. In addition, an engineer can change a constraint and use the approximation models to quickly reoptimize the configuration. Generating the neural network and the regression models is a time-consuming process, but this exercise has to be carried out only once. Furthermore, an automated process can reduce calculations substantially.

Patnaik, Surya N.↗

NASA NTRS1992

Identification of aerodynamic coefficients using computational neural networks

Precise, smooth aerodynamic models are required for implementing adaptive, nonlinear control strategies. Accurate representations of aerodynamic coefficients can be generated for the complete flight envelope by combining computational neural network models with an Estimation-Before-Modeling paradigm for on-line training information. A novel method of incorporating first-partial-derivative information is employed to estimate the weights in individual feedforward neural networks for each aerodynamic coefficient. The method is demonstrated by generating a model of the normal force coefficient of a twin-jet transport aircraft from simulated flight data, and promising results are obtained.

Linse, Dennis J.↗

DOE OSTI2026

Learning turbulent flows with generative models for super resolution and sparse flow reconstruction

Neural operators are promising surrogates for dynamical systems but when trained with standard L 2 losses they tend to oversmooth fine-scale turbulent structures. Here, we show that combining operator learning with generative modeling overcomes this limitation. We consider three practical turbulent-flow challenges where conventional neural operators fail: spatio-temporal super-resolution, forecasting, and sparse flow reconstruction. For Schlieren jet super-resolution, an adversarially trained neural operator (adv-NO) reduces the energy-spectrum error by 15 × while preserving sharp gradients at neural operator-like inference cost. For 3D homogeneous isotropic turbulence, adv-NO trained on only 160 timesteps from a single trajectory forecasts accurately for five eddy-turnover times and offers 114 × wall-clock speed-up at inference than the baseline diffusion-based forecasters, enabling near-real-time rollouts. For reconstructing cylinder wake flows from highly sparse Particle Tracking Velocimetry-like inputs, a conditional generative model infers full 3D velocity and pressure fields with correct phase alignment and statistics. These advances enable accurate reconstruction and forecasting at low compute cost, bringing near-real-time analysis and control within reach in experimental and computational fluid mechanics.

Fluid dynamics↗

DOE OSTI2023

A publicly available PyTorch-$\mathrm{ABAQUS}$ $\mathrm{UMAT}$ deep-learning framework for level-set plasticity

Here this paper introduces a publicly available PyTorch-ABAQUS deep-learning framework of a family of plasticity models where the yield surface is implicitly represented by a scalar-valued function. In particular, our focus is to introduce a practical framework that can be deployed for engineering analysis that employs a user-defined material subroutine (UMAT/VUMAT) for ABAQUS, which is written in FORTRAN. To accomplish this task while leveraging the back-propagation learning algorithm to speed up the neural-network training, we introduce an interface code where the weights and biases of the trained neural networks obtained via the PyTorch library can be automatically converted into a generic FORTRAN code that can be a part of the UMAT/VUMAT algorithm. To enable third-party validation, we purposely make all the data sets, source code used to train the neural-network-based constitutive models, and the trained models available in a public repository. Furthermore, the practicality of the workflow is then further tested on a dataset for anisotropic yield function to showcase the extensibility of the proposed framework. A number of representative numerical experiments are used to examine the accuracy, robustness and reproducibility of the results generated by the neural network models.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Technical note: Deep learning for creating surrogate models of precipitation in Earth system models

Abstract. We investigate techniques for using deep neural networks to produce surrogatemodels for short-term climate forecasts. A convolutional neural network istrained on 97 years of monthly precipitation output from the 1pctCO2 run (theCO 2 concentration increases by 1 % per year) simulated by the second-generation Canadian Earth System Model (CanESM2). The neural network clearly outperforms a persistence forecast anddoes not show substantially degraded performance even when the forecast lengthis extended to 120 months. The model is prone to underpredicting precipitationin areas characterized by intense precipitation events. Scheduled sampling(forcing the model to gradually use its own past predictions rather than groundtruth) is essential for avoiding amplification of early forecasting errors.However, the use of scheduled sampling also necessitates preforecasting(generating forecasts prior to the first forecast date) to obtain adequateperformance for the first few prediction time steps. We document the trainingprocedures and hyperparameter optimization process for researchers who wish toextend the use of neural networks in developing surrogate models.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2024

Usage of ChatGPT for Engineering Design and Analysis Tool Development

ChatGPT, a generative AI large language model, has recently captured significant attention in both the computer science community and the broader public domain. It has demonstrated a wide range of capabilities, from answering simple questions to writing fully functional computer code. This study spotlights both the capabilities and limitations of ChatGPT when addressing engineering problems. The model's capacity to generate practical engineering tools is highlighted through an example of a prompt that leads to an interactive plotting tool, enabling the examination of the fluid boundary layer around a fan blade. Subsequently, the paper also uncovers potential pitfalls in ChatGPT’s application, shown through an unsuccessful attempt to use ChatGPT to automate a process in Ansys Workbench through scripting. The research further investigates ChatGPT's proficiency in addressing inquiries and providing explanations about the functionalities of OpenMDAO, an open-source, multidisciplinary design, analysis, and optimization tool developed at NASA Glenn Research Center. Finally, an optimization methodology, developed with ChatGPT’s help, is applied to the structural optimization of a fan blade. The developed optimization method utilizes T-Blade3 for geometry generation, Ansys Mechanical for meshing and finite element analysis, and sci-kit learn’s MLPRegressor method to generate a trained neural network model of the design space. OpenMDAO is then used to find the optimal point within the design space. The outcome is a significant reduction in stress in the optimized model—less than one-fifth of the stress value in the baseline model.

Design↗

NASA NTRSDate not supplied

Usage of ChatGPT for Engineering Design and Analysis Tool Development

Design↗

DOE OSTI2022

Making Invisible Visible: Data-Driven Seismic Inversion With Spatio-Temporally Constrained Data Augmentation

Deep learning and data-driven approaches have shown great potential in scientific domains. The promise of data-driven techniques relies on the availability of a large volume of high-quality training datasets. Due to the high cost of obtaining data through expensive physical experiments, instruments, and simulations, data augmentation techniques for scientific applications have emerged as a new direction for obtaining scientific data recently. However, existing data augmentation techniques originating from computer vision yield physically unacceptable data samples that are not helpful for the domain problems that we are interested in. In this article, we develop new data augmentation techniques based on convolutional neural networks. Specifically, our generative models leverage different physics knowledge (such as governing equations, observable perception, and physics phenomena) to improve the quality of the synthetic data. To validate the effectiveness of our data augmentation techniques, we apply them to solve a subsurface seismic full-waveform inversion using simulated CO 2 leakage data. Our interest is to invert for subsurface velocity models associated with very small CO 2 leakage. We validate the performance of our methods using comprehensive numerical tests. Here via comparison and analysis, we show that data-driven seismic imaging can be significantly enhanced by using our data augmentation techniques. Particularly, the imaging quality has been improved by 15% in test scenarios of general-sized leakage and 17% in small-sized leakage when using an augmented training set obtained with our techniques.

58 GEOSCIENCES↗

DOE OSTI2020

Inversion of Time-Lapse Seismic Reservoir Monitoring Data Using CycleGAN: A Deep Learning-Based Approach for Estimating Dynamic Reservoir Property Changes

Carbon capture and storage is being pursued globally as a geoengineering measure for reducing the emission of anthropogenic CO 2 the atmosphere. Comprehensive monitoring, verification, and accounting programs must be established for demonstrating the safe storage of injected CO 2 . One of the most commonly deployed monitoring techniques is time-lapse seismic reservoir monitoring (also known as 4-D seismic), which involves comparing 3-D seismic survey data taken at the same study site but over different times. Analyses of 4-D seismic data volumes can help improve the quality of storage reservoir characterization, track the movement of injected CO 2 plume, and identify potential CO 2 spillover/leakage from the storage reservoirblue. However, the derivation of high-resolution CO 2 saturation maps from 4-D seismic data is a highly nonlinear and ill-posed inverse problem, often requiring significant computational effort. In this research, we apply a physics-based deep learning method to facilitate the solution of both the forward and inverse problems in seismic inversion while honoring physical constraints. A cycle generative adversarial neural network (CycleGAN) model is trained to learn the bidirectional functional mappings between the reservoir dynamic property changes and seismic attribute changes, such that both forward and inverse solutions can be obtained efficiently from the trained model. We show that our CycleGAN-based approach not only improves the reliability of 4-D seismic inversion but also expedites the quantitative interpretation. Our deep learning-based workflow is generic and can be readily used for reservoir characterization and reservoir model updates involving the use of 4-D seismic data.

58 GEOSCIENCES↗

DOE OSTI2020

Deblending galaxies with variational autoencoders: A joint multiband, multi-instrument approach

ABSTRACT Blending of galaxies has a major contribution in the systematic error budget of weak-lensing studies, affecting photometric and shape measurements, particularly for ground-based, deep, photometric galaxy surveys, such as the Rubin Observatory Legacy Survey of Space and Time (LSST). Existing deblenders mostly rely on analytic modelling of galaxy profiles and suffer from the lack of flexible yet accurate models. We propose to use generative models based on deep neural networks, namely variational autoencoders (VAE), to learn probabilistic models directly from data. We train a VAE on images of centred, isolated galaxies, which we reuse, as a prior, in a second VAE-like neural network in charge of deblending galaxies. We train our networks on simulated images including six LSST bandpass filters and the visible and near-infrared bands of the Euclid satellite, as our method naturally generalizes to multiple bands and can incorporate data from multiple instruments. We obtain median reconstruction errors on ellipticities and r-band magnitude between ±0.01 and ±0.05, respectively, in most cases, and ellipticity multiplicative bias of 1.6 per cent for blended objects in the optimal configuration. We also study the impact of decentring and prove the method to be robust. This method only requires the approximate centre of each target galaxy, but no assumptions about the number of surrounding objects, pointing to an iterative detection/deblending procedure we leave for future work. Finally, we discuss future challenges about training on real data and obtain encouraging results when applying transfer learning.

Arcelin, Bastien↗

DOE OSTI2026

An Iterative Machine Learning Framework for Event Classification and Monte Carlo Tuning in SpinQuest

The E1039/SpinQuest experiment at Fermi National Accelerator Laboratory uses a 120~GeV proton beam from the Main Injector incident on transversely polarized proton and deuteron targets, using $NH_3$ and $ND_3$, respectively. In addition to measuring the Sivers asymmetry in Drell--Yan $pp$ and $pd$ scattering from sea quarks, SpinQuest will study transverse-spin effects, particularly the transverse single-spin asymmetry (TSSA) in $J/\psi$ production. The angular distributions from the $J/\psi$ decay could play an important role in understanding the gluon contribution to the proton spin structure. However, before extracting these angular distributions, it is necessary to isolate signal events originating from the target from events produced by other sources and from the combinatorial background. To effectively and accurately classify the target events, it is important to ensure that the simulated events are properly tuned to the experimental physics channels. We have introduced an iterative technique to match simulated and experimental events and to classify the physics channels using deep neural networks and a generative model based on normalizing flows.

Hossain, Forhad [Virginia U. (main)] (ORCID:000000↗

DOE OSTI2021

A systematic approach to generating accurate neural network potentials: the case of carbon

Abstract Availability of affordable and widely applicable interatomic potentials is the key needed to unlock the riches of modern materials modeling. Artificial neural network-based approaches for generating potentials are promising; however, neural network training requires large amounts of data, sampled adequately from an often unknown potential energy surface. Here we propose a self-consistent approach that is based on crystal structure prediction formalism and is guided by unsupervised data analysis, to construct an accurate, inexpensive, and transferable artificial neural network potential. Using this approach, we construct an interatomic potential for carbon and demonstrate its ability to reproduce first principles results on elastic and vibrational properties for diamond, graphite, and graphene, as well as energy ordering and structural properties of a wide range of crystalline and amorphous phases.

Chemistry↗

DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗