Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A transported Livengood-Wu integral model for knock prediction in CFD simulation

This work describes the development of a transported Livengood-Wu (L-W) integral model for computational fluid dynamics (CFD) simulation to predict auto-ignition and engine knock tendency. The currently employed L-W integral model considers both single-stage and two-stage ignition processes, thus can be generally applied to different fuels such as paraffin, olefin, aromatics and alcohol. The model implementation is first validated in simulations of homogeneous charge compression ignition combustion for three different fuels, showing good accuracy in prediction of auto-ignition timing for fuels with either single-stage or two-stage ignition characteristics. Then, the L-W integral model is coupled with G-equation model to indicate end-gas auto-ignition and knock tendency in CFD simulations of a direct injection spark ignition engine. This modeling approach is about 10 times more efficient than the ones that based on detailed chemistry calculation and pressure oscillation analysis. Two fuels with same Research Octane Number (RON) but different octane sensitivity are studied, namely Co-Optima Alkylate and Co-Optima E30. Feed-forward neural network model in conjunction with multi-variable minimization technique is used to generate fuel surrogates with targets of matched RON, octane sensitivity and ethanol content. The CFD model is validated against experimental data in terms of pressure traces and heat release rate for both fuels under a wide range of operating conditions. The knock tendency indicated by the fuel energy contained in the auto-ignited region of the two fuels at different load conditions correlates well with the experimental results and the fuel octane sensitivity, implying the current knock modeling approach can capture the octane sensitivity effect and can be applied to further investigation on composition of octane sensitivity.

Yue, Zongyu↗
DOE OSTI2024

UNNT: A novel Utility for comparing Neural Net and Tree-based models

The use of deep learning (DL) is steadily gaining traction in scientific challenges such as cancer research. Advances in enhanced data generation, machine learning algorithms, and compute infrastructure have led to an acceleration in the use of deep learning in various domains of cancer research such as drug response problems. In our study, we explored tree-based models to improve the accuracy of a single drug response model and demonstrate that tree-based models such as XGBoost (eXtreme Gradient Boosting) have advantages over deep learning models, such as a convolutional neural network (CNN), for single drug response problems. However, comparing models is not a trivial task. To make training and comparing CNNs and XGBoost more accessible to users, we developed an open-source library called UNNT (A novel Utility for comparing Neural Net and Tree-based models). The case studies, in this manuscript, focus on cancer drug response datasets however the application can be used on datasets from other domains, such as chemistry.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Approximating Trajectory Constraints With Machine Learning – Microgrid Islanding With Frequency Constraints

Here, we introduce deep earning aided constraint encoding to tackle the frequency-constraint microgrid scheduling problem. The nonlinear function between system operating condition and frequency nadir is approximated by using a neural network, which admits an exact mixed-integer formulation (MIP). This formulation is then integrated with the scheduling problem to encode the frequency constraint. With the stronger representation power of the neural network, the resulting commands can ensure adequate frequency response in a realistic setting in addition to islanding success. The proposed method is validated on a modified 33-node system. Successful islanding with a secure response is simulated under the scheduled commands using a detailed three-phase model in Simulink. The advantages of our model are particularly remarkable when the inertia emulation functions from wind turbine generators are considered.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2023

Neural Radiance Fields for Shadow Removal and Relighting of Satellite Imagery

Multi-view stereo photogrammetric techniques have been conventionally utilized to generate Global Digital Elevation Models (GDEM) of planetary and lunar surfaces. However, these methods, relying on conventional feature detectors, are often subject to inaccuracies caused by changes in lighting conditions, including diffuse reflection and harsh shading. This has limited the ability of these methods to accurately reconstruct shadowed regions in orbital imagery, such as highly shaded urban areas and the permanently shadowed regions (PSRs) located in the lunar surface, which are critical targets for NASA's Artemis program. Neural Radiance Fields (NeRFs) offer a novel solution to these limitations by breaking away from traditional photogrammetric assumptions of ridged, opaque surfaces. NeRFs are capable of reconstructing 3D objects with variably transmissive properties, such as fluids, PSRs, and reflective surfaces. In this work, we demonstrate the robustness of NeRFs in generating high-fidelity 3D models of terrain from highly shaded orbital imagery acquired from small satellites in low earth orbit (LEO). Our results showcase the ability of NeRF to overcome the limitations of traditional photogrammetric methods and provide a promising solution for reconstructing complex scenes in challenging lighting conditions.

Caleb Ashmore Adams↗
DOE OSTI2023

A deep neural network regressor for phase constitution estimation in the high entropy alloy system Al-Co-Cr-Fe-Mn-Nb-Ni

High Entropy Alloys (HEAs) are composed of more than one principal element and constitute a major paradigm in metals research. The HEA space is vast and an exhaustive exploration is improbable. Therefore, a thorough estimation of the phases present in the HEA is of paramount importance for alloy design. Machine Learning presents a feasible and non-expensive method for predicting possible new HEAs on-the-fly. A deep neural network (DNN) model for the elemental system of: Mn, Ni, Fe, Al, Cr, Nb, and Co is developed using a dataset generated by high-throughput computational thermodynamic calculations using Thermo-Calc. The features list used for the neural network is developed based on literature and freely available databases. A feature significance analysis matches the reported HEAs phase constitution trends on elemental properties and further expands it by providing so far-overlooked features. The final regressor has a coefficient of determination ( r 2 ) greater than 0.96 for identifying the most recurrent phases and the functionality is tested by running optimization tasks that simulate those required in alloy design. The DNN developed constitutes an example of an emulator that can be used in fast, real-time materials discovery/design tasks.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗
NASA NTRS1998

A Feasibility Study of Synthesizing Subsurfaces Modeled with Computational Neural Networks

This paper investigates the feasibility of synthesizing substructures modeled with computational neural networks. Substructures are modeled individually with computational neural networks and the response of the assembled structure is predicted by synthesizing the neural networks. A superposition approach is applied to synthesize models for statically determinate substructures while an interface displacement collocation approach is used to synthesize statically indeterminate substructure models. Beam and plate substructures along with components of a complicated Next Generation Space Telescope (NGST) model are used in this feasibility study. In this paper, the limitations and difficulties of synthesizing substructures modeled with neural networks are also discussed.

Wang, John T.↗
DOE OSTI2020

Power System Event Classification and Localization Using a Convolutional Neural Network

Detection and timely identification of power system disturbances are essential for situation awareness and reliable electricity grid operation. Because records of actual events in the system are limited, ensemble simulation-based events are needed to provide adequate data for building event-detection models through deep learning; e.g., a convolutional neural network (CNN). An ensemble numerical simulation-based training data set have been generated through dynamic simulations performed on the Polish system with various types of faults in different locations. Such data augmentation is proven to be able to provide adequate data for deep learning. The synchronous generators’ frequency signals are used and encoded into images for developing and evaluating CNN models for classification of fault types and locations. With a time-domain stacked image set as the benchmark, two different time-series encoding approaches, i.e., wavelet decomposition-based frequency-domain stacking and polar coordinate system-based Gramian Angular Field (GAF) stacking, are also adopted to evaluate and compare the CNN model performance and applicability. The various encoding approaches are suitable for different fault types and spatial zonation. With optimized settings of the developed CNN models, the classification and localization accuracies can go beyond 84 and 91%, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Feasibility of Using Fourier Neural Operators for 3D Elastic Seismic Simulations

Seismic simulations in three-dimensional (3D) Earth models are important in many of the seismological applications related to our lab’s mission, however, accurate high-fidelity simulations are computationally costly. We investigated the capabilities of the newly developed Fourier Neural Operator (FNO) to solve the 3D elastic wave equations for seismic simulations. We generated simulation data for training the FNO model, and analyzed its performance on various test cases, such as the performance with different number of training data, different resolutions, and on more canonical structures. We found the FNO model can reproduce 3D seismic simulations with high accuracy and ~169 times faster on a small spatial grid size of 16 x 16 x16. When applied on higher resolution data, we found that transfer learning with fine-tuning on a small amount of data achieves reasonable results. This feasibility study showed promising results for using FNO’s for 3D seismic simulations and represents the foundation of further research to develop this into a more mature approach for different seismological applications. The potential impact of this project will provide techniques for large-scale or real-time applications of solving PEDs in support of national lab’s programs.

58 GEOSCIENCES↗
DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗
DOE OSTI2024

Utilization of Synthetic Near-Infrared Spectra via Generative Adversarial Network to Improve Wood Stiffness Prediction

Near-infrared (NIR) spectroscopy is widely used as a nondestructive evaluation (NDE) tool for predicting wood properties. When deploying NIR models, one faces challenges in ensuring representative training data, which large datasets can mitigate but often at a significant cost. Machine learning and deep learning NIR models are at an even greater disadvantage because they typically require higher sample sizes for training. In this study, NIR spectra were collected to predict the modulus of elasticity (MOE) of southern pine lumber (training set = 573 samples, testing set = 145 samples). To account for the limited size of the training data, this study employed a generative adversarial network (GAN) to generate synthetic NIR spectra. The training dataset was fed into a GAN to generate 313, 573, and 1000 synthetic spectra. The original and enhanced datasets were used to train artificial neural networks (ANNs), convolutional neural networks (CNNs), and light gradient boosting machines (LGBMs) for MOE prediction. Overall, results showed that data augmentation using GAN improved the coefficient of determination (R 2 ) by up to 7.02% and reduced the error of predictions by up to 4.29%. ANNs and CNNs benefited more from synthetic spectra than LGBMs, which only yielded slight improvement. All models showed optimal performance when 313 synthetic spectra were added to the original training data; further additions did not improve model performance because the quality of the datapoints generated by GAN beyond a certain threshold is poor, and one of the main reasons for this can be the size of the initial training data fed into the GAN. LGBMs showed superior performances than ANNs and CNNs on both the original and enhanced training datasets, which highlights the significance of selecting an appropriate machine learning or deep learning model for NIR spectral-data analysis. The results highlighted the positive impact of GAN on the predictive performance of models utilizing NIR spectroscopy as an NDE technique and monitoring tool for wood mechanical-property evaluation. Further studies should investigate the impact of the initial size of training data, the optimal number of generated synthetic spectra, and machine learning or deep learning models that could benefit more from data augmentation using GANs.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Neural Network Prediction of Strong Lensing Systems with Domain Adaptation and Uncertainty Quantification

Modeling strong gravitational lenses is computationally expensive for the complex data from modern and next-generation cosmic surveys. Deep learning has emerged as a promising approach for finding lenses and predicting lensing parameters, such as the Einstein radius. Mean-variance Estimators (MVEs) are a common approach for obtaining aleatoric (data) uncertainties from a neural network prediction. However, neural networks have not been demonstrated to perform well on out-of-domain target data successfully - e.g., when trained on simulated data and applied to real, observational data. In this work, we perform the first study of the efficacy of MVEs in combination with unsupervised domain adaptation (UDA) on strong lensing data. The source domain data is noiseless, and the target domain data has noise mimicking modern cosmology surveys. We find that adding UDA to MVE increases the accuracy on the target data by a factor of about two over an MVE model without UDA. Including UDA also permits much more well-calibrated aleatoric uncertainty predictions. Advancements in this approach may enable future applications of MVE models to real observational data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Evaluating pulse-shaping capabilities of next-generation pulsed power architectures

This project evaluated the pulse shaping capabilities of next-generation pulsed power (NGPP) architectures. NGPP architectures share several common attributes including multiple independent pulse-generation lines, a radial water-insulated impedance transformer, and a central vacuum insulated load region. A multi-module circuit model was developed, incorporating independent pulse-generation lines and a 2-D transmission line mesh of the radial impedance transformer to assess the effects of azimuthal asymmetry in pulse-shaped experiments. Circuit model simulations demonstrated that NGPP architectures are able to produce the the desired current pulse shapes for exemplar NGPP experiments. Additionally, the project explored automated methods for experiment design, including derivative -ree optimization and machine learning. Pulse-shaped experiments require designers to determine machine parameters that reliably produce the desired current pulse at the load, a process that typically relies on expert knowledge and iterative adjustments using the Z circuit model. Given the increased complexity of NGPP systems, this manual approach may be impractical. While the evaluated methods do not eliminate the need for manual iteration, they can reduce the time required for experiment design. Derivative-free optimization automates much of the trial-and-error process, providing a close starting point for manual adjustments or making small modifications to near-final designs. Meanwhile, deep neural network methods can generate a good qualitative match to the desired current pulse in under one second without requiring circuit model simulations.

42 ENGINEERING↗
DOE OSTI2023

Enabling site-specific well leakage risk estimation during geologic carbon sequestration using a modular deep-learning-based wellbore leakage model

Geologic carbon sequestration (GCS) is a promising technology for mitigating net carbon emissions and growing climate concern by storing CO 2 in reservoirs. Oil and gas brownfields are an attractive option for CO 2 storage, but these sites have many historical wellbores from petroleum production and can be a potential leakage pathway for CO 2 or formation brine. Therefore, risk management of GCS operations requires an assessment of potential well leakage. Due to the high uncertainty of the system, stochastic approaches are ideal for quantifying the range of risk behaviors, but they must be computationally efficient in the face of complex physics. Here, we develop a new physics-centric deep learning wellbore model to predict the leakage of CO 2 and brine through leaky wellbores. Multi-physics numerical simulations were used to generate data sets, and physics-informed features were introduced. Neural networks were optimized with an automated searching algorithm. Feature analysis quantifies the impact of each feature on model prediction and confirms the role of physics-inspired parameters. The model shows high predictive performance across a wide range of geologic and injection conditions and well attributes. In conclusion, a case study illustrates how the model is applied to assess well leakage in GCS operations.

58 GEOSCIENCES↗
DOE OSTI2024

Autonomous control for Heat-Pipe microreactor using Data-Driven model predictive control

To enable a self-regulating capability for heat pipe (HP) microreactors, an anticipatory control strategy achieved via model predictive control (MPC) could proactively respond to potential disturbances and deviations in operating setpoints. This paper demonstrates data-driven methods for predicting the distribution and transient of temperatures and heat fluxes at selected components and regions in a 37-HP system, based on which the optimal control actions in response to changes in user-defined setpoints can be found. We present the development and validation of linear state-space model, feedfoward, and recurrent neural networks. Here, we compare the performance of MPCs with different modeling approaches in terms of following setpoints for temperatures and averaged output heat fluxes. The accuracies of the three data-driven models are similar, but the control actions initiated by neural-network-based MPC can better adapt to drastic changes in setpoints yet generate the smallest errors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Data from: Learning coagulation processes with combinatorially-invariant neural networks

This dataset contains all the necessary information to recreate the study presented in the paper entitled "Learning coagulation processes with combinatorially-invariant neural networks". This consists of (1) the aggregated output files used for machine learning, (2) the machine learning codes used to learn the presented models, (3) the PartMC model source code that was used to generate the simulation data and (4) the Python scripts used construct the scenario library for training and testing simulations. This data was used to investigate a method (combinatorally-invariant neural network) for learning the aerosol process of coagulation. This data may be useful for application of other methods.

Atmospheric chemistry↗
DOE OSTI2025

Huge ensembles – Part 2: Properties of a huge ensemble of hindcasts generated with spherical Fourier neural operators

Abstract. In Part 1, we created an ensemble based on spherical Fourier neural operators. As initial condition perturbations, we used bred vectors, and as model perturbations, we used multiple checkpoints trained independently from scratch. Based on diagnostics that assess the ensemble's physical fidelity, our ensemble has comparable performance to operational weather forecasting systems. However, it requires orders-of-magnitude fewer computational resources. Here in Part 2, we generate a huge ensemble (HENS), with 7424 members initialized each day of summer 2023. We enumerate the technical requirements for running huge ensembles at this scale. HENS precisely samples the tails of the forecast distribution and presents a detailed sampling of internal variability. HENS has two primary applications: (1) as a large dataset with which to study the statistics and drivers of extreme weather and (2) as a weather forecasting system. For extreme climate statistics, HENS samples events 4σ away from the ensemble mean. At each grid cell, HENS increases the skill of the most accurate ensemble member and enhances coverage of possible future trajectories. As a weather forecasting model, HENS issues extreme weather forecasts with better uncertainty quantification. It also reduces the probability of outlier events, in which the verification value lies outside the ensemble forecast distribution.

Mahesh, Ankur↗
DOE OSTI2024

Solving the Orszag–Tang vortex magnetohydrodynamics problem with physics-constrained convolutional neural networks

We study the 2D Orszag–Tang vortex magnetohydrodynamics (MHD) problem through the use of physics-constrained convolutional neural networks (PCNNs) for forecasting the density, ρ, and the magnetic field, B, as well as the prediction of B given the velocity field v of the fluid. In addition to translation equivariance from the convolutional architecture, other physics constraints were embedded: absence of magnetic monopoles, non-negativity of ρ, use of only relevant variables, and the periodic boundary conditions of the problem. The use of only relevant variables and the hard constraint of non-negative ρ were found to facilitate learning greatly. The divergenceless condition ∇·B=0 was implemented as a hard constraint up to machine precision through the use of a magnetic potential to define B=∇×A. Residual networks and data augmentation were also used to improve performance. This allowed for some of the residual models to function as surrogate models and provide reasonably accurate simulations. For the prediction task, the PCNNs were evaluated against a physics-informed neural network, which had the ideal MHD induction equation as a soft constraint. Several models were able to generate highly accurate fields, which are visually almost indistinguishable and have low mean squared error. Only methods with built-in hard constraints produced physical fields with ∇·B=0. The use of PCNNs for MHD has the potential to produce physically consistent real-time simulations to serve as virtual diagnostics in cases where inferences must be made with limited observables.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗