Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Generative Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

ChemPix: automated recognition of hand-drawn hydrocarbon structures using deep learning

Inputting molecules into chemistry software, such as quantum chemistry packages, currently requires domain expertise, expensive software and/or cumbersome procedures. Leveraging recent breakthroughs in machine learning, we develop ChemPix: an offline, hand-drawn hydrocarbon structure recognition tool designed to remove these barriers. A neural image captioning approach consisting of a convolutional neural network (CNN) encoder and a long short-term memory (LSTM) decoder learned a mapping from photographs of hand-drawn hydrocarbon structures to machine-readable SMILES representations. We generated a large auxiliary training dataset, based on RDKit molecular images, by combining image augmentation, image degradation and background addition. Additionally, a small dataset of ~600 hand-drawn hydrocarbon chemical structures was crowd-sourced using a phone web application. These datasets were used to train the image-to-SMILES neural network with the goal of maximizing the hand-drawn hydrocarbon recognition accuracy. By forming a committee of the trained neural networks where each network casts one vote for the predicted molecule, we achieved a nearly 10 percentage point improvement of the molecule recognition accuracy and were able to assign a confidence value for the prediction based on the number of agreeing votes. The ensemble model achieved an accuracy of 76% on hand-drawn hydrocarbons, increasing to 86% if the top 3 predictions were considered.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Denoising Convolutional Networks to Accelerate Detector Simulation [Poster]

The high accuracy of detector simulation is crucial for modern particle physics experiments. However, this accuracy comes with a high computational cost, which will be exacerbated by the large datasets and complex detector upgrades associated with next-generation facilities such as the High Luminosity LHC. We explore the viability of regression-based machine learning (ML) approaches using convolutional neural networks (CNN) to ``denoise'' faster, lower-quality detector simulations, augmenting them to produce a higher-quality final result with a reduced computational burden. The denoising CNN works in concert with classical detector simulation software rather than replacing it entirely, increasing its reliability compared to other ML approaches to simulation. We obtain promising results from a prototype based on photon showers in the CMS electromagnetic calorimeter. Future directions are also discussed.

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

Power Prediction of Airborne Wind Energy Systems Using Multivariate Machine Learning

Kites can be used to harvest wind energy at higher altitudes while using only a fraction of the material required for conventional wind turbines. In this work, we present the kite system of Kyushu University and demonstrate how experimental data can be used to train machine learning regression models. The system is designed for 7 kW traction power and comprises an inflatable wing with suspended kite control unit that is either tethered to a fixed ground anchor or to a towing vehicle to produce a controlled relative flow environment. A measurement unit was attached to the kite for data acquisition. To predict the generated tether force, we collected input–output samples from a set of well-designed experimental runs to act as our labeled training data in a supervised machine learning setting. We then identified a set of key input parameters which were found to be consistent with our sensitivity analysis using Pearson input–output correlation metrics. Finally, we designed and tested the accuracy of a neural network, among other multivariate regression models. The quality metrics of our models show great promise in accurately predicting the tether force for new input/feature combinations and potentially guide new designs for optimal power generation.

17 WIND ENERGY↗
DOE OSTI2022

Structure-Informed Graph Learning of Networked Dependencies for Online Prediction of Power System Transient Dynamics

Online transient analysis plays an increasingly important role in dynamic power grids as the renewable generation continues growing. Traditional numerical methods for transient analysis not only are computationally intensive but also require precise contingency information as input, and therefore, are not suitable for online applications. Existing online transient assessment studies focus on the determination of post-contingency system stability or stability margin. Here, this paper develops a novel graph-learning framework, Deep-learning Neural Representation or DNR, for online prediction, of the time-series trajectories of the system states using initial system responses that can be measured by phasor measurement units (PMUs). The proposed DNR framework consists of two sequential modules: a Network Constructor that captures network dependencies among generators, and a Dynamics Predictor that predicts the system trajectories. The key to improved prediction performance is the introduction of the spatio-temporal message-passing operations into graph neural networks with structural knowledge. Its effectiveness and scalability are validated through comparative studies, demonstrating the prediction performance under different contingency scenarios for systems of different sizes. This framework provides a solution to online predicting post-fault system dynamics based on real-time PMU measurements. Additionally, it can also be applied to facilitate the offline transient simulation without simulating the entire trajectories.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Augmented signal processing in Liquid Argon Time Projection Chambers with a deep neural network

The Liquid Argon Time Projection Chamber (LArTPC) is an advanced neutrino detector technology widely used in recent and upcoming accelerator neutrino experiments. It features a low energy threshold and high spatial resolution that allow for comprehensive reconstruction of event topologies. In current-generation LArTPCs, the recorded data consist of digitized waveforms on wires produced by induced signal on wires of drifting ionization electrons, which can also be viewed as two-dimensional (2D) (time versus wire) projection images of charged-particle trajectories. For such an imaging detector, one critical step is the signal processing that reconstructs the original charge projections from the recorded 2D images. For the first time, we introduce a deep neural network in LArTPC signal processing to improve the signal region of interest detection. By combining domain knowledge (e.g., matching information from multiple wire planes) and deep learning, this method shows significant improvements over traditional methods. This work details the method, software tools, and performance evaluated with realistic detector simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Machine learning elastic constants of multi-component alloys

The present manuscript explores application of machine learning methods for determining elastic constants and other derived mechanical properties of multi-component alloys. Here, a number of machine learning models, including linear regression, neural network and random forest based models, are trained and tested on a dataset of binary alloys generated using density functional theory (DFT) calculations and spanning over a large number of elemental species in the periodic table. Starting with a wide range of simple and easily accessible compositionally-averaged elemental features, a correlation-based feature selection strategy was used to systematically down-select a set of most relevant features towards the prediction of the elasticity tensor components. The true predictive performance and the associated uncertainties of the models were established by testing on unseen data and bootstrapping, respectively. A single and pair-wise feature partial dependence analysis was performed to visualize the average property trends in the multi-dimensional feature space in order to further understand the achieved predictive performance. The utility of the trained model is further demonstrated by obtaining sufficiently accurate yet highly efficient approximations for bulk modulus, Young’s modulus, shear modulus and Poisson’s ratio for alloys beyond the binary space (i.e., two-component alloys) on which the model was originally trained. More importantly, we test and validate the predictive performance of the developed model directly against the experimentally measured elastic constants of technologically relevant multi-component alloys (such as, Ni- and Ti-based alloys). Finally, utility of such a data-enabled route is demonstrated by predicting the possible range of various elastic properties for vast composition space available within the five component Ni-Cr-Fe-Mo-W alloy system in a high-throughput manner.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Prediction of chronic kidney disease progression using recurrent neural network and electronic health records

Chronic kidney disease (CKD) is a progressive loss in kidney function. Early detection of patients who will progress to late-stage CKD is of paramount importance for patient care. To address this, we develop a pipeline to process longitudinal electronic heath records (EHRs) and construct recurrent neural network (RNN) models to predict CKD progression from stages II/III to stages IV/V. The RNN model generates predictions based on time-series records of patients, including repeated lab tests and other clinical variables. Our investigation reveals that using a single variable, the recorded estimated glomerular filtration rate (eGFR) over time, the RNN model achieves an average area under the receiver operating characteristic curve (AUROC) of 0.957 for predicting future CKD progression. When additional clinical variables, such as demographics, vital information, lab test results, and health behaviors, are incorporated, the average AUROC increases to 0.967. In both scenarios, the standard deviation of the AUROC across cross-validation trials is less than 0.01, indicating a stable and high prediction accuracy. Our analysis results demonstrate the proposed RNN model outperforms existing standard approaches, including static and dynamic Cox proportional hazards models, random forest, and LightGBM. The utilization of the RNN model and the time-series data of previous eGFR measurements underscores its potential as a straightforward and effective tool for assessing the clinical risk of CKD patients concerning their disease progression.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Augmenting machine learning of Grad–Shafranov equilibrium reconstruction with Green's functions

This work presents a method for predicting plasma equilibria in tokamak fusion experiments and reactors. The approach involves representing the plasma current as a linear combination of basis functions using principal component analysis of plasma toroidal current densities (J t ) from the EFIT-AI equilibrium database. Then utilizing EFIT's Green's function tables, basis functions are created for the poloidal flux (ψ) and diagnostics generated from the toroidal current (J t ). Similar to the idea of a physics-informed neural network (NN), this physically enforces consistency between ψ, J t , and the synthetic diagnostics. First, the predictive capability of a least squares technique to minimize the error on the synthetic diagnostics is employed. The results show that the method achieves high accuracy in predicting ψ and moderate accuracy in predicting J t with median R 2 = 0.9993 and R 2 = 0.978, respectively. A comprehensive NN using a network architecture search is also employed to predict the coefficients of the basis functions. The NN demonstrates significantly better performance compared to the least squares method with median R 2 = 0.9997 and 0.9916 for J t and ψ, respectively. The robustness of the method is evaluated by handling missing or incorrect data through the least squares filling of missing data, which shows that the NN prediction remains strong even with a reduced number of diagnostics. Additionally, the method is tested on plasmas outside of the training range showing reasonable results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

SOC Microstructural Property Estimator

This pre-trained ML model is a tool that uses basic compositional parameters for porous solid oxide cell (SOC) electrodes - the phase fractions and mean particle/pore diameters – as inputs and uses them to estimate additional electrochemical performance parameters: active (i.e., connected) TPB density, all tortuosity factors, and phase pair specific interfacial areas. The electrode is assumed to be composed of two solid phases and a pore phase. The property calculations are performed using neural network regression models trained on a large bank of synthetic electrode microstructural data that NETL has generated using the program DREAM3D (that bank is also hosted on EDX: https://edx.netl.doe.gov/dataset/soc-synthetic-microstructure-bank). This means the generated parameters are based on training from actual measured properties from 3D microstructures, not estimated from geometric simplifications. This tool was developed and is intended to replace percolation theory calculations in models that use hypothetical electrode properties. An example use case would be running SOC performance simulations across a parametric sweep of electrode designs (e.g., varying phase fractions and particle sizes) and assessing how it impacts the electrochemical performance of the SOC. Within the parameter space of the training data (statistics of that parameter space is provided in the readme file), this model achieves sub-5% mean absolute percent errors, an order of magnitude less error than percolation theory across the same parameter space. However, be aware that this tool was developed with parametric simulations in mind, and users are encouraged to assess accuracy for their own specific use case rather than taking accuracy metrics at face value. More info, including a usage guide, is in the included readme file. This tool should be cited with the DOI number provided.

Electrode Microstructure↗
DOE OSTI2025

Development and Evaluation of a General Drag Model for Gas-Solid Flows via Deep Learning

This project presents the development and evaluation of a general drag model for gas–solid multiphase flows using deep learning techniques. A comprehensive database of more than 4,000 experimental and numerical data points for spherical and non spherical particles was compiled, incorporating geometric features such as sphericity, aspect ratio, and orientation. Several predictive approaches—including traditional em pirical correlations, machine learning, and deep neural networks—were benchmarked, with the proposed Drag Coefficient Correlation-aided Deep Neural Network (DCC DNN) demonstrating superior accuracy. To account for particle–particle interactions, additional drag data were generated using CFD-based simulations of packed and flu idized beds, leading to the development of a retrained model capable of incorporat ing volume fraction effects. Integration of the trained model with the MFiX CFD solver was achieved using FTorch, enabling drag predictions during discrete element method (DEM) simulations. Validation against experimental data for single particles and fluidized beds confirmed the model’s improved predictive ability, particularly for non-spherical geometries. While the model performed strongly under fluidized con ditions, limitations remained in unfluidized regimes, suggesting a need for expanded datasets. Overall, this study demonstrates the feasibility of combining deep learning with physics-informed CFD to improve drag modeling for gas–solid flows, with promis ing implications for scaling multiphase simulations in industrial applications.

42 ENGINEERING↗
NASA NTRSDate not supplied

Machine-Learning for Safety Critical Airborne Applications Part II: Case Study

The exceptional progress in the field of Artificial Intelligence (AI) systems, enabled by Machine Learning (ML) technology in recent years provides historic opportunities for the aviation industry. Current certification standards for avionics were developed prior to the ML renaissance and have several fundamental incompatibilities with the ML technology. WG-114 is working hard to release a new standard as soon as possible but for now there is no recognized means of compliance for ML based systems even of low criticality. In this talk, we present the custom ML workflow that can be used comply with all objectives of the current certification standards for a low-criticality (DAL D and C) ML-based system. To illustrate the practical application of the custom ML workflow we present a case study of a system based on a Deep Neural Network (DNN) intended to detect and identify airport runway signs. We present the system design, data generation, training, and verification in detail and describe how the design assurance objectives can be met for a DAL D and DAL C systems.

Johann Schumann↗
DOE OSTI2022

Computational Workflow for Accelerated Molecular Design Using Quantum Chemical Simulations and Deep Learning Models

Efficient methods for searching the chemical space of molecular compounds are needed to automate and accelerate the design of new functional molecules such as pharmaceuticals. Given the high cost in both resources and time for experimental efforts, computational approaches play a key role in guiding the selection of promising molecules for further investigation. Here, we construct a workflow to accelerate design by combining approximate quantum chemical methods [i.e. density-functional tight-binding (DFTB)], a graph convolutional neural network (GCNN) surrogate model for chemical property prediction, and a masked language model (MLM) for molecule generation. Property data from the DFTB calculations are used to train the surrogate model; the surrogate model is used to score candidates generated by the MLM. The surrogate reduces computation time by orders of magnitude compared to the DFTB calculations, enabling an increased search of chemical space. Furthermore, the MLM generates a diverse set of chemical modifications based on pre-training from a large compound library. We utilize the workflow to search for near-infrared photoactive molecules by minimizing the predicted HOMO-LUMO gap as the target property. Our results show that the workflow can generate optimized molecules outside of the original training set, which suggests that iterations of the workflow could be useful for searching vast chemical spaces in a wide range of design problems.

Blanchard, Andrew↗
DOE OSTI2020

Accelerated version of NUBEAM capabilities in DIII-D using neural networks

A neural network model of the effects of neutral beam injection on DIII-D has been developed. The training and testing data used by the model have been generated by the NUBEAM module of TRANSP for experimental discharges from the 2018 DIII-D campaign. Using a principle component analysis to reduce the dimensionality of profile data, the model has been shown to reproduce the results of the Monte Carlo code NUBEAM with a high level of accuracy and an execution time orders of magnitude faster than the execution time of NUBEAM. Furthermore, this makes the neural network model uniquely suited to applications in model-based scenario planning (off-line) and active control (on-line), where a large number of simulation runs are required by the associated optimization tasks that need to be performed before and during the discharge.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1994

VLSI neuroprocessors

Electronic and optoelectronic hardware implementations of highly parallel computing architectures address several ill-defined and/or computation-intensive problems not easily solved by conventional computing techniques. The concurrent processing architectures developed are derived from a variety of advanced computing paradigms including neural network models, fuzzy logic, and cellular automata. Hardware implementation technologies range from state-of-the-art digital/analog custom-VLSI to advanced optoelectronic devices such as computer-generated holograms and e-beam fabricated Dammann gratings. JPL's concurrent processing devices group has developed a broad technology base in hardware implementable parallel algorithms, low-power and high-speed VLSI designs and building block VLSI chips, leading to application-specific high-performance embeddable processors. Application areas include high throughput map-data classification using feedforward neural networks, terrain based tactical movement planner using cellular automata, resource optimization (weapon-target assignment) using a multidimensional feedback network with lateral inhibition, and classification of rocks using an inner-product scheme on thematic mapper data. In addition to addressing specific functional needs of DOD and NASA, the JPL-developed concurrent processing device technology is also being customized for a variety of commercial applications (in collaboration with industrial partners), and is being transferred to U.S. industries. This viewgraph p resentation focuses on two application-specific processors which solve the computation intensive tasks of resource allocation (weapon-target assignment) and terrain based tactical movement planning using two extremely different topologies. Resource allocation is implemented as an asynchronous analog competitive assignment architecture inspired by the Hopfield network. Hardware realization leads to a two to four order of magnitude speed-up over conventional techniques and enables multiple assignments, (many to many), not achievable with standard statistical approaches. Tactical movement planning (finding the best path from A to B) is accomplished with a digital two-dimensional concurrent processor array. By exploiting the natural parallel decomposition of the problem in silicon, a four order of magnitude speed-up over optimized software approaches has been demonstrated.

Kemeny, Sabrina E.↗
NASA NTRS2018

Deep Neural Network for Precision Multi-band Infrared Image Segmentation

Image segmentation is one of the fundamental steps in computer vision. Separating targets from background clutter with high precision is a challenging operation for both humans and computers. Currently, segmenting objects from IR images is done by tedious manual work. The implementation of a Deep Neural Network (DNN) to perform precision segmentation of multi-band IR video images is presented. A customized pix2pix DNN with multiple layers of generative encoder/decoder and discriminator architecture is used in the IR image segmentation process. Real and synthetic images and ground truths are employed to train the DNN. Iterative training is performed to achieve optimum accuracy of segmentation using a minimal number of training data. Special training images are created to enhance the missing features and to increase the segmentation accuracy of the objects. Retraining strategies are developed to minimize the DNN training time. Single pixel accuracy has been achieved in IR target boundary segmentation using DNNs. The segmentation accuracy between the customized pix2pix DNN and simple thresholding, GraphCut, simple neural network and ResNet models are compared.

Torres, Gil↗
DOE OSTI2024

Prediction of laser beam spatial profiles in a high-energy laser facility by use of deep learning

We adapt the significant advances achieved recently in the field of generative artificial intelligence/machine-learning to laser performance modeling in multipass, high-energy laser systems with application to high-shot-rate facilities relevant to inertial fusion energy. Advantages of neural-network architectures include rapid prediction capability, data-driven processing, and the possibility to implement such architectures within future low-latency, low-power consumption photonic networks. Four models were investigated that differed in their generator loss functions and utilized the U-Net encoder/decoder architecture with either a reconstruction loss alone or combined with an adversarial network loss. We achieved inference times of 1.3 ms for a 256 × 256 pixel near-field beam with errors in predicted energy of the order of 1% over most of the energy range. It is shown that prediction errors are significantly reduced by ensemble averaging the models with different weight initializations. These results suggest that including the temporal dimension in such models may provide accurate, real-time spatiotemporal predictions of laser performance in high-shot-rate laser systems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Neural network potential from bispectrum components: A case study on crystalline silicon

In this article, we present a systematic study on developing machine learning force fields (MLFFs) for crystalline silicon. While the main-stream approach of fitting a MLFF is to use a small and localized training set from molecular dynamics simulations, it is unlikely to cover the global features of the potential energy surface. Additionally, to remedy this issue, we used randomly generated symmetrical crystal structures to train a more general Si-MLFF. Furthermore, we performed substantial benchmarks among different choices of material descriptors and regression techniques on two different sets of silicon data. Our results show that neural network potential fitting with bispectrum coefficients as descriptors is a feasible method for obtaining accurate and transferable MLFFs.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗