Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

Two-Dimensional High-Lift Aerodynamic Optimization Using Neural Networks

The high-lift performance of a multi-element airfoil was optimized by using neural-net predictions that were trained using a computational data set. The numerical data was generated using a two-dimensional, incompressible, Navier-Stokes algorithm with the Spalart-Allmaras turbulence model. Because it is difficult to predict maximum lift for high-lift systems, an empirically-based maximum lift criteria was used in this study to determine both the maximum lift and the angle at which it occurs. The 'pressure difference rule,' which states that the maximum lift condition corresponds to a certain pressure difference between the peak suction pressure and the pressure at the trailing edge of the element, was applied and verified with experimental observations for this configuration. Multiple input, single output networks were trained using the NASA Ames variation of the Levenberg-Marquardt algorithm for each of the aerodynamic coefficients (lift, drag and moment). The artificial neural networks were integrated with a gradient-based optimizer. Using independent numerical simulations and experimental data for this high-lift configuration, it was shown that this design process successfully optimized flap deflection, gap, overlap, and angle of attack to maximize lift. Once the neural nets were trained and integrated with the optimizer, minimal additional computer resources were required to perform optimization runs with different initial conditions and parameters. Applying the neural networks within the high-lift rigging optimization process reduced the amount of computational time and resources by 44% compared with traditional gradient-based optimization procedures for multiple optimization runs.

Greenman, Roxana M.↗
DOE OSTI2024

Underground hydrogen storage leakage detection and characterization based on machine learning of sparse seismic data

Underground hydrogen storage (UHS) is considered as a scalable approach for massive storage and seasonal extraction of hydrogen (H 2 ). Although conventional leakage detection and characterization methods based on time-lapse seismic imaging and inversion generally apply to H 2 leakage detection problem, a high-fidelity yet cost effective geophysics approach is still missing to reliably inform leakage location and properties based on very sparse data. In response, we develop a novel supervised machine learning method to detect and characterize H 2 leakage from UHS. The input to our neural network are sparse time-lapse seismic waveforms, while the output from the neural network includes the spatial location and physical properties of a H 2 leakage. Here, we generate high-quality time-lapse waveforms using the elastic-wave equations to train the neural network. We train and validate our machine learning model and find that it attains high accuracy in using extremely sparse time-lapse seismic data to detect and characterize H 2 leakage. Our investigation is the first systematic study that focuses on applying machine learning to subsurface H 2 leakage detection and characterization and could potentially serve as a cost-effective geophysical tool for underground hydrogen leakage detection and characterization with high fidelity.

08 HYDROGEN↗
DOE OSTI2023

Machine Learning Predictions of Simulated Self-Diffusion Coefficients for Bulk and Confined Pure Liquids

Diffusion properties of bulk fluids have been predicted using empirical expressions and machine learning (ML) models, suggesting that predictions of diffusion also should be possible for fluids in confined environments. The ability to quickly and accurately predict diffusion in porous materials would enable new discoveries and spur development in relevant technologies such as separations, catalysis, batteries, and subsurface applications. Here in this work, we apply artificial neural network (ANN) models to predict the simulated self-diffusion coefficients of real liquids in both bulk and pore environments. The training data sets were generated from molecular dynamics (MD) simulations of Lennard-Jones particles representing a diverse set of 14 molecules ranging from ammonia to dodecane over a range of liquid pressures and temperatures. Planar, cylindrical, and hexagonal pore models consisted of walls composed of carbon atoms. Our simple model for these liquids was primarily used to generate ANN training data, but the simulated self-diffusion coefficients of bulk liquids show excellent agreement with experimental diffusion coefficients. ANN models based on simple descriptors accurately reproduced the MD diffusion data for both bulk and confined liquids, including the trend of increased mobility in large pores relative to the corresponding bulk liquid.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Fast predictions of liquid-phase acid-catalyzed reaction rates using molecular dynamics simulations and convolutional neural networks

The rates of liquid-phase, acid-catalyzed reactions relevant to the upgrading of biomass into high-value chemicals are highly sensitive to solvent composition and identifying suitable solvent mixtures is theoretically and experimentally challenging. We show that the complex atomistic configurations of reactant–solvent environments generated by classical molecular dynamics simulations can be exploited by 3D convolutional neural networks to enable accurate predictions of Brønsted acid-catalyzed reaction rates for model biomass compounds. We develop a 3D convolutional neural network, which we call SolventNet, and train it to predict acid-catalyzed reaction rates using experimental reaction data and corresponding molecular dynamics simulation data for seven biomass-derived oxygenates in water–cosolvent mixtures. We show that SolventNet can predict reaction rates for additional reactants and solvent systems an order of magnitude faster than prior simulation methods. This combination of machine learning with molecular dynamics enables the rapid, high-throughput screening of solvent systems and identification of improved biomass conversion conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

Fingerprinting Anomalous Computation with RNN for GPU-accelerated HPC Machines

This paper presents a workload classification framework that discriminates illicit computation from authorized workloads on GPU-accelerated HPC systems. As such systems become more and more powerful, they are exploited by attackers to run malicious and for-profit programs that typically require extremely high computing ability to be successful. Our classification framework leverages the distinctive signatures between illicit and authorized workloads, and explore machine learning methods to learn the workloads and classify them. The framework uses lightweight, non-intrusive workload profiling to collect model input data, and explores multiple machine learning methods, particularly recurrent neural network (RNN) that is suitable for online anomalous workload detection. Evaluation results on four generations of GPU machines demonstrate that the workload classification framework can tell apart the illicit authorized workloads with a high accuracy of over 95%.

Zou, Pengfei↗
DOE OSTI2026

Accelerating Traction Motor Optimization Design with AI Surrogate Models

The advancement of artificial intelligence systems enables the use of data-driven physics-based surrogate models to explore design spaces rapidly and deeply for engineering projects. This work presents a surrogate model workflow that accelerates electric traction motor design optimization by replacing finite element analysis (FEA) with an artificial neural network (ANN) and using this model in a genetic algorithm for design optimization. A baseline interior permanent-magnet motor is parameterized and sampled to generate FEA-labeled training data, after which a feed-forward ANN predicts key outputs (e.g., loss components and weight). The validated surrogate enables genetic-algorithm optimization and deep search over the design space without new FEA runs, producing Pareto-optimal trade-offs between weight and losses and set of optimized designs for rapid downselection of manufacturable motor designs.

Ribeiro, Pedro [ORNL] (ORCID:0009000921026641)↗
DOE OSTI2020

Detecting Anomalous Computation with RNNs on GPU-Accelerated HPC Machines

This paper presents a workload classification framework that discriminates illicit computation from authorized workloads on GPU-accelerated HPC systems. As such systems become more and more powerful, they are exploited by attackers to run malicious and for-profit programs that typically require extremely high computing ability to be successful. Our classification framework leverages the distinctive signatures between illicit and authorized workloads, and explore machine learning methods to learn the workloads and classify them. The framework uses lightweight, non-intrusive workload profiling to collect model input data, and explores multiple machine learning methods, particularly recurrent neural network (RNN) that is suitable for online anomalous workload detection. Evaluation results on three generations of GPU machines demonstrate that the workload classification framework can tell apart the illicit authorized workloads with a high accuracy of over 95%.

Pengfei, Zou↗
DOE OSTI2025

Data and Code for Understanding Generative AI Content with Embedding Models

This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).

Vargas, Max [Pacific Northwest National Laboratory↗
NASA NTRS1990

Investigation of automated task learning, decomposition and scheduling

The details and results of research conducted in the application of neural networks to task planning and decomposition are presented. Task planning and decomposition are operations that humans perform in a reasonably efficient manner. Without the use of good heuristics and usually much human interaction, automatic planners and decomposers generally do not perform well due to the intractable nature of the problems under consideration. The human-like performance of neural networks has shown promise for generating acceptable solutions to intractable problems such as planning and decomposition. This was the primary reasoning behind attempting the study. The basis for the work is the use of state machines to model tasks. State machine models provide a useful means for examining the structure of tasks since many formal techniques have been developed for their analysis and synthesis. It is the approach to integrate the strong algebraic foundations of state machines with the heretofore trial-and-error approach to neural network synthesis.

Livingston, David L.↗
NASA NTRS2001

Do nonlinearities play a significant role in short term, beat-to-beat variability?

Numerous studies of short-term beat-to-beat variability in cardiovascular signals have not resolved the debate about the completeness of linear analysis techniques. This aim of this paper is to evaluate further the role of nonlinearities in short-term, beat-to-beat variability. We compared linear autoregressive moving average (ARMA) and nonlinear neural network (NN) models for predicting instantaneous heart rate (HR) and mean arterial blood pressure (BP) from past HR and BP. To evaluate these models, we used HR and BP time series from the MIMIC database. Experimental results indicate that NN-based nonlinearities do not play a significant role and suggest that ARMA linear analysis techniques provide adequate characterization of the system dynamics responsible for generating short-term, beat-to-beat variability.

NASA Discipline Cardiopulmonary↗
DOE OSTI2023

Using Ultrasound Image Augmentation and Ensemble Predictions to Prevent Machine-Learning Model Overfitting

Deep learning predictive models have the potential to simplify and automate medical imaging diagnostics by lowering the skill threshold for image interpretation. However, this requires predictive models that are generalized to handle subject variability as seen clinically. Here, we highlight methods to improve test accuracy of an image classifier model for shrapnel identification using tissue phantom image sets. Using a previously developed image classifier neural network—termed ShrapML—blind test accuracy was less than 70% and was variable depending on the training/test data setup, as determined by a leave one subject out (LOSO) holdout methodology. Introduction of affine transformations for image augmentation or MixUp methodologies to generate additional training sets improved model performance and overall accuracy improved to 75%. Further improvements were made by aggregating predictions across five LOSO holdouts. This was done by bagging confidences or predictions from all LOSOs or the top-3 LOSO confidence models for each image prediction. Top-3 LOSO confidence bagging performed best, with test accuracy improved to greater than 85% accuracy for two different blind tissue phantoms. This was confirmed by gradient-weighted class activation mapping to highlight that the image classifier was tracking shrapnel in the image sets. Overall, data augmentation and ensemble prediction approaches were suitable for creating more generalized predictive models for ultrasound image analysis, a critical step for real-time diagnostic deployment.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Explicit physics-informed neural networks for nonlinear closure: The case of transport in tissues

In upscaling methods, closures for nonlinear problems present a well-known challenge. While a number of theoretical methods have been proposed for handling such closures, nonlinearities still remain a significant obstacle for many problems. In this work, we use a combination of formal upscaling and data-driven machine learning for explicitly closing a nonlinear transport and reaction process in multiscale tissues. The classical effectiveness factor model is used to formulate the macroscale reaction kinetics. We train a multilayer perceptron network using training data generated by direct numerical simulations over microscale examples. Once trained, the network is used in an algorithm for numerically solving the upscaled (coarse-grained) differential equation describing mass transport and reaction in two example tissues. The network is described as being explicit in the sense that the network is trained using macroscale concentrations and gradients of concentration as components of the feature space rather than incorporating them as part of a constraint in the optimization process. Network training and solutions to the macroscale transport equations were computed for two different tissues. The two tissue types (brain and liver) exhibit markedly different geometrical complexity and spatial scale (cell size and sample size). The upscaled solutions for the average concentration are compared with numerical solutions derived from the microscale concentration fields by a posteriori averaging. There are three outcomes of this work of particular note. 1) Our overall approach results in an upscaled nonlinear PDE. The PDE is closed using a neural network, and our approach results in the definition of the classical effectiveness factor for effecting closure. 2) We identify particular source terms for the closure problem that are important for representing the structure of the closure. These source terms involve macroscale concentrations and their gradients. We adopt these source terms to use as explicit features in the learning algorithm. We find the trained networks that include the macroscale source terms generate models that are able to predict the correction factor with increased fidelity over those that do not. 3) We find that the trained network exhibits good generalizability, and it is able to predict the effectiveness factor with high fidelity for realistically-structured tissues despite the significantly different scale and geometrical complexity of the two example tissue types. This latter result emphasizes our purposeful connection between conventional averaging methods with the use of machine learning for closure; this contrasts with some machine learning methods for upscaling where the exact form of the macroscale equation remains unknown.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Latent-Space Dynamics for Prediction and Fault Detection in Geothermal Power Plant Operations

This paper presents a latent-space dynamic neural network (LSDNN) model for the multi-step-ahead prediction and fault detection of a geothermal power plant’s operation. The model was trained to learn the dynamics of the power generation process from multivariate time-series data and the effects of exogenous variables, such as control adjustment and ambient temperature. In the LSDNN model, an encoder–decoder architecture was designed to capture cross-correlation among different measured variables. In addition, a latent space dynamic structure was proposed to propagate the dynamics in the latent space to enable prediction. The prediction power of the LSDNN was utilized for monitoring a geothermal power plant and detecting abnormal events. The model was integrated with principal component analysis (PCA)-based process monitoring techniques to develop a fault-detection procedure. The performance of the proposed LSDNN model and fault detection approach was demonstrated using field data collected from a geothermal power plant.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Deep learning surrogate model for kinetic Landau-fluid closure with collision

In this work, the kinetic Landau-fluid (LF) closure with collision and periodic boundary condition is used in the development of the deep learning (DL) surrogate model. A classical neural network, namely, feedforward neural network or sometimes termed multilayer perceptron, is constructed and trained to learn the kinetic LF closure in the static limit and arbitrary mean free path in configuration space. The preliminary relation between best hyperparameters and critical parameters for data generation is found. Compared with the numerical approach (non-Fourier method) of the LF closure, the deep learning surrogate model shows an order of magnitude of improvement in terms of accuracy. Perhaps most importantly, the surrogate model closure has been integrated for the first time with fluid simulations. Our DL-enabled fluid simulations, for the first time, give the correct Landau damping rate for a wide range of wave vectors, while the Hammett–Perkins closure cannot produce the correct damping rate. We correctly connect the collisionless Hammett–Perkins closure and collisional Braginskii closure to reproduce the intrinsic nonlocal feature of the heat flux with DL techniques. We address the most concerning error accumulation problem and find that simulations with the deep learning surrogate model are as good as, if not better than, simulations with the analytic closure in terms of long-term numerical stability in the linear Landau damping test.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Hybrid Cyber-attack Detection in Photovoltaic Farms

Here, to address the cyber-physical security in PV farms, a hybrid cyber-attack detection is proposed in this manuscript. To secure PV farms, the proposed method integrates model-based and data-driven methods by fusing the detection score at the device and system levels. First, a model-based cyber-attack detection method is developed for each PV inverter. A residual between the estimation of the Kalman filter and measurement is calculated. By leveraging the calculated residual from all inverters, a squared Mahalanobis distance is developed for device detection score generation. At the system level, a convolutional neural network (CNN) is proposed to detect cyber-attack using the waveform data at the point of common coupling (PCC) in PV farms. To improve the CNN detection accuracy, a set of well-designed features are extracted from the raw waveform data. Finally, a weighted detection score fusion method is proposed to combine device and system detection scores by using their complementary strength. The feasibility and robustness of the proposed method are validated by testing cases and a comparative experiment.

14 SOLAR ENERGY↗
DOE OSTI2025

Data-Driven State of Health Estimation for Second-Life Batteries Using Interpolated Synthetic Data and Feature Selection

Accurate estimation of the State of Health (SOH) for second-life batteries (SLBs) is crucial given their increasing use in energy storage applications. Precise SOH prediction is essential for safe operation and robust battery management systems. A major challenge is the limited availability of datasets for building reliable degradation models. To address this, synthetic data generation through linear interpolation is performed to extend the available data, making it more representative of real-world battery operating conditions. By analyzing feature correlation with SOH, the most relevant features are selected for the model. The proposed approach employs a convolutional neural network (CNN) model trained on this interpolated, feature-selected dataset, using time series data of voltage, temperature, and current over a cycle. By focusing on highly correlated features, the model achieves over 95% accuracy, with mean absolute error and root mean squared error up to 2.27% and 2.64%, respectively, in SOH estimation for two battery datasets tested. These results highlight the potential of combining synthetic data generation and feature selection to enhance SOH predictions, showcasing the superior performance of the proposed CNN model for both new batteries and SLBs.

feature selection↗
DOE OSTI2025

Physics-Informed Graph Neural Networks for Collaborative Dynamic Reconfiguration and Voltage Regulation in Unbalanced Distribution Systems

Network reconfiguration has long been employed as a strategic approach to minimize power distribution system losses and effectively regulate voltage levels. Tap-changing voltage regulators are also critical for controlling bus voltages, especially in accommodating the increasing integration of distributed energy resources (DERs) with intermittent outputs. This paper introduces novel methodologies to address the challenges of dynamic reconfiguration and optimal tap setting in unbalanced three-phase distribution systems. We propose an approximated mixed-integer quadratically constrained program (MIQCP) to model dynamic reconfiguration, along with a pioneering formulation for voltage regulator (VR) tap-setting based on Special Ordered Set type 1 (SOS1). To mitigate computational complexity, we propose a physics-informed spatial-temporal graph convolutional network (STGCN) with an integrated link classifier. The proposed approach enables efficient solution generation by fixing specific variables in the MIQCP instance and solving the simplified sub-MIP using an MIP solver. Numerical studies demonstrate the superior prediction accuracy of our STGCN model compared to baseline neural network models, resulting in reduced DER curtailment and voltage deviation with shorter computation time.

dynamic reconfiguration↗