Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Identifying Genomic Islands with Deep Neural Networks

Background Horizontal gene transfer is the main source of adaptability for bacteria, through which genes are obtained from different sources including bacteria, archaea, viruses, and eukaryotes. This process promotes the rapid spread of genetic information across lineages, typically in the form of clusters of genes referred to as genomic islands (GIs). Different types of GIs exist, and are often classified by the content of their cargo genes or their means of integration and mobility. While various computational methods have been devised to detect different types of GIs, no single method is capable of detecting all types. Results We propose a method, which we call Shutter Island, that uses a deep learning model (Inception V3, widely used in computer vision) to detect genomic islands. The intrinsic value of deep learning methods lies in their ability to generalize. Via a technique called transfer learning, the model is pre-trained on a large generic dataset and then re-trained on images that we generate to represent genomic fragments. We demonstrate that this image-based approach generalizes better than the existing tools. Conclusions We used a deep neural network and an image-based approach to detect the most out of the correct GI predictions made by other tools, in addition to making novel GI predictions. The fact that the deep neural network was re-trained on only a limited number of GI datasets and then successfully generalized indicates that this approach could be applied to other problems in the field where data is still lacking or hard to curate.

Computer Vision↗

Low responsiveness of machine learning models to critical or deteriorating health conditions

Machine learning (ML) based mortality prediction models can be immensely useful in intensive care units. Such a model should generate warnings to alert physicians when a patient’s condition rapidly deteriorates, or their vitals are in highly abnormal ranges. Before clinical deployment, it is important to comprehensively assess a model’s ability to recognize critical patient conditions. We develop multiple medical ML testing approaches, including a gradient ascent method and neural activation map. We systematically assess these machine learning models’ ability to respond to serious medical conditions using additional test cases, some of which are time series. Guided by medical doctors, our evaluation involves multiple machine learning models, resampling techniques, and four datasets for two clinical prediction tasks. We identify serious deficiencies in the models’ responsiveness, with the models being unable to recognize severely impaired medical conditions or rapidly deteriorating health. For in-hospital mortality prediction, the models tested using our synthesized cases fail to recognize 66% of the injuries. In some instances, the models fail to generate adequate mortality risk scores for all test cases. Our study identifies similar kinds of deficiencies in the responsiveness of 5-year breast and lung cancer prediction models. Using generated test cases, we find that statistical machine-learning models trained solely from patient data are grossly insufficient and have many dangerous blind spots. Most of the ML models tested fail to respond adequately to critically ill patients. How to incorporate medical knowledge into clinical machine learning models is an important future research direction.

60 APPLIED LIFE SCIENCES↗

Quantum Neural Networks: Issues, Training, and Applications

Our work in the field aims at explaining the limitations and expressive power of Quantum Machine Learning models, as well as finding feasible training algorithms that could be implemented in near-term Quantum Computers. The promise of Quantum Machine Learning is that by incorporating quantum effects, such as entanglement, into machine learning models researchers can improve model performance and understand more complex datasets. This pledge is particularly pronounced in the design of Quantum neural networks (QNNs), a promising framework for creating quantum algorithms, that promise to outperform classical models by combining the speedups of quantum computation with the widespread successes of deep learning. We show that applying this approach alone to quantum deep learning is problematic given that an excess of entanglement between the hidden and visible layers can destroy the predictive power of our QNN models. We address the barren plateau problem by suggesting the use of a generative, unbounded, nonlinear loss function with simple gradients. The loss function quantifies how much the quantum states generated by the QNNs differ from the data and the goal during training is to minimize it. Finally, we showcase how to use generative training to construct a "classical-quantum" neural network to accurately interpolate between the ground states of a Molecular Hamiltonian, a central question in Quantum Chemistry.

97 MATHEMATICS AND COMPUTING↗

Data-driven wind turbine wake modeling via probabilistic machine learning

Wind farm design primarily depends on the variability of the wind turbine wake flows to the atmospheric wind conditions and the interaction between wakes. Physics-based models that capture the wake flow field with high-fidelity are computationally very expensive to perform layout optimization of wind farms, and, thus, data-driven reduced-order models can represent an efficient alternative for simulating wind farms. In this work, we use real-world light detection and ranging (LiDAR) measurements of wind-turbine wakes to construct predictive surrogate models using machine learning. Specifically, we first demonstrate the use of deep autoencoders to find a low-dimensional latent space that gives a computationally tractable approximation of the wake LiDAR measurements. Then, we learn the mapping between the parameter space and the (latent space) wake flow fields using a deep neural network. Additionally, we also demonstrate the use of a probabilistic machine learning technique, namely, Gaussian process modeling, to learn the parameter-space-latent-space mapping in addition to the epistemic and aleatoric uncertainty in the data. Finally, to cope with training large datasets, we demonstrate the use of variational Gaussian process models that provide a tractable alternative to the conventional Gaussian process models for large datasets. Furthermore, we introduce the use of active learning to adaptively build and improve a conventional Gaussian process model predictive capability. Overall, we find that our approach provides accurate approximations of the wind-turbine wake flow field that can be queried at an orders-of-magnitude cheaper cost than those generated with high-fidelity physics-based simulations.

Deep neural networks↗

Assessing parallel path cooling tower performance via artificial neural networks

Real-time monitoring of a research nuclear reactor, a system in which all generated power is dissipated to the environment, can be performed via analysis of the heat rejection from the cooling system. Given an inlet water temperature and flow rate, the reactor power can be well-approximated from the outlet water temperature; however, the instrumentation to measure outlet conditions may not be robust or accurate. If we know how a cooling tower performs from historical data, but cannot measure the outlet temperature, a mathematical representation of the system can be inverted to obtain the outlet water temperature that describes the cooling capacity. Unfortunately, model inversion processes are computationally expensive. To address this, an artificial neural network (ANN) is implemented to assess the performance of a multi-cell cooling tower for a nuclear reactor. This approach leverages the Merkel model to obtain an extensive data set describing performance of the cooling tower cells throughout a wide array of potential operating conditions. The Merkel model is expressed as a function of four parameters: the inlet and outlet water temperatures, inlet air wet bulb temperature, and ratio of liquid-to-gas mass flow rates (L/G), which together provide a non-dimensional number indicative of cooling tower performance, called the Merkel integral. Computing a 4-dimensional data structure that describes finite combinations of the Merkel integral, an inverse model is then generated using an ANN to determine the cell outlet water temperature from the other three model parameters along with the computed Merkel integral. Compared to traditional model inversion methods, the ANN reduces the computational time by approximately 4 orders of magnitude, with effectively no sacrifice to solution accuracy, and could be applied for different cooling towers in the event the performance curve is known. Finally, three use cases of the ANN are then reviewed: (1) determining the cell outlet water temperatures when gas flow at rated conditions (GFRC) is known, (2) performing the prior case without knowledge of the GRFC, and (3) assessing performance differences between the individual tower cells.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Optimization with Neural Network Feasibility Surrogates: Formulations and Application to Security-Constrained Optimal Power Flow

In many areas of constrained optimization, representing all possible constraints that give rise to an accurate feasible region can be difficult and computationally prohibitive for online use. Satisfying feasibility constraints becomes more challenging in high-dimensional, non-convex regimes which are common in engineering applications. A prominent example that is explored in the manuscript is the security-constrained optimal power flow (SCOPF) problem, which minimizes power generation costs, while enforcing system feasibility under contingency failures in the transmission network. In its full form, this problem has been modeled as a nonlinear two-stage stochastic programming problem. In this work, we propose a hybrid structure that incorporates and takes advantage of both a high-fidelity physical model and fast machine learning surrogates. Neural network (NN) models have been shown to classify highly non-linear functions and can be trained offline but require large training sets. In this work, we present how model-guided sampling can efficiently create datasets that are highly informative to a NN classifier for non-convex functions. We show how the resultant NN surrogates can be integrated into a non-linear program as smooth, continuous functions to simultaneously optimize the objective function and enforce feasibility using existing non-linear solvers. Overall, this allows us to optimize instances of the SCOPF problem with an order of magnitude CPU improvement over existing methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Lunar Gateway Charging and Effects on Low-Energy Electron Measurements

The Electron Electrostatic Analyzer (EEA) is part of the HERMES instrument suite on Lunar Gateway and is designed to measure electrons in the energy range from 1 eV to 18 keV. Electrons in this range can be deflected or blocked from entering a detector by electric fields from a charged spacecraft platform, so many heliophysics missions (MMS, Cluster, etc.) use active potential control to minimize stray electric fields which would corrupt low-energy plasma measurements. Gateway, however, does not have such capabilities so the surface charging effects will influence the electron data. Further, some elements of Gateway partially block the instrument’s field of view to space. The effects of the Gateway platform on the low-energy electron population must be understood to maximize the accuracy of the EEA data. To investigate the electrostatic environment near Gateway and its effects on the EEA measurements, we use Nascap-2K to model the expected charging of Gateway under representative solar wind, magnetosheathic, and magnetospheric conditions. Next, we use the potential and electric field outputs from Nascap-2K along with in-house particle tracing codes to investigate how low-energy electrons are perturbed, blocked, or deflected by the spacecraft. We discuss how these perturbations impact the plasma moments computed from the EEA data and possible approaches for correcting the charging effects including application of advanced methods such as neural networks. In addition to electrons from the space environment being influenced by the electrostatic environment around Gateway, photoelectrons generated on Gateway surfaces will also be measured by EEA. We are investigating how knowledge of the potentials of various Gateway surfaces can be extracted from these measured photoelectrons using advanced numerical modeling and analysis. Such information would be valuable for validating charging models and planning mission operations around the platform. Finally, we discuss how the specific workflow developed for Gateway can be extended to a generalized process that is applicable to other future missions. As the space industry prepares for human exploration beyond low-Earth orbit, the need to obtain accurate space weather data from rideshare science payloads is critical. Our process for removing platform effects from the science data serves as a pathfinder for leveraging advanced simulations to maximize science returns and ensure the safety and success of future crewed missions.

Miles Bengtson↗

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗

High‐Resolution National‐Scale Water Modeling Is Enhanced by Multiscale Differentiable Physics‐Informed Machine Learning

Abstract The National Water Model (NWM) is a key tool for flood forecasting, planning, and water management. Key challenges facing the NWM include calibration and parameter regionalization when confronted with big data. We present two novel versions of high‐resolution (∼37 km 2 ) differentiable models (a type of hybrid model): one with implicit, unit‐hydrograph‐style routing and another with explicit Muskingum‐Cunge routing in the river network. The former predicts streamflow at basin outlets whereas the latter presents a discretized product that seamlessly covers rivers in the conterminous United States (CONUS). Both versions use neural networks to provide a multiscale parameterization and process‐based equations to provide a structural backbone, which were trained simultaneously (“end‐to‐end”) on 2,807 basins across the CONUS and evaluated on 4,997 basins. Both versions show great potential to elevate future NWM performance for extensively calibrated as well as ungauged sites: the median daily Nash‐Sutcliffe efficiency of all 4,997 basins is improved to around 0.68 from 0.48 of NWM3.0. As they resolve spatial heterogeneity, both versions greatly improved simulations in the western CONUS and also in the Prairie Pothole Region, a long‐standing modeling challenge. The Muskingum‐Cunge version further improved performance for basins >10,000 km 2 . Overall, our results show how neural‐network‐based parameterizations can improve NWM performance for providing operational flood predictions while maintaining interpretability and multivariate outputs. The modeling system supports the Basic Model Interface (BMI), which allows seamless integration with the next‐generation NWM. We also provide a CONUS‐scale hydrologic data set for further evaluation and use.

Song, Yalan [Civil and Environmental Engineering T↗

Using Machine Learning to Infer Pre-Entry Properties for Asteroid Threat Analysis

Accurately assessing asteroid threats relies on knowledge of the asteroid’s pre-entry properties such as size, velocity, and mass. Directly measuring these properties can be infeasible due to the sparsity of events and the accuracy and fidelity of various sensors. Current analysis of an asteroid’s pre-entry properties involves modeling the asteroid’s entry into the Earth’s atmosphere. This process can be time consuming and can require manual adjustment of uncertain modeling specific parameters. NASA Ames has developed a genetic algorithm that can help automate asteroid modeling using the Fragment-Cloud Model (FCM). The algorithm generates realistic energy deposition curves based on actual energy deposition curves from real, observed asteroids. By using these synthetic, labeled energy deposition curves, we developed a one-dimensional convolutional neural network that can predict an asteroid’s pre-entry parameters.

ATAP↗

Adaptive 3D convolutional neural network-based reconstruction method for 3D coherent diffraction imaging

We present a novel adaptive machine-learning based approach for reconstructing three-dimensional (3D) crystals from coherent diffraction imaging. We represent the crystals using spherical harmonics (SH) and generate the corresponding synthetic diffraction patterns. We utilize 3D convolutional neural networks (CNNs) to learn a mapping between 3D diffraction volumes and the SH, which describe the boundary of the physical volumes from which they were generated. We use the 3D CNN-predicted SH coefficients as the initial guesses, which are then fine-tuned using adaptive model-independent feedback for improved accuracy. We also adaptively tune the locations, intensities, and decay rates of collections of radial basis functions in order to reproduce the non-uniform internal structure of 3D objects and demonstrate the method for a synthetic volume that has an internal void and a density ramp.

36 MATERIALS SCIENCE↗

Semi-supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach

Modeling dynamical systems is crucial for a wide range of tasks, but it remains challenging due to complex nonlinear dynamics, limited observations, or lack of prior knowledge. Recently, data-driven approaches such as Neural Ordinary Differential Equations (NODE) have shown promising results by leveraging the expressive power of neural networks to model unknown dynamics. However, these approaches often suffer from limited labeled training data, leading to poor generalization and suboptimal predictions. On the other hand, semi-supervised algorithms can utilize abundant unlabeled data and have demonstrated good performance in classification and regression tasks. We propose TS-NODE, the first semi-supervised approach to modeling dynamical systems with NODE. TS-NODE explores cheaply generated synthetic pseudo rollouts to broaden exploration in the state space and to tackle the challenges brought by lack of ground-truth system data under a teacher-student model. TS-NODE employs an unified optimization framework that corrects the teacher model based on the student's feedback while mitigating the potential false system dynamics present in pseudo rollouts. TS-NODE demonstrates significant performance improvements over a baseline Neural ODE model on multiple dynamical system modeling tasks.

Wang, Yu↗

Accelerating defect predictions in semiconductors using graph neural networks

First-principles computations reliably predict the energetics of point defects in semiconductors but are constrained by the expense of using large supercells and advanced levels of theory. Machine learning models trained on computational data, especially ones that sufficiently encode defect coordination environments, can be used to accelerate defect predictions. Here, we develop a framework for the prediction and screening of native defects and functional impurities in a chemical space of group IV, III–V, and II–VI zinc blende semiconductors, powered by crystal Graph-based Neural Networks (GNNs) trained on high-throughput density functional theory (DFT) data. Using an innovative approach of sampling partially optimized defect configurations from DFT calculations, we generate one of the largest computational defect datasets to date, containing many types of vacancies, self-interstitials, anti-site substitutions, impurity interstitials and substitutions, as well as some defect complexes. We applied three types of established GNN techniques, namely crystal graph convolutional neural network, materials graph network, and Atomistic Line Graph Neural Network (ALIGNN), to rigorously train models for predicting defect formation energy (DFE) in multiple charge states and chemical potential conditions. We find that ALIGNN yields the best DFE predictions with root mean square errors around 0.3 eV, which represents a prediction accuracy of 98% given the range of values within the dataset, improving significantly on the state-of-the-art. We further show that GNN-based defective structure optimization can take us close to DFT-optimized geometries at a fraction of the cost of full DFT. The current models are based on the semi-local generalized gradient approximation-Perdew–Burke–Ernzerhof (PBE) functional but are highly promising because of the correlation of computed energetics and defect levels with higher levels of theory and experimental data, the accuracy and necessity of discovering novel metastable and low energy defect structures at the PBE level of theory before advanced methods could be applied, and the ability to train multi-fidelity models in the future with new data from non-local functionals. The DFT-GNN models enable prediction and screening across thousands of hypothetical defects based on both unoptimized and partially optimized defective structures, helping identify electronically active defects in technologically important semiconductors.

Rahman, Md Habibur (ORCID:000000027705984X)↗

Statistical Treatment of Convolutional Neural Network Superresolution of Inland Surface Wind for Subgrid-Scale Variability Quantification

Abstract Machine learning models have been employed to perform either physics-free data-driven or hybrid dynamical downscaling of climate data. Most of these implementations operate over relatively small downscaling factors because of the challenge of recovering fine-scale information from coarse data. This limits their compatibility with many global climate model outputs, often available between ∼50- and 100-km resolution, to scales of interest such as cloud resolving or urban scales. This study systematically examines the capability of a type of superresolving convolutional neural network (SR-CNNs) to downscale surface wind speed data over land from different coarse resolutions (25-, 48-, and 100-km resolution) to 3 km. For each downscaling factor, we consider three convolutional neural network (CNN) configurations that generate superresolved predictions of fine-scale wind speed, which take between one and three input fields: coarse wind speed, fine-scale topography, and diurnal cycle. In addition to fine-scale wind speeds, probability density function parameters are generated through which sample wind speeds can be generated, accounting for the intrinsic stochasticity of wind speed. For assessing generalization to new data, CNN models are tested on regions with different topography and climate that are unseen during training. The evaluation of superresolved predictions focuses on subgrid-scale variability and the recovery of extremes. Models with coarse wind and fine topography as inputs exhibit the best performance when compared with other model configurations, operating across the same downscaling factor. Our diurnal cycle encoding results in lower out-of-sample generalizability when compared with other input configurations.

17 WIND ENERGY↗

Toward ultra-efficient high-fidelity predictions of wind turbine wakes: Augmenting the accuracy of engineering models with machine learning

This study proposes a novel machine learning (ML) methodology for the efficient and cost-effective prediction of high-fidelity three-dimensional velocity fields in the wake of utility-scale turbines. The model consists of an autoencoder convolutional neural network with U-Net skipped connections, fine-tuned using high-fidelity data from large-eddy simulations (LES). The trained model takes the low-fidelity velocity field cost-effectively generated from the analytical engineering wake model as input and produces the high-fidelity velocity fields. The accuracy of the proposed ML model is demonstrated in a utility-scale wind farm for which datasets of wake flow fields were previously generated using LES under various wind speeds, wind directions, and yaw angles. Comparing the ML model results with those of LES, the ML model was shown to reduce the error in the prediction from 20% obtained from the Gauss Curl hybrid (GCH) model to less than 5%. In addition, the ML model captured the non-symmetric wake deflection observed for opposing yaw angles for wake steering cases, demonstrating a greater accuracy than the GCH model. The computational cost of the ML model is on par with that of the analytical wake model while generating numerical outcomes nearly as accurate as those of the high-fidelity LES.

Mechanics↗

AI-Powered Knowledge Graphs for Neuromorphic and Energy-Efficient Computing

The surge in scientific literature obscures breakthroughs and hinders the discovery of new research paths. We propose an artificial intelligence (AI) powered framework using large language models (LLMs) and knowledge graphs (KGs) to automate parts of scientific discovery, focusing on energy-efficient AI circuits. Our hybrid approach combines LLMs, structured data, and ontology-based reasoning to construct a comprehensive knowledge graph that integrates insights across computational neuroscience, spiking neuron models, learning rules, architectural motifs, and neuromorphic device technologies. This multi-domain representation enables the generation of hypotheses that connect biological function with implementable, energy-efficient hardware architectures. Using KG embeddings and graph neural networks, the framework generates hypotheses for novel circuits, validates them through optimization on exascale HPC systems, and with tools like SuperNeuro and Fugu, the most promising designs will be prototyped in hardware. This open-source system aims to accelerate discoveries and bridging neuroscience with hardware innovation, drive collaboration, and unlock new opportunities in low-power AI computing.

Gautam, Ashish [ORNL]↗

One-Step Ahead Prediction of Thermal Mixing Tee Sensors with Long Short Term Memory (LSTM) Neural Networks

High-temperature advanced reactors under development, such as sodium fast reactors (SFR) and molten salt cooled reactors (MSCR), are expected to offer lower levelized cost of energy (LCOE) compared to existing light water reactor (LWR’s). In the existing light water reactors (LWR’s), operation and maintenance (O&M) expenses constitute the largest fraction of the total operating cost. Some of the O&M costs are related maintenance of sensors which can fail due to exposure to harsh environment in a reactor. The O&M costs of Advanced Reactor (AR)’s are expected to constitute a significant fraction of the total cost as well, because of high temperature and radiation level in AR are likely to cause material fatigue and premature failure of sensors and components. The O&M costs in AR’s could be reduced through integration of advanced informatics of performance-related sensors into a digital twin designed for reactor monitoring. For example, machine learning (ML) could be employed for real-time validation and correction of performance-related sensors, and reducing the number of performance-related physical sensor units through virtual sensing. As part of the effort, we investigate real-time validation of thermal hydraulic sensors through one-step ahead forecasting of sensor values using long short-term memory (LSTM) recurrent neural networks (RNN). The sensors are installed in a flow loop containing a thermal mixing tee, which is a common experimental model to study thermal fatigue in a thermal hydraulic loop. In addition, nonlinear transients generated in a thermal mixing tee constitute a good challenge data set for training and validation of ML algorithms. Sensors in this study include thermocouples, flow meters, and optical fibers for distributed temperature sensing. In one experiment, measurement data sets were obtained for a loop was filled with water, and in another experiment, measurements were performed on a loop filled with liquid metal Galinstan. We have also conducted preliminary investigation of one-step ahead prediction of fiber optics-based distributed temperature sensing with LSTM networks. In predicting fiber-based temperature measurements, we treated each gauge pitch of the fiber as an independent sensor. Accuracy of one-step ahead forecasting was estimated by calculating root mean square error (RMSE) for the test segment of time series of each sensor. RMSE’s for temperature sensors in water loop were, for the most part, lower than for the same sensors in Galinstan loop. The RMSE’s for flow meters were similar for both loops. The RMSE’s for distributed temperature measured with the fiber optic sensor were similar to those of the point sensors. Results of this study demonstrated the capability of LSTM one-step ahead forecasting with RMSE comparable to uncertainty in sensor measurements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗