Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Improving deep learning performance for predicting large-scale geological ${{CO}_{2}}$ sequestration modeling through feature coarsening

Physics-based reservoir simulation for fluid flow in porous media is a numerical simulation method to predict the temporal-spatial patterns of state variables (e.g. pressure p) in porous media, and usually requires prohibitively high computational expense due to its non-linearity and the large number of degrees of freedom (DoF). This work describes a deep learning (DL) workflow to predict the pressure evolution as fluid flows in large-scale 3-dimensional(3D) heterogeneous porous media. In particular, we develop an efficient feature coarsening technique to extract the most representative information and perform the training and prediction of DL at the coarse scale, and further recover the resolution at the fine scale by spatial interpolation. We validate the DL approach to predict pressure field against physics-based simulation data for a field-scale 3D geologic CO 2 sequestration reservoir model. We evaluate the impact of feature coarsening on DL performance, and observe that the feature coarsening not only decreases the training time by >74% and reduces the memory consumption by >75%, but also maintains temporal error 0.63% on average. Besides, the DL workflow provides predictive efficiency with 1406 times speedup compared to physics-based numerical simulation. The key findings from this research significantly improve the training and prediction efficiency of deep learning model to deal with large-scale heterogeneous reservoir models, and thus it can also be further applied to accelerate workflows of history matching and reservoir optimization for close-loop reservoir management.

58 GEOSCIENCES↗
DOE OSTI2025

Performance evaluation of automated data-driven feature extraction and selection methods for practical and scalable building energy consumption prediction models

Here, this study quantifies the impact of automated feature engineering methods (feature extraction and selection) on the quality and accuracy of machine learning models that predict building energy consumption. The case study compares model performance for three main scenarios: baseline (no feature extraction and selection), feature extraction only, and feature extraction combined with feature selection (filter and/or wrapper methods) for fully trained machine learning models for 200 metered/sub-metered energy measurements across 118 real buildings. For consistency, the same machine learning model architecture (a black box deep learning neural network with probabilistic forecast output) was used for all scenarios. Based on results, all feature engineering methods provided noticeable prediction accuracy improvements (e.g., 29%-68% median prediction improvement) compared to baseline scenarios. However, in this application, feature selection methods provide little practical value due to their limited performance gains and high computational cost. Smarter algorithm development supported by better computational environments will be needed before feature selection methods can reliably and efficiently improve predictive model performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bayesian GAN-Based False Data Injection Attack Detection in Active Distribution Grids With DERs

Advancements in information and communication technologies have revolutionized monitoring and control capabilities within smart grids. However, it also brings new vulnerabilities to data acquisition systems and state estimation functions, which attackers can subtly tamper with the measurement data through compromising the communication network. Moreover, the high penetration of renewable energy sources with the inherited characteristics of uncertainty and variability further complicates the design of effective intrusion detection systems. In this paper, a Bayesian deep learning-based approach is developed to detect cyber attacks and maintain the security of smart grids. Our method specifically addresses the prevalent issue of imbalanced data in real power systems, which arises from the predominance of normal system operations over compromised or attacked states. Employing a novel Bayesian GAN-based technique, our approach successfully discriminates between secure and compromised measurement data, even in scenarios with significant data imbalance. Furthermore, the proposed method accommodates various practical application factors, ensuring accurate intrusion detection despite the presence of measurement noise. The feasibility and effectiveness of the proposed detection mechanism are validated by testing on IEEE 13-node and 123-node test systems. Simulation results and comparisons with literature methods demonstrate the superiority of proposed cybersecurity solutions.

Bayesian GAN↗
DOE OSTI2023

Advancing energy storage through solubility prediction: leveraging the potential of deep learning

Solubility prediction plays a crucial role in energy storage applications, such as redox flow batteries, because it directly affects the efficiency and reliability. Researchers have developed various methods that utilize quantum calculations and descriptors to predict the aqueous solubilities of organic molecules. Notably, machine learning models based on descriptors have shown promise for solubility prediction. As deep learning tools, graph neural networks (GNNs) have emerged to capture complex structure–property relationships for material property prediction. Specifically, MolGAT, a type of GNN model, was designed to incorporate n-dimensional edge attributes, enabling the modeling of intricacies in molecular graphs and enhancing the prediction capabilities. In a previous study, MolGAT successfully screened 23 467 promising redox-active molecules from a database of over 500 000 compounds, based on redox potential predictions. This study focused on applying the MolGAT model to predict the aqueous solubility (log S) of a broad range of organic compounds, including those previously screened for redox activity. The model was trained on a diverse sample of 8494 organic molecules from AqSolDB and benchmarked against literature data, demonstrating superior accuracy compared with other state of the art graph-based and descriptor-based models. Subsequently, the trained MolGAT model was employed to screen redox-active organic compounds identified in the first phase of high-throughput virtual screening, targeting favorable solubility in energy storage applications. The second round of screening, which considered solubility, yielded 12 332 promising redox-active and soluble organic molecules suitable for use in aqueous redox flow batteries. Thus, the two-phase high-throughput virtual screening approach utilizing MolGAT, specifically trained for redox potential and solubility, is an effective strategy for selecting suitable intrinsically soluble redox-active molecules from extensive databases, potentially advancing energy storage through reliable material development. This indicates that the model is reliable for predicting the solubility of various molecules and provides valuable insights for energy storage, pharmaceutical, environmental, and chemical applications.

25 ENERGY STORAGE↗
DOE OSTI2022

Device-Centric Firmware Malware Detection for Smart Inverters using Deep Transfer Learning

Since future power grids are inverter-dominant grids and inverters are getting smarter by incorporating remote access and seamless firmware update, it is anticipated that malware attackers will directly target smart inverters. However, malware threats targeting smart inverters have been less studied yet. This paper explores potential malware attacks targeting smart inverters and proposes a deep transfer-learning (DTL)-based malware detection framework for smart inverters. The proposed DTL method can significantly reduce development time and efforts for an artificial intelligence-based malware detection algorithm while improving detection accuracy. The experimental result shows that the proposed method achieves 98% of firmware malware detection accuracy. Furthermore, this approach will be transformative to other smart grid devices enabling seamless firmware update.

artificial intelligence↗
DOE OSTI2024

The expressivity of classical and quantum neural networks on entanglement entropy

Abstract Analytically continuing the von Neumann entropy from Rényi entropies is a challenging task in quantum field theory. While then-th Rényi entropy can be computed using the replica method in the path integral representation of quantum field theory, the analytic continuation can only be achieved for some simple systems on a case-by-case basis. In this work, we propose a general framework to tackle this problem using classical and quantum neural networks with supervised learning. We begin by studying several examples with known von Neumann entropy, where the input data is generated by representing$${\text {Tr}}\rho _A^n$$ Tr ρ A n with a generating function. We adopt KerasTuner to determine the optimal network architecture and hyperparameters with limited data. In addition, we frame a similar problem in terms of quantum machine learning models, where the expressivity of the quantum models for the entanglement entropy as a partial Fourier series is established. Our proposed methods can accurately predict the von Neumann and Rényi entropies numerically, highlighting the potential of deep learning techniques for solving problems in quantum information theory.

Physics↗
DOE OSTI2022

Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics

We present a novel application of the machine learning / artificial intelligence method called boosted decision trees to estimate physical quantities on field programmable gate arrays (FPGA). The software package fwXmachina features a new architecture called parallel decision paths that allows for deep decision trees with arbitrary number of input variables. It also features a new optimization scheme to use different numbers of bits for each input variable, which produces optimal physics results and ultraefficient FPGA resource utilization. Problems in high energy physics of proton collisions at the Large Hadron Collider (LHC) are considered. Estimation of missing transverse momentum (E T miss ) at the first level trigger system at the High Luminosity LHC (HL-LHC) experiments, with a simplified detector modeled by Delphes, is used to benchmark and characterize the firmware performance. The firmware implementation with a maximum depth of up to 10 using eight input variables of 16-bit precision gives a latency value of $\mathcal{O}$(10) ns, independent of the clock speed, and $\mathcal{O}$(0.1)% of the available FPGA resources without using digital signal processors.

Instruments & Instrumentation↗
DOE OSTI2023

Atomic protein structure refinement using all-atom graph representations and SE(3)-equivariant graph transformer

Abstract Motivation The state-of-art protein structure prediction methods such as AlphaFold are being widely used to predict structures of uncharacterized proteins in biomedical research. There is a significant need to further improve the quality and nativeness of the predicted structures to enhance their usability. In this work, we develop ATOMRefine, a deep learning-based, end-to-end, all-atom protein structural model refinement method. It uses a SE(3)-equivariant graph transformer network to directly refine protein atomic coordinates in a predicted tertiary structure represented as a molecular graph. Results The method is first trained and tested on the structural models in AlphaFoldDB whose experimental structures are known, and then blindly tested on 69 CASP14 regular targets and 7 CASP14 refinement targets. ATOMRefine improves the quality of both backbone atoms and all-atom conformation of the initial structural models generated by AlphaFold. It also performs better than two state-of-the-art refinement methods in multiple evaluation metrics including an all-atom model quality score—the MolProbity score based on the analysis of all-atom contacts, bond length, atom clashes, torsion angles, and side-chain rotamers. As ATOMRefine can refine a protein structure quickly, it provides a viable, fast solution for improving protein geometry and fixing structural errors of predicted structures through direct coordinate refinement. Availability and implementation The source code of ATOMRefine is available in the GitHub repository (https://github.com/BioinfoMachineLearning/ATOMRefine). All the required data for training and testing are available at https://doi.org/10.5281/zenodo.6944368.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Spatiotemporal predictions of toxic urban plumes using deep learning

Industrial accidents, chemical spills, and structural fires can release large amounts of harmful materials that disperse into urban atmospheres and impact populated areas. Computer models are typically used to predict the transport of toxic plumes by solving fluid dynamical equations. However, these models can be computationally expensive due to the need for many grid cells to simulate turbulent flow and resolve individual buildings and streets. In emergency response situations, alternative methods are needed that can run quickly and adequately capture important spatiotemporal features. Here, we present a novel deep learning model called ST-GasNet inspired by the mathematical equations that govern the behavior of plumes as they disperse through the atmosphere. ST-GasNet learns the spatiotemporal dependencies from a limited set of temporal sequences of ground-level toxic urban plumes generated by a high-resolution large eddy simulation model. On independent sequences, ST-GasNet accurately predicts the late-time spatiotemporal evolution, given the early-time behavior as an input, even when a building splits a large plume into smaller plumes. By incorporating large-scale wind boundary condition information, ST-GasNet achieves a prediction accuracy of at least 90% on test data for the entire prediction period.

Civil and Environmental Engineering↗
DOE OSTI2023

Combining pairwise structural similarity and deep learning interface contact prediction to estimate protein complex model accuracy in CASP15

Abstract Estimating the accuracy of quaternary structural models of protein complexes and assemblies (EMA) is important for predicting quaternary structures and applying them to studying protein function and interaction. The pairwise similarity between structural models is proven useful for estimating the quality of protein tertiary structural models, but it has been rarely applied to predicting the quality of quaternary structural models. Moreover, the pairwise similarity approach often fails when many structural models are of low quality and similar to each other. To address the gap, we developed a hybrid method (MULTICOM_qa) combining a pairwise similarity score (PSS) and an interface contact probability score (ICPS) based on the deep learning inter‐chain contact prediction for estimating protein complex model accuracy. It blindly participated in the 15th Critical Assessment of Techniques for Protein Structure Prediction (CASP15) in 2022 and performed very well in estimating the global structure accuracy of assembly models. The average per‐target correlation coefficient between the model quality scores predicted by MULTICOM_qa and the true quality scores of the models of CASP15 assembly targets is 0.66. The average per‐target ranking loss in using the predicted quality scores to rank the models is 0.14. It was able to select good models for most targets. Moreover, several key factors (i.e., target difficulty, model sampling difficulty, skewness of model quality, and similarity between good/bad models) for EMA are identified and analyzed. The results demonstrate that combining the multi‐model method (PSS) with the complementary single‐model method (ICPS) is a promising approach to EMA.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Classifying handedness in chiral nanomaterials using label error robust deep learning

Abstract High-throughput scanning electron microscopy (SEM) coupled with classification using neural networks is an ideal method to determine the morphological handedness of large populations of chiral nanoparticles. Automated labeling removes the time-consuming manual labeling of training data, but introduces label error, and subsequently classification error in the trained neural network. Here, we evaluate methods to minimize classification error when training from automated labels of SEM datasets of chiral Tellurium nanoparticles. Using the mirror relationship between images of opposite handed particles, we artificially create populations of varying label error. We analyze the impact of label error rate and training method on the classification error of neural networks on an ideal dataset and on a practical dataset. Of the three training methods considered, we find that a pretraining approach yields the most accurate results across label error rates on ideal datasets, where size and other morphological variables are held constant, but that a co-teaching approach performs the best in practical application.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Data Arrays for Microearthquake (MEQ) Monitoring using Deep Learning for the Newberry EGS Sites

The 'Machine Learning Approaches to Predicting Induced Seismicity and Imaging Geothermal Reservoir Properties' project looks to apply machine learning (ML) methods to Microearthquake (MEQ) data for imaging geothermal reservoir properties and forecasting seismic events, in order to advance geothermal exploration and safe geothermal energy production. As part of the project, this submission provides data arrays for 149 microearthquakes between the year 2012 and 2013 at the Newberry EGS Site for use with the Deep Learning Algorithm that has been developed. The data provided includes raw waveform data, location data, normalized waveform data, and processed waveform data. Penn State Geothermal Team has shared the following files from the project: - 149 microearthquakes (MEQs) between 2012 and 2013 at Newberry EGS sites, 'Normalized Waveform Inputs.npz' are normalized waveforms. - labels of 149 MEQs: Processed Waveform Inputs.npz - location labels of 149 MEQs: Location Data.npz Note: .npz is the python file format by NumPy that provides storage of array data.

15 GEOTHERMAL ENERGY↗
DOE OSTI2026

APSO-enhanced algebraic derivative estimation approach for real-time traffic flow prediction on critical road sections during wildfire evacuation

In rapid-onset disaster scenarios such as wildfires, evacuation traffic often significantly deviates from historical patterns, rendering conventional data-driven forecasting methods less effective. To address this challenge, we propose an improved algebraic derivative estimation (ADE) incorporating particle swarm optimization (PSO) for real-time traffic flow prediction. Our approach dynamically adjusts the ADE prediction time window at each step by minimizing a cost function based on the mean and variance of accumulated forecasting errors within the window, thereby balancing bias and variability. We evaluate the method using traffic data from the January 2025 California wildfires, focusing on key road segments critical for large-scale evacuations. The results demonstrate that our approach surpasses established machine learning and deep learning models—XGBoost, LSTM, and GRU—in predictive accuracy and maintains high computational efficiency. Notably, the proposed method eliminates the need for offline model training. Moreover, rapid PSO-based tuning enables real-time deployment, which provides a crucial advantage in scenarios where evacuation timings and road closures change dynamically. In conclusion, these findings highlight the benefits of the PSO-enhanced ADE framework for emergency traffic management, where rapid, data-sparse forecasts are essential for effective evacuation planning.

Algebraic derivative estimation↗
DOE OSTI2023

Mechanisms of Biomolecular Self-Assembly Investigated Through In Situ Observations of Structures and Dynamics

Biomolecular self-assembly of hierarchical materials is a precise and adaptable bottom-up approach to synthesizing across scales with considerable energy, health, environment, sustainability, and information technology applications. To achieve desired functions in biomaterials, it is essential to directly observe assembly dynamics and structural evolutions that reflect the underlying energy landscape and the assembly mechanism. This report will summarize the current understanding of biomolecular assembly mechanisms based on in situ characterization and discuss the broader significance and achievements of newly gained insights. In addition, we will also introduce how emerging deep learning/machine learning-based approaches, multiparametric characterization, and high-throughput methods can boost the development of biomolecular self-assembly. The objective of this review is to accelerate the development of in situ characterization approaches for biomolecular self-assembly and to inspire the next generation of biomimetic materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Mechanisms of Biomolecular Self–Assembly Investigated Through In Situ Observations of Structures and Dynamics

Biomolecular self-assembly of hierarchical materials is a precise and adaptable bottom-up approach to synthesizing across scales with considerable energy, health, environment, sustainability, and information technology applications. To achieve desired functions in biomaterials, it is essential to directly observe assembly dynamics and structural evolutions that reflect the underlying energy landscape and the assembly mechanism. This review will summarize the current understanding of biomolecular assembly mechanisms based on in situ characterization and discuss the broader significance and achievements of newly gained insights. In addition, we will also introduce how emerging deep learning/machine learning-based approaches, multiparametric characterization, and high-throughput methods can boost the development of biomolecular self-assembly. Furthermore, the objective of this review is to accelerate the development of in situ characterization approaches for biomolecular self-assembly and to inspire the next generation of biomimetic materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

SA-GAT-SR: self-adaptable graph attention networks with symbolic regression for high-fidelity material property prediction

Recent advances in machine learning have demonstrated an enormous utility of deep learning approaches, particularly Graph Neural Networks (GNNs) for materials science. These methods have emerged as powerful tools for high-throughput prediction of material properties, offering a compelling enhancement and alternative to traditional first-principles calculations. While the community has predominantly focused on developing increasingly complex and universal models to enhance predictive accuracy, such approaches often lack physical interpretability and insights into materials behavior. Here, we introduce a novel computational paradigm—Self-Adaptable Graph Attention Networks integrated with Symbolic Regression (SA-GAT-SR)—that synergistically combines the predictive capability of GNNs with the interpretative power of symbolic regression. Our framework employs a self-adaptable encoding algorithm that automatically identifies and adjust attention weights so as to screen critical features from an expansive 180-dimensional feature space while maintaining O(n) computational scaling. The integrated SR module subsequently distills these features into compact analytical expressions that explicitly reveal quantum-mechanically meaningful relationships, achieving 23 × acceleration compared to conventional SR implementations that heavily rely on first-principle calculations-derived features as input. This work suggests a new framework in computational materials science, bridging the gap between predictive accuracy and physical interpretability, offering valuable physical insights into material behavior.

36 MATERIALS SCIENCE↗
DOE OSTI2023

The entropy of galaxy spectra: how much information is encoded?

Abstract The inverse problem of extracting the stellar population content of galaxy spectra is analysed here from a basic standpoint based on information theory. By interpreting spectra as probability distribution functions, we find that galaxy spectra have high entropy, thus leading to a rather low effective information content. The highest variation in entropy is unsurprisingly found in regions that have been well studied for decades with the conventional approach. We target a set of six spectral regions that show the highest variation in entropy – the 4000 Å break being the most informative one. As a test case with real data, we measure the entropy of a set of high-quality spectra from the Sloan Digital Sky Survey, and contrast entropy-based results with the traditional method based on line strengths. The data are classified into star-forming (SF), quiescent (Q), and active galactic nucleus (AGN) galaxies, and show – independently of any physical model – that AGN spectra can be interpreted as a transition between SF and Q galaxies, with SF galaxies featuring a more diverse variation in entropy. The high level of entanglement complicates the determination of population parameters in a robust, unbiased way, and affects traditional methods that compare models with observations, as well as machine learning (especially deep learning) algorithms that rely on the statistical properties of the data to assess the variations among spectra. Entropy provides a new avenue to improve population synthesis models so that they give a more faithful representation of real galaxy spectra.

Ferreras, Ignacio (ORCID:0000000345843127)↗