SEARCH · Engineering Papers

Results for “Traditional Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Single Cell RNA-Seq and Machine Learning Reveal Novel Subpopulations in Low-Grade Inflammatory Monocytes With Unique Regulatory Circuits

Subclinical doses of LPS (SD-LPS) are known to cause low-grade inflammatory activation of monocytes, which could lead to inflammatory diseases including atherosclerosis and metabolic syndrome. Sodium 4-phenylbutyrate is a potential therapeutic compound which can reduce the inflammation caused by SD-LPS. To understand the gene regulatory networks of these processes, we have generated scRNA-seq data from mouse monocytes treated with these compounds and identified 11 novel cell clusters. We have developed a machine learning method to integrate scRNA-seq, ATAC-seq, and binding motifs to characterize gene regulatory networks underlying these cell clusters. Using guided regularized random forest and feature selection, our method achieved high performance and outperformed a traditional enrichment-based method in selecting candidate regulatory genes. Our method is particularly efficient in selecting a few candidate genes to explain observed expression pattern. In particular, among 531 candidate TFs, our method achieves an auROC of 0.961 with only 10 motifs. Finally, we found two novel subpopulations of monocyte cells in response to SD-LPS and we confirmed our analysis using independent flow cytometry experiments. Our results suggest that our new machine learning method can select candidate regulatory genes as potential targets for developing new therapeutics against low grade inflammation.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics

Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that are simultaneously accurate, transferable, and interpretable.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

A machine learning pipeline for membrane segmentation of cryo-electron tomograms

We describe how to use several machine learning techniques organized in a learning pipeline to segment and identify cell membrane structures from cryo electron tomograms. These tomograms are difficult to analyze with traditional segmentation tools. The learning pipeline in our approach starts from supervised learning via a special convolutional neural network trained with simulated data. It continues with semi-supervised reinforcement learning and/or a region merging technique that tries to piece together disconnected components belonging to the same membrane structure. A parametric or non-parametric fitting procedure is then used to enhance the segmentation results and quantify uncertainties in the fitting. Domain knowledge is used in generating the training data for the neural network and in guiding the fitting procedure through the use of appropriately chosen priors and constraints. We demonstrate that the approach proposed here works well for extracting membrane surfaces in two real tomogram datasets.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Can artificial intelligence and data-driven machine learning models match or even replace process-driven hydrologic models for streamflow simulation?: A case study of four watersheds with different hydro-climatic regions across the CONUS

With recent developments in computational techniques, Data-driven Machine Learning Models (DMLs) have shown great potential in simulating streamflow and capturing the rainfall-runoff relationship in given watersheds, which are traditionally fulfilled by Process-based Hydrologic Models (PHMs). There are debates on whether the DMLs can outperform and possibly replace the classical PHMs for streamflow simulation and river forecasting, but no clear conclusions have been made. This study aims to investigate whether the newer DMLs have any potential in further improving the simulation accuracy of classical PHMs, and vice versa. To do this, we compared a few popular PHMs and DMLs over four watersheds across the Continental US (CONUS) that are associated with different input, climate, and regional conditions. A total of five hydrologic models were chosen, including (1) two classical lumped models, i.e., the Sacramento Soil Moisture Accounting (SAC-SMA) and Xinanjiang (XAJ); (2) one modern distributed model, termed Coupled Routing and Excess Storage (CREST); (3) and two DMLs including an Artificial Neural Networks (ANN) and a deep learning model, termed Long Short Term Memory (LSTM). Our results demonstrated that the DMLs still significantly biased when using the baseline input scenario with the PHMs. However, the DMLs fed with delayed input scenarios had great potential and can reach high simulation accuracy. The DMLs, especially the ANN, outperformed other employed models under the rainfall-runoff relationship in which rainfall dominantly drives. Furthermore, the DMLs also showed better performance in the high-flow regime, while the PHMs had a better performance for the low-flow regime, implying both PHMs and DMLs have their own merits and are worthy of joint development. In general, our study indicated a great potential of using DMLs to simulate streamflow, but further studies are still needed to verify the transferability and scalability of DMLs in large-scale experiments, such as the Distributed Model Intercomparison Projects 1&2 conducted by National Weather Services but to compare modern DMLs and PHMs.

58 GEOSCIENCES↗

DOE OSTI2024

Artificial Intelligence for Conjugated Polymers

Conjugated polymers have garnered significant attention due to their diverse applications in electronics, photonics, and energy storage. However, realizing their full potential poses a formidable challenge, as their design has historically relied on iterative adjustments and continuous inspiration from researchers. Traditional methods often struggle to efficiently navigate their vast chemical landscape. In this work, the application of artificial intelligence (AI), specifically machine learning (ML), needs to be discussed in the realm of conjugated polymers. Our paper emphasizes the importance of understanding the structure–property relationships of these polymers and how ML can facilitate property prediction and inverse-design. We delve into various chemical fingerprints, structural descriptors, and ML algorithms, showcasing their utility across a spectrum of applications, including simulations, glass transition temperature determination, photovoltaics, reorganization energy for charge transport, photocatalysts, and sensors. Finally, we give some outlooks in this filed and propose unexplored areas within the field that hold the potential to benefit from ML techniques.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Application of Modified Meshgraphnets for Subsurface Prediction during CO2 Sequestration

In the face of the increasingly dire consequences of anthropogenic climate change, capturing and storing carbon dioxide is paramount. However, several impediments exist to the safe and effective subsurface storage of CO2, such as cost of transport, identification of suitable sites for subsurface storage, and assessment of long-term risk from storage in subsurface aquifers. Accurate subsurface modeling is necessary to ensure that CO2 storage is both safe and effective. Still, such modeling has traditionally required either substantial time and computational power (numerical simulation) or a substantial amount of pre-existing data for training (machine learning models). Additionally, these models lack flexibility in dealing with both changes in discretization of the input data and generalizability beyond the data on which they are trained. In order to address these issues, this research applies graph neural networks (GNNs) to predict subsurface saturation and pressure during CO₂ injection in a model of the Illinois Basin-Decatur Project (IBDP). GNNs provide a flexible, intuitive method for representing and manipulating complex unstructured data, which is often found in many practical domain problems such as fluid flow and subsurface characterization. These unstructured grids are easily represented in GNNs by representing spatially-localized features such as permeability, porosity, saturation, and pressure as nodes in a graph and relationships between these properties as edges connecting these nodes. This research applies a specific GNN model called MeshGraphNets (MGN) to model the change in CO2 saturation and pressure over a 50-month time period (36 months of injection, 14 months post-injection). The MGN model leverages a message passing process that allows the network to learn both the spatial and temporal dynamics of this system simultaneously. Additionally, training on a limited dataset (64 realizations, 20 time points each) resulted in a high degree of accuracy in saturation prediction both within the same timeframe as the training (20 months, 0.039 average RMSE) and when projecting out to the end of injection (36 months, 0.053 average RMSE). Temporal predictions such as those generated by MGNs and other similar models are prone to accumulated error over time; in order to address this, a multi-step rollout (MSR) training process was applied to calculate training loss. This method mimics the forward prediction during inference by “rolling out” multiple time points in a single training step using the previous prediction as input to the MGN model. By calculating the loss several time steps forward from the current prediction, the model is forced to find a more stable state over time. Application of MSR to the MGN model resulted in an average 15% reduction in inference error over time during forward prediction. This study showcases the immense potential of GNNs as a game-changing methodology for predicting pressure and saturation evolution in CCS projects, ultimately paving the way for more sustainable and effective carbon storage solutions. Presentation prepared for the 2024 AiChE Annual Meeting, October 27 to November 1 2024, San Diego, CA.

Holcomb, Paul↗

DOE OSTI2025

Landscaper v1

Understanding the inner workings of machine learning models through their loss landscapes offers crucial insights into model properties, optimization dynamics, and generalizability. However, accessing these insights has traditionally required specialized mathematical expertise, limiting broader adoption. Landscaper is an open-source Python package designed to bridge this gap. Landscaper seamlessly integrates a suite of multi-dimensional loss landscape analyses with cutting-edge topological data analysis (TDA) methods. This powerful combination makes both fundamental loss landscape analysis and advanced TDA techniques accessible to the broader scientific ML community, without requiring deep pre-existing mathematical knowledge. Landscaper offers three key functionalities: * Construction: Builds detailed loss landscape representations through versatile low and high-dimensional sampling techniques. * Quantification: Applies advanced metrics, including a novel topological data analysis (TDA) based smoothness metric, enabling new perspectives on model behavior. * Visualization: Offers intuitive tools to visualize and interpret loss landscapes, providing actionable insights beyond traditional performance metrics.

Weber, Gunther [Lawrence Berkeley National Laborat↗

NASA NTRSDate not supplied

Power and Propulsion: Small Core Advanced Thermal Management Project Overview

This presentation is intended to provide an overview of the various efforts under the Advanced Air Transport Technology (AATT) project related to engine thermal management. Sustainable flight is the key motivator for all our efforts, and innovative thermal technologies play a crucial role in achieving this goal. The technologies described in this presentation can be applied to both traditional gas turbines and advanced cycles which utilize alternative fuels. Within our project, we utilize the capabilities of artificial intelligence (AI), machine learning (ML), and additive manufacturing. This ranges from using AI to generate heat exchanger fin topologies, to using ML for a reduction in computational cost which allows for a more thorough design exploration. Many times, the resulting topologies can only be realized through additive manufacturing techniques. Most of what’s presented is currently low TRL, but the intent is to achieve TRL 4 by the end of the project.

Propulsion↗

DOE OSTI2020

Learning Optimal Solutions for Extremely Fast AC Optimal Power Flow

We develop, in this paper, a machine learning approach to optimize the real-time operation of electric power grids. In particular, we learn feasible solutions to the AC optimal power flow (OPF) problem with negligible optimality gaps. The AC OPF problem aims at identifying optimal operational conditions of the power grids that minimize power losses and/or generation costs. Due to the computational challenges with solving this nonconvex problem, many efforts have focused on linearizing or approximating the problem in order to solve the AC OPF on faster timescales. However, many of these approximations can be fairly poor representations of the actual system state and still require solving an optimization problem, which can be time consuming for large networks. In this work, we learn a mapping between the system loading and optimal generation values, enabling us to find near-optimal and feasible AC OPF solutions. This allows us to bypass solving the traditionally nonconvex AC OPF problem, resulting in a significant decrease in computational burden for grid operators.

machine learning↗

DOE OSTI2020

Learning Optimal Solutions for Extremely Fast AC Optimal Power Flow: Preprint

machine learning↗

DOE OSTI2026

Machine Learning Interatomic Potentials for Modeling Framework Flexibility and Water Uptake in NbOFFIVE-1-Ni Metal–Organic Framework

Metal–organic frameworks (MOFs), with their distinctive porous structures and tunable chemical properties, have shown immense promise in the separation and storage of gases. Currently, the accurate simulation of their adsorptive properties remains challenging, especially for systems where the molecules fit very tightly into the pores. Traditional simulation methods often approximate the frameworks as rigid and do not account for the framework flexibility seen in materials such as NbOFFIVE-1-Ni. First-principles molecular dynamics (FPMD) simulations offer the desired accuracy in modeling this flexibility but are limited by their extensive computational demands, rendering them impractical for long simulations. Conversely, classical force field-based simulations offer computational efficiency but lack the necessary accuracy. Here, to break this accuracy-efficiency trade-off, we have developed machine learning interatomic potentials trained on energies and forces from FPMD to model the framework flexibility of NbOFFIVE-1-Ni in the presence of water over nanosecond time scales. Furthermore, by integrating MLIP-driven molecular dynamics (MLIP-MD) with grand canonical Monte Carlo (GCMC) simulations, we further incorporated framework flexibility into adsorption predictions, yielding water adsorption isotherms that better align with experimental data compared to those of conventional GCMC simulations. These advances offer new opportunities for the design and optimization of MOFs in gas storage and separation applications.

adsorption↗

DOE OSTI2026

Describing Point Defect Topology in 2D Energy Materials Through Computer Vision

Point defects such as vacancies and impurity atoms strongly impact the performance of 2D materials. Traditional efforts often rely on manual detection, a process that is time-intensive, prone to human error, and challenging to scale. Here we leverage machine learning (ML) methods to identify and quantify vacancies within 2D transition metal carbides (Ti3C2, MXenes), aiming to expedite detection while improving accuracy. MXenes exhibit valuable defect-defined electrochemical properties, but we currently lack statistical understanding of defect topology needed to fully harness these materials. Here we employ a convolutional neural network for semantic segmentation of experimental MXene images, opening an opportunity to conduct a rigorous statistical study on defect hierarchy while investigating local relaxation in the lattice. We show how the integration of ML can yield fundamental insight into point defects, providing a powerful tool that will play an increasingly crucial role in the future of materials science. ML is often not just a matter of straightforward application, and pretrained models proved ineffective in this case. Instead, we trained our own neural network (NN) and applied data augmentation techniques and fine-tuning to the training dataset. Since labeled microscopy data is often scarce, we developed training data from a previously published wide-frame MXene image, using customized Gaussian fitting to locate atomic positions. Our trained model was then applied to a large dataset of experimental images, enabling a statistical study of defect configurations across three samples prepared with different HF etchant concentrations (5%, 9.1%, and 12.5%), as shown in Fig. 1. This also allowed us to investigate local strain around vacancies, though we find that we are limited by the precision of measurements using high-angle annular dark field (HAADF) images, as shown in Fig. 2. This study demonstrates how ML enables large-scale, quantitative analysis of atomic defects - an otherwise infeasible task with traditional methods. While our NN was specialized for Ti3C2 MXenes, the pipeline we developed provides a foundation for future ML models tailored to other materials. Ultimately, we envision embedding the NN onto the microscope to give real-time feedback to the user. To make this a reality, continued work is necessary to fully understand the NN's capabilities and limitations. This study gets one step closer to our goals of automated experimentation moving away from traditional methods of manual labeling. As ML capabilities advance, we hope to continue adapting and applying these techniques in microscopy.

2D materials↗

DOE OSTI2024

Trust-Based Detection and Mitigation of Cyber Attacks in Distributed Cooperative Control of Islanded AC Microgrids

In this study, we address the challenge of detecting and mitigating cyber attacks in the distributed cooperative control of islanded AC microgrids, with a particular focus on detecting False Data Injection Attacks (FDIAs), a significant threat to the Smart Grid (SG). The SG integrates traditional power systems with communication networks, creating a complex system with numerous vulnerable links, making it a prime target for cyber attacks. These attacks can lead to the disclosure of private data, control network failures, and even blackouts. Unlike machine learning-based approaches that require extensive datasets and mathematical models dependent on accurate system modeling, our method is free from such dependencies. To enhance the microgrid’s resilience against these threats, we propose a resilient control algorithm by introducing a novel trustworthiness parameter into the traditional cooperative control algorithm. Our method evaluates the trustworthiness of distributed energy resources (DERs) based on their voltage measurements and exchanged information, using Kullback-Leibler (KL) divergence to dynamically adjust control actions. We validated our approach through simulations on both the IEEE-34 bus feeder system with eight DERs and a larger microgrid with twenty-two DERs. The results demonstrated a detection accuracy of around 100%, with millisecond range mitigation time, ensuring rapid system recovery. Additionally, our method improved system stability by up to almost 100% under attack scenarios, showcasing its effectiveness in promptly detecting attacks and maintaining system resilience. These findings highlight the potential of our approach to enhance the security and stability of microgrid systems in the face of cyber threats.

Computer Science↗

DOE OSTI2025

Nanobody screening and machine learning guided identification of cross-variant anti-SARS-CoV-2 neutralizing heavy-chain only antibodies

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) continues to persist, demonstrating the risks posed by emerging infectious diseases to national security, public health, and the economy. Development of new vaccines and antibodies for emerging viral threats requires substantial resources and time, and traditional development platforms for vaccines and antibodies are often too slow to combat continuously evolving immunological escape variants, reducing their efficacy over time. Previously, we designed a next-generation synthetic humanized nanobody (Nb) phage display library and demonstrated that this library could be used to rapidly identify highly specific and potent neutralizing heavy chain-only antibodies (HCAbs) with prophylactic and therapeutic efficacy in vivo against the original SARS-CoV-2. In this study, we used a combination of high throughput screening and machine learning (ML) models to identify HCAbs with potent efficacy against SARS-CoV-2 viral variants of interest (VOIs) and concern (VOCs). To start, we screened our highly diverse Nb phage display library against several pre-Omicron VOI and VOC receptor binding domains (RBDs) to identify panels of cross-reactive HCAbs. Using HCAb affinity for SARS-CoV-2 VOI and VOCs (pre-Omicron variants) and model features from other published data, we were able to develop a ML model that successfully identified HCAbs with efficacy against Omicron variants, independent of our experimental biopanning workflow. This biopanning informed ML approach reduced the experimental screening burden by 78% to 90% for the Omicron BA.5 and Omicron BA.1 variants, respectively. The combined approach can be applied to other emerging viruses with pandemic potential to rapidly identify effective therapeutic antibodies against emerging variants.

Antibodies↗

DOE OSTI2020

Cataloging accreted stars within Gaia DR2 using deep learning

The goal of this study is to present the development of a machine learning based approach that utilizes phase space alone to separate the Gaia DR2 stars into two categories: those accreted onto the Milky Way from those that are in situ. Traditional selection methods that have been used to identify accreted stars typically rely on full 3D velocity, metallicity information, or both, which significantly reduces the number of classifiable stars. The approach advocated here is applicable to a much larger portion of Gaia DR2. Methods. A method known as “transfer learning” is shown to be effective through extensive testing on a set of mock Gaia catalogs that are based on the F IRE cosmological zoom-in hydrodynamic simulations of Milky Way-mass galaxies. The machine is first trained on simulated data using only 5D kinematics as inputs and is then further trained on a cross-matched Gaia /RAVE data set, which improves sensitivity to properties of the real Milky Way. Results. The result is a catalog that identifies ~767 000 accreted stars within Gaia DR2. This catalog can yield empirical insights into the merger history of the Milky Way and could be used to infer properties of the dark matter distribution.

Astronomy & Astrophysics↗

DOE OSTI2023

MACHINE LEARNING-ENABLED PREDICTION OF TRANSIENT INJECTION MAP IN AUTOMOTIVE INJECTORS WITH UNCERTAINTY QUANTIFICATION

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.

machine learning↗

DOE OSTI2020

Using Machine Learning Tools to Predict Compressor Stall

Clean energy has become an increasingly important consideration in today’s power systems. As the push for clean energy continues, many coal-fired power plants are being decommissioned in favor of renewable power sources such as wind and solar. However, the intermittent nature of renewables means that dynamic load following traditional power systems is crucial to grid stability. With high flexibility and fast response at a wide range of operating conditions, gas turbine systems are poised to become the main load following component in the power grid. Yet, rapid changes in load can lead to fluid flow instabilities in gas turbine power systems. These instabilities often lead to compressor surge and stall, which are some of the most critical problems facing the safe and efficient operation of compressors in turbomachinery today. Although the topic of compressor surge and stall has been extensively researched, no methods for early prediction have been proven effective. This study explores the utilization of machine learning tools to predict compressor stall. The long short-term memory (LSTM) model, a form of recurrent neural network (RNN), was trained using real compressor stall datasets from a 100 kW recuperated gas turbine power system designed for hybrid configuration. Two variations of the LSTM model, classification and regression, were tested to determine optimal performance. The regression scheme was determined to be the most accurate approach, and a tool for predicting compressor stall was developed using this configuration. Overall, results show that the tool is capable of predicting stalls 5–20 ms before they occur. With a high-speed controller capable of 5 ms time-steps, mitigating action could be taken to prevent compressor stall before it occurs.

42 ENGINEERING↗

DOE OSTI2023

Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. Further, to demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed. Further, to help facilitate the deployment of UQ in Scientific Machine Learning research and practice, we present and develop in [1] an open-source Python library (github.com/Crunch-UQ4MI/neuraluq), termed NeuralUQ, that is accompanied by an educational tutorial and additional computational experiments.

11 physics-informed neural networks↗