Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80

High-Resolution Meteorology with Climate Change Impacts from Global Climate Model Data Using Generative Machine Learning

As renewable energy generation increases, the impacts of weather and climate on energy generation and demand become critical to the reliability of the energy system. However, these impacts are often overlooked. Global climate models (GCMs) can be used to understand possible changes to our climate, but their coarse resolution makes them difficult to use in energy system modelling. Here we present open-source generative machine learning methods that produce meteorological data at a nominal spatial resolution of 4 km at an hourly frequency based on inputs from 100 km daily-average GCM data. These methods run 40 times faster than traditional downscaling methods and produce data that have high-resolution spatial and temporal attributes similar to historical datasets. We demonstrate that these methods can be used to downscale projected changes in wind, solar and temperature variables across multiple GCMs including projections for more frequent low-wind and high-temperature events in the Eastern United States.

climate change↗

Machine learning for surrogate process models of bioproduction pathways

Technoeconomic analysis and life-cycle assessment are critical to guiding and prioritizing bench-scale experiments and to evaluating economic and environmental performance of biofuel or biochemical production processes at scale. Traditionally, commercial process simulation tools have been used to develop detailed models for these purposes. However, developing and running such models can be costly and computationally intensive, which limits the degree to which they can be shared and reproduced in the broader research community. This study evaluates the potential of an automated machine learning approach to develop surrogate models based on conventional process simulation models. The analysis focuses on several high-value biofuels and bioproducts for which pathways of production from biomass feedstocks have been well-established. The results demonstrate that surrogate models can be an accurate and effective tool for approximating the cost, mass and energy balance outputs of more complex process simulations at a fraction of the computational expense.

09 BIOMASS FUELS↗

Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging: Theory, algorithms, and applications

Plug-and-play (PnP) priors constitute one of the most widely used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data fidelity term to promote data consistency and imposing a learned regularizer in the form of an image denoiser. Recent highly successful applications of PnP algorithms include biomicroscopy, computerized tomography (CT), magnetic resonance imaging (MRI), and joint ptychotomography. This article presents a unified and principled review of PnP by tracing its roots, describing its major variations, summarizing main results, and discussing applications in computational imaging. Additionally, we also point the way toward further developments by discussing recent results on equilibrium equations that formulate the problem associated with PnP algorithms.

97 MATHEMATICS AND COMPUTING↗

A Machine Learning-based Reliability Evaluation Model for Integrated Power-Gas Systems

This article proposes a hybrid machine learning method for the reliability evaluation of integrated power-gas systems (IPGS) under the uncertain component failure probability distributions. The Random Forest (RF) method is designed to select important features to solve the insufficient quantity of data and the curse of dimensionality problems. The Extreme Gradient Boosting (XGBoost) regression algorithm is developed to quantify the relationship between the uncertain parameters and reliability metrics. Moreover, a ten-fold cross-validation method is employed to further improve the accuracy of the regression model. Simulation results on three test systems show that the proposed method can achieve high accuracy for the reliability evaluation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Remote Sensing Technique to Upscale Methane Emission Flux in a Subtropical Peatland

Abstract Quantification of methane (CH 4 ) gas emission from peat is critical to understand CH 4 budget from natural wetlands under a climate warming scenario. Previous studies have focused on prediction and mapping of CH 4 emission flux using process‐based models, while application of statistical‐empirical models for upscaling spatially sparse in situ measurements is scarce. In this study, we developed an empirical remote sensing upscaling approach to estimate CH 4 emission flux in the Everglades using limited in situ point‐based CH 4 emission flux measurements and Landsat data during 2013–2018. We spatially and temporally linked in situ data with Landsat surface reflectance based on temporally composite data sets and developed an object‐based machine learning framework to model and map CH 4 emission flux. An ensemble analysis of two machine learning models, k ‐Nearest Neighbor ( k ‐NN) and Support Vector Machine (SVM), shows that the upscaling approach is promising for predicting CH 4 emission flux with a R 2 of 0.65 and 0.87 based on a fivefold cross‐validation for a dry season and wet season estimation, respectively. We generated emission flux map products that successfully revealed the spatial and temporal heterogeneity of CH 4 emission within the dominant freshwater marsh ecosystem in the Everglades. We conclude that Landsat is promising for upscaling and monitoring CH 4 emission flux and reducing the uncertainty in emission estimates from wetlands.

Zhang, Caiyun↗

Progressive transfer learning for advancing machine learning-based reduced-order modeling

Abstract To maximize knowledge transfer and improve the data requirement for data-driven machine learning (ML) modeling, a progressive transfer learning for reduced-order modeling (p-ROM) framework is proposed. A key concept of p-ROM is to selectively transfer knowledge from previously trained ML models and effectively develop a new ML model(s) for unseen tasks by optimizing information gates in hidden layers. The p-ROM framework is designed to work with any type of data-driven ROMs. For demonstration purposes, we evaluate the p-ROM with specific Barlow Twins ROMs (p-BT-ROMs) to highlight how progress learning can apply to multiple topological and physical problems with an emphasis on a small training set regime. The proposed p-BT-ROM framework has been tested using multiple examples, including transport, flow, and solid mechanics, to illustrate the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. In both similar and different topologies, p-BT-ROM achieves improved model accuracy with much less training data. For instance, p-BT-ROM with four-parent (i.e., pre-trained models) outperforms the no-parent counterpart trained on data nine times larger. The p-ROM framework is poised to significantly enhance the capabilities of ML-based ROM approaches for scientific and engineering applications by mitigating data scarcity through progressively transferring knowledge.

97 MATHEMATICS AND COMPUTING↗

Using machine learning to improve efficiency and accuracy of burnup measurements at PBR reactors [Slides]

The outline of the slides include: Motivations of the work; Modeling and simulation; Machine learning model; Results and comparison study with linear regression; and Conclusions. This work was done to help PBR designers and operators understand the burnup measurement better. We look forward to discussing the results in detail with industrial collaborators.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Developing predictive models for µ opioid receptor binding using machine learning and deep learning techniques

Opioids exert their analgesic effect by binding to the µ opioid receptor (MOR), which initiates a downstream signaling pathway, eventually inhibiting pain transmission in the spinal cord. However, current opioids are addictive, often leading to overdose contributing to the opioid crisis in the United States. Therefore, understanding the structure-activity relationship between MOR and its ligands is essential for predicting MOR binding of chemicals, which could assist in the development of non-addictive or less-addictive opioid analgesics. This study aimed to develop machine learning and deep learning models for predicting MOR binding activity of chemicals. Chemicals with MOR binding activity data were first curated from public databases and the literature. Molecular descriptors of the curated chemicals were calculated using software Mold2. The chemicals were then split into training and external validation datasets. Random forest, k-nearest neighbors, support vector machine, multi-layer perceptron, and long short-term memory models were developed and evaluated using 5-fold cross-validations and external validations, resulting in Matthews correlation coefficients of 0.528–0.654 and 0.408, respectively. Furthermore, prediction confidence and applicability domain analyses highlighted their importance to the models’ applicability. Our results suggest that the developed models could be useful for identifying MOR binders, potentially aiding in the development of non-addictive or less-addictive drugs targeting MOR.

Research & Experimental Medicine↗

Spectroscopic Probe Molecule Selection Using Quantum Theory, First-Principles Calculations, and Machine Learning

Probe molecule vibrational spectra have a long history of being used to characterize materials including metals, oxides, metal-organic frameworks, and even human proteins. Furthermore, recent advances in machine learning have enabled computationally generated spectra to aid in detailed characterization of complex surfaces with probe molecules. Despite widespread use of probe molecules, the science of probe molecule selection is underdeveloped. Here, we develop physical concepts, including orbital interaction energy and the energy overlap integral, to explain and predict the ability of probe molecules to discriminate structural descriptors. We resolve the crystal orbital overlap population (COOP) to specific molecular orbitals and quantify their bonding character, which directly influences vibrational frequencies. Using only a single adsorbate calculation from density function theory (DFT), we compute the interaction energy of individual adsorbate molecular orbitals with adsorption site atomic orbitals across many different sites. Combining the molecular orbital resolved COOP and changes in orbital interaction energy enables probe molecule selection for improved discrimination of various sites. We demonstrate these concepts by comparing the predicted effectiveness of carbon monoxide (CO), nitric oxide (NO), and ethylene (C 2 H 4 ) to probe Pt adsorption sites. Finally, using a previously developed machine learning framework, we show that models trained on hundreds of thousand C 2 H 4 spectra, computed from DFT, which regress surface binding-type and generalized coordination number (GCN), outperform those trained using CO and NO spectra. Lastly, a python package, pDOS_overlap, for implementing the electron density based analysis on any combination of adsorbates and materials, is also made available.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Machine learning surrogates for surface complexation model of uranium sorption to oxides

Abstract The safety assessments of the geological storage of spent nuclear fuel require understanding the underground radionuclide mobility in case of a leakage from multi-barrier canisters. Uranium, the most common radionuclide in non-reprocessed spent nuclear fuels, is immobile in reduced form (U(IV) and highly mobile in an oxidized state (U(VI)). The latter form is considered one of the most dangerous environmental threats in the safety assessments of spent nuclear fuel repositories. The sorption of uranium to mineral surfaces surrounding the repository limits their mobility. We quantify uranium sorption using surface complexation models (SCMs). Unfortunately, numerical SCM solvers often encounter convergence problems due to the complex nature of convoluted equations and correlations between model parameters. This study explored two machine learning surrogates for the 2-pK Triple Layer Model of uranium retention by oxide surfaces if released as U(IV) in the oxidizing conditions: random forest regressor and deep neural networks. Our surrogate models, particularly DNN, accurately reproduce SCM model predictions at a fraction of the computational cost without any convergence issues. The safety assessment of spent fuel repositories, specifically the migration of leaked radioactive waste, will benefit from having ultrafast AI/ML surrogates for the computationally expensive sorption models that can be easily incorporated into larger-scale contaminant migration models. One such model is presented here.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Machine Learning-Based Regression Models for Ironmaking Blast Furnace Automation

Computational fluid dynamics (CFD)-based simulation has been the traditional way to model complex industrial systems and processes. One very large and complex industrial system that has benefited from CFD-based simulations is the steel blast furnace system. The problem with the CFD-based simulation approach is that it tends to be very slow for generating data. The CFD-only approach may not be fast enough for use in real-time decisionmaking. To address this issue, in this work, the authors propose the use of machine learning techniques to train and test models based on data generated via CFD simulation. Regression models based on neural networks are compared with tree-boosting models. In particular, several areas (tuyere, raceway, and shaft) of the blast furnace are modeled using these approaches. The results of the model training and testing are presented and discussed. The obtained R 2 metrics are, in general, very high. The results appear promising and may help to improve the efficiency of operator and process engineer decisionmaking when running a blast furnace.

97 MATHEMATICS AND COMPUTING↗

Machine-Learning Assisted Identification of Accurate Battery Lifetime Models with Uncertainty

Reduced-order battery lifetime models, which consist of algebraic expressions for various aging modes, are widely utilized for extrapolating degradation trends from accelerated aging tests to real-world aging scenarios. Identifying models with high accuracy and low uncertainty is crucial for ensuring that model extrapolations are believable, however, it is difficult to compose expressions that accurately predict multivariate data trends; a review of cycling degradation models from literature reveals a wide variety of functional relationships. Here, a machine-learning assisted model identification method is utilized to fit degradation in a stand-out LFP-Gr aging data set, with uncertainty quantified by bootstrap resampling. The model identified in this work results in approximately half the mean absolute error of a human expert model. Models are validated by converting to a state-equation form and comparing predictions against cells aging under varying loads. Parameter uncertainty is carried forward into an energy storage system simulation to estimate the impact of aging model uncertainty on system lifetime. The new model identification method used here reduces life-prediction uncertainty by more than a factor of three (86% ± 5% relative capacity at 10 years for human-expert model, 88.5% ± 1.5% for machine-learning assisted model), empowering more confident estimates of energy storage system lifetime.

25 ENERGY STORAGE↗

Building an ab initio solvated DNA model using Euclidean neural networks

Accurately modeling large biomolecules such as DNA from first principles is fundamentally challenging due to the steep computational scaling of ab initio quantum chemistry methods. This limitation becomes even more prominent when modeling biomolecules in solution due to the need to include large numbers of solvent molecules. We present a machine-learned electron density model based on a Euclidean neural network framework that includes a built-in understanding of equivariance to model explicitly solvated double-stranded DNA. By training the machine learning model using molecular fragments that sample the key DNA and solvent interactions, we show that the model predicts electron densities of arbitrary systems of solvated DNA accurately, resolves polarization effects that are neglected by classical force fields, and captures the physics of the DNA-solvent interaction at the ab initio level.

59 BASIC BIOLOGICAL SCIENCES↗

Physics-Reinforced Machine Learning Algorithms for Multiscale Closure Model Discovery

The central objective of this project was to address the challenge of modeling and simulating complex multiscale turbulence phenomena by leveraging physics-guided machine learning (PGML) and hybrid modeling approaches. By integrating physics-based methods with data-driven models, the research focused on achieving robust and scalable solutions for geophysical turbulence, enhancing numerical weather prediction and climate research tools. The project resulted in significant advancements in computational modeling paradigms, predictive tools for reduced-order modeling, and innovative algorithms for fluid dynamics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Machine learning prediction of incidence of Alzheimer’s disease using large-scale administrative health data

Nationwide population-based cohort provides a new opportunity to build an automated risk prediction model based on individuals’ history of health and healthcare beyond existing risk prediction models. We tested the possibility of machine learning models to predict future incidence of Alzheimer’s disease (AD) using large-scale administrative health data. From the Korean National Health Insurance Service database between 2002 and 2010, we obtained de-identified health data in elders above 65 years (N = 40,736) containing 4,894 unique clinical features including ICD-10 codes, medication codes, laboratory values, history of personal and family illness and socio-demographics. To define incident AD we considered two operational definitions: “definite AD” with diagnostic codes and dementia medication (n = 614) and “probable AD” with only diagnosis (n = 2026). We trained and validated random forest, support vector machine and logistic regression to predict incident AD in 1, 2, 3, and 4 subsequent years. For predicting future incidence of AD in balanced samples (bootstrapping), the machine learning models showed reasonable performance in 1-year prediction with AUC of 0.775 and 0.759, based on “definite AD” and “probable AD” outcomes, respectively; in 2-year, 0.730 and 0.693; in 3-year, 0.677 and 0.644; in 4-year, 0.725 and 0.683. The results were similar when the entire (unbalanced) samples were used. Important clinical features selected in logistic regression included hemoglobin level, age and urine protein level. This study may shed a light on the utility of the data-driven machine learning model based on large-scale administrative health data in AD risk prediction, which may enable better selection of individuals at risk for AD in clinical trials or early detection in clinical settings.

97 MATHEMATICS AND COMPUTING↗

Findings on Subtask 3.1 - Bakken Rich Gas Enhanced Oil Recovery Project

Total in-place oil for the Bakken petroleum system (BPS) (which includes the Bakken and Three Forks Formations) has been estimated to be 600 billion barrels (bbl). However, BPS wells have decline rates as high as 85% over the first 3 years of their lives, and primary recovery factors typically range from 3% to 10% of original oil in place. Given the low initial recovery rates, even small incremental productivity improvements could dramatically increase technically recoverable oil in the BPS. One potential solution is enhanced oil recovery (EOR) using gas injection, such as carbon dioxide (CO2) or hydrocarbon (HC) gases. While commonly used in conventional reservoirs, CO2 EOR in unconventional tight oil reservoirs has been limited to pilot tests. EOR using rich gas (mixture of methane, ethane, and propane) has also been employed in numerous pilots in several unconventional plays and has recently been successfully applied in the Eagle Ford play. If successful, large-scale gas-based EOR in the BPS could dramatically increase oil productivity and recovery factors and extend the life of the play for decades. While CO2 may be a technically suitable working fluid for EOR in the BPS, supplies are limited and costs for using CO2 in EOR pilots are prohibitively high. Meanwhile, produced gas flaring has presented challenges for BPS operators in North Dakota. Analysis conducted by the North Dakota Pipeline Authority indicates that the current gas-gathering infrastructure in North Dakota is insufficient to accommodate all of the associated gas that is produced from the BPS. The geographically isolated location of North Dakota relative to large natural gas markets, combined with suppressed natural gas prices, has made it economically challenging for industry to invest capital in expanding gas-gathering infrastructure in the state. These circumstances led to a research program conducted by the Energy & Environmental Research Center (EERC) in partnership with Liberty Resources Management Company LLC (LR) to examine the potential to use rich gas injection for EOR and mitigate flaring. A rich gas EOR pilot test was designed and executed by LR at its Stomping Horse development area in Williams County, North Dakota. From July 2018 through May 2019, a total of 160 million standard cubic feet (MMscf) of rich produced gas was injected into the BPS using five different wells in a sequential injection strategy. LR’s Leon–Gohrick drill spacing unit (DSU) was used as the test site. Regulatory oversight was provided by the North Dakota Industrial Commission (NDIC). Technical support was provided by the EERC through a series of laboratory, modeling, and field-based activities, and additional post-pilot research activities incorporated learnings from the test, developed new laboratory data, improved fracture modeling methods, and developed machine learning and big data analytics. The results from the Stomping Horse rich gas EOR pilot activities indicate that developing an effective, economical EOR approach for the BPS will require more field tests. Another key lesson learned from the Stomping Horse tests is that detailed pre- and posttest data on reservoir conditions and fluids production are essential. Robust reservoir characterization provides information that is crucial to creating realistic geomodels and conducting valid dynamic simulations of potential EOR scenarios. A detailed understanding of the completions and production history of offset wells is also necessary for valid test result interpretations. This knowledge is essential to designing the operational parameters of injectivity tests and interpreting the results. A conformance control strategy is also essential to success. Laboratory-based examinations of rich gas interactions with reservoir fluids and rocks were conducted, with an emphasis on determining the ability to mobilize oil in the tight reservoir rocks and shales of the BPS. Injection fluid composition was shown to have a positive impact on reducing reservoir oil minimum miscibility pressure (MMP), reducing interfacial tension (IFT), and altering wettability. IFT and contact angle measurements demonstrated that wettability can be altered in the presence of rich gas, suggesting the potential to improve oil recovery. Iterative modeling of surface infrastructure and reservoir performance using data generated by the various project activities was conducted. A geologic model of the Stomping Horse area was built; history-matched oil, gas, and water production was used in simulations of various EOR scenarios. Early programmatic modeling results were used to support LR’s design and operation of the EOR pilot and to provide insight regarding optimization of future commercial-scale BPS EOR design and operations. Post-pilot modeling focused on alternative methods of understanding complex fracture networks and accelerating simulation time. These led to improved simulation run times and provide excellent history-matching results. Several of these iterative models were used as the bases for developing algorithms into machine learning and big data analytics. History matching in reservoir simulation is time-consuming and computer processing-intensive. Machine learning algorithms were created, and an automated history-matching tool was developed. A large set of synthetic reservoir simulations were created to generate well responses (oil, gas, and water production, well bottomhole pressure [BHP], and tracer or propane breakthrough) for a set of EOR operating parameters that included offset well status (open or closed), injectate (rich gas or propane), injection rate, and injection well BHP. A user interface was developed to provide real-time visualization. Machine learning-based models were developed to provide rapid forecasting of well performance given a set of user-defined EOR operating parameters. These predictive models allow the user to modify the offset well status, injection rate, and injection well BHP and rapidly forecast future production performance. The combination of real-time visualization tools with real-time forecasting tools provides a framework for real-time control—operational changes that the EOR site operator can enact (e.g., changing gas injection rates) to affect the observed performance and potentially improve the EOR outcome. There is great reason to be optimistic about the future of EOR in the Bakken. The results of the laboratory studies suggest significant potential for high rates of oil mobilization using produced field gas injection under the right conditions. The results of the lab studies, combined with rigorous statistical analysis of well production data and associated modeling efforts, confirm the notion that fluid mobility within the reservoir is controlled by fractures. As more knowledge is gained about the nature and distribution of fracture networks in the Bakken, the industry will be in a better position to predict and, ultimately, influence fluid mobility. New field tests are necessary to develop a more complete understanding of those conditions. Thoughtful and creatively engineered field tests within a well-characterized geologic setting will yield the fundamental knowledge needed to take Bakken oil production to the next level. This subtask was cofunded through the EERC–U.S. Department of Energy Joint Program on Research and Development for Fossil Energy-Related Resources Cooperative Agreement No. DE-FE0024233. Nonfederal funding was provided by the North Dakota Industrial Commission’s Oil and Gas Research Program and Computer Modelling Group.

04 OIL SHALES AND TAR SANDS↗

FutureTense

Protective vaccines and reliable diagnostics are essential tools for controlling viral diseases. However, the efficacy of these tools can be diminished by mutations in viral genomes. The delay between the emergence of new viral strains and the redesign of vaccines and diagnostics allows for continued viral transmission. Is it possible to address this challenge by computationally predicting viral genome sequence evolution? Can we “future-proof” vaccines and diagnostics by targeting both current and anticipated future sequence variants? While predicting viral evolution is still an unsolved, “grand challenge” problem in biology, the large, and rapidly growing, number of SARS-CoV-2 genome sequences provide an opportunity to quantify the ability of machine learning to predict viral genome sequence evolution. Towards this end, we have developed a simple computational model for predicting viral evolution at the level of individual nucleotides. The key metric for quantifying the per-base, prediction accuracy for viral evolution is the Mann-Whitney U statistic (or, equivalently, the area under the receiver operator curve). Since the Mann-Whitney U statistic is not a differentiable function, existing deep leaning packages (like Pytorch and Keras/TensorFlow) are not useful, as they require that the accuracy metric/objective function be analytically differentiable with respect to the model parameters. To overcome this challenge, we have implemented custom software, “FutureTense”, that can train a machine learning model by maximizing the non-differentiable Mann-Whitney U statistic. This software trains a machine learning model by exploring along the direction of the discrete gradient of the Mann-Whitney U statistic in the model parameter space. Parallel computing and genome sequence-specific optimizations are used to accelerate model training. The resulting machine learning model learns the observed high C->U mutation rates in the SARS-CoV-2 genome (which are potentially induced by host defenses) and provides prediction accuracies that are significantly better than one would expect from random chance. While predicting viral evolution is still quite far from a solved problem, the surprising performance of this simple model gives hope that the accuracy of predicting viral genome evolution can be further increased by more sophisticated approaches.

Gans, Jason↗

Spatial Transferability of Machine Learning Based Volume Estimation Models

High-quality traffic volume data is essential for efficient transportation planning and operations. However, such high-quality data is expensive to collect, owing primarily to the high capital cost of installing and maintaining continuous counting stations (CCSs). Recent availability of probe-based vehicle data offers a cost-effective solution for increasing the observability of traffic volumes. However, having ample ground truth traffic data is a prerequisite for developing robust volume estimation models. Though this might not be a big issue in many states, states with scarce CCS data might be able to benefit from robust volume estimation models developed in (adjacent) data-rich states. While there is a reasonable amount of spatial transferability research in the transportation domain, there is a dearth of knowledge on the spatial transferability of probe-based volume estimation models. To address this gap, this paper explores spatial transferability of volume estimation models developed from data in three states (Colorado, North Carolina, and Pennsylvania). Results indicate that it is extremely important to maintain temporal consistency when attempting spatial transferability of volume estimation models. It was also found that models trained on regions with lower peak traffic volumes will limit the performance of models transferred to states with higher peak hourly traffic volumes. Corroborating findings from existing spatial transferability research on other topics, it was found that a meta-model (developed using data from multiple states) performs better than volume estimation models developed within any one of the states.

ADVANCED PROPULSION SYSTEMS↗