Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “regression models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

MULTIPHYSICS-MODELING OF FIRE-INDUCED URANIUM AEROSOL FORMATION – A-POSTERIORI BENCHMARKING OF EXPERIMENTS

The formulation and solution of a model that properly described the temperature profiles of uranium in fire conditions was prepared, verified and validated, to support a-posteriori benchmarking of historical experiments. Data from the multi-physics model combined with data regression from the experiments provides a useful tool for exploring sources of potential bias in historical experiments. The approaches used are augmented by visual observations and photographic evidence allowing for correlations between various phenomena, independent of whether or not that specific phenomena are part of the validated model. In this manner, the validated model is a tool for a-posteriori benchmarking, and not a model designed to replicate the experimental output exactly for any specific experiment. For experiments like the Clark (2015) experiments, this integrated model (including chemistry) demonstrated that the largest bias was associated in the metallurgy of the coupons. The model developed does serve as a basis for defending which of the historical alloys tested are the most appropriate for any other alloy being considered (e.g., one that has not yet been tested). The model identifies key reactions that impact uranium chemistry and can significantly bias experimental results. Through use of the model and energy balances, data from historical experiments by Elder and Tinkle (1980) were regressed and benchmarked. This study identifies that all uranium alloys are not equal and that there are distinct differences between the dominant alloys representing distinct metallurgical phases.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Machine learning models for estimating contamination across different curbside collection strategies

Contaminated recyclables, which are frequently discarded as waste, pose a significant challenge to the implementation of a circular economy. These contaminated recyclables impede the circulation of resources, resulting in higher processing costs at material recovery facilities (MRFs). Over the past few decades, machine learning (ML) models such as linear regression (LR), support vector machine (SVM), and random forest (RF) have evolved to provide new methods for predicting inbound contamination rates in addition to traditional statistical models. In this study, we applied ML models to predict inbound contamination rates using demographic features from 15 counties in the U.S. with different curbside collection strategies. In general, we found that ML models outperformed linear mixed models. Specifically, SVM models had the highest performance (R 2 = 0.75; mean absolute error (MAE) = 0.06), which may be due to their ability to model nonlinear relationships between features and inbound contamination rates. Further, the key predictor was population, with poverty rate being positively correlated and median age negatively correlated with inbound contamination rates. To improve the management of contamination and enhance the implementation of a circular economy, better models are needed to understand and estimate inbound contamination rates as well as identify critical factors in the present and future.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Using electronic health record metadata to predict housing instability amongst veterans

Housing instability is considered a significant life stressor and preemptive screening should be applied to identify those at risk for homelessness as early as possible so that they can be targeted for specialized care. We developed models to classify patient outcomes for an established VA Homelessness Screening Clinical Reminder (HSCR), which identifies housing instability, in the two months prior to its administration. Logistic Regression and Random Forest models were fit to classify responses using the last 18 months of document activity. We measure concentration of risk across stratifications of predicted probability and observe an enriched likelihood of finding confirmed false negative responses from veterans with diagnosed housing instability. Positive responses were 34 times more likely to be detected within the top 1 % of patients predicted at risk than from those randomly selected. There is a 1 in 4 chance of detecting false negatives within the top 1 % of predicted risk. Machine learning methods can classify between episodes of housing instability using a data-driven approach that does not rely on variables curated from domain experts. This method has the potential to improve clinicians’ ability to identify veterans who are experiencing housing instability but are not captured by HSCR.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Uncertainty quantification for neural network potential foundation models

Abstract For neural network potentials (NNPs) to gain widespread use, researchers must be able to trust model outputs. However, the blackbox nature of neural networks and their inherent stochasticity are often deterrents, especially for foundation models trained over broad swaths of chemical space. Uncertainty information provided at the time of prediction can help reduce aversion to NNPs. In this work, we detail two uncertainty quantification (UQ) methods. Readout ensembling, by finetuning the readout layers of an ensemble of foundation models, provides information about model uncertainty, while quantile regression, by replacing point predictions with distributional predictions, provides information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to the foundation model and a series of finetuned models. The uncertainties produced by the readout ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Uncertainty Aware Deep Learning for Particle Accelerators

Standard deep learning models for classification and regression applications are ideal for capturing complex system dynamics. However, their predictions can be arbitrarily inaccurate when the input samples are not similar to the training data. Implementation of distance aware uncertainty estimation can be used to detect these scenarios and provide a level of confidence associated with their predictions. In this paper, we present results from using Deep Gaussian Process Approximation (DGPA) methods for errant beam prediction at Spallation Neutron Source (SNS) accelerator (classification) and we provide an uncertainty aware surrogate model for the Fermi National Accelerator Lab (FNAL) Booster Accelerator Complex (regression).

Rajput, Kishansingh↗
DOE OSTI2023

Association Between Injection and Microseismicity in Geothermal Fields With Multiple Wells: Data-Driven Modeling of Rotokawa, New Zealand, and Húsmúli, Iceland

Understanding injection-induced microseismicity in geothermal systems can provide insight into reservoir connectedness. However, fault and reservoir complexity are difficult to represent in simple analytical models, which makes it difficult to discern clear relationships from incidental associations. Here, we have used data-driven models to study how fluid injection and microseismicity are related in the Rotokawa (New Zealand) and Hellisheiði (Iceland) geothermal fields. We tested two classes of model: (a) lagged linear regression of seismicity rate as a function of well injection rates; and (b) systematic extraction of injection time series features that are then evaluated for associations with the seismicity. These models allowed us to determine which wells had the greatest correlation with microseismicity and to explain this association in a reservoir context. Finally, exploring different data types and transformations, we were unable to establish a link between rapid changes in injection rate and seismicity spikes, as suggested by some theoretical models.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Machine-Learning of Nonlocal Kernels for Anomalous Subsurface Transport from Breakthrough Curves

Anomalous behavior is ubiquitous in subsurface solute transport due to the presence of high degrees of heterogeneity at different scales in the media. Although fractional models have been extensively used to describe the anomalous transport in various subsurface applications, their application is hindered by computational challenges. Simpler nonlocal models characterized by integrable kernels and finite interaction length represent a computationally feasible alternative to fractional models; yet, the informed choice of their kernel functions still remains an open problem. We propose a general data-driven framework for the discovery of optimal kernels on the basis of very small and sparse data sets in the context of anomalous subsurface transport. Using spatially sparse breakthrough curves recovered from fine-scale particle-density simulations, we learn the best coarse-scale nonlocal model using a nonlocal operator regression technique. Predictions of the breakthrough curves obtained using the optimal nonlocal model show good agreement with fine-scale simulation results even at locations and time intervals different from the ones used to train the kernel, confirming the excellent generalization properties of the proposed algorithm. A comparison with trained classical models and with black-box deep neural networks confirms the superiority of the predictive capability of the proposed model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Metamodeling Time-Temperature Creep Parameters

Abstract There exist many time-temperature parameter (TTP) models for creep rupture prediction of components including the Larson–Miller (LM), Manson–Haferd (MH), Manson–Brown (MB), Orr–Sherby–Dorn (OSD), Manson–Succop (MS), Graham–Walles (GW), Chitty–Duval (CD), Goldhoff–Sherby (GS) models. It remains a challenge to determine which model is “best”, capable of accurate interpolation and physically realistic extrapolation of creep rupture data for a given material. In this study, metamodeling is applied to create a unified TTP metamodel that combines and regresses into twelve TTP models (eight existing and four newly derived). An analysis of the mathematical problems that exist in TTP models is provided. A matlab code is written that can: (1) calibrate the material constants of any of the twelve TTP models (using the metamodel); (2) determine the most suitable stress-parameter function; (3) and report the normalized mean square error (NMSE) of rupture predictions for a given material database. Using the metamodel, and code, a design engineer can make an intelligent selection of the “best” TTP model for creep resistant design. This process is demonstrated using four isotherms of alloy P91 creep rupture data. To assess the influence of material, further validation is performed on alloys Hastelloy X, 304SS, and 316SS. It is determined that the “best” model is dependent on material type and the quality and quantity of available data.

Engineering↗
DOE OSTI2024

A Data-Driven Methodology for Contextual Unit Commitment Using Regression Residuals

Day after day, system operators are faced with the challenge of taking unit commitment (UC) decisions under uncertain net load conditions. The standard operating procedure for taking UC decisions begins by leveraging auxiliary data on covariates (such as the day of the week or latest weather information) to generate a point prediction for net load, which is used in solving a deterministic UC problem. Such an approach, however, is known to deliver a notoriously poor out-of-sample (OOS) performance, as it completely disregards the stochastic nature of net load. While stochastic programming models explicitly represent uncertainty, they mostly do so using a generic set of scenarios that neglect covariate observations, squandering useful auxiliary data that could be harnessed to glean insights into uncertainty. In this article, we discuss a contextual stochastic optimization approach to UC, which effectively exploits covariate observations while explicitly assessing uncertainty so as to boost the OOS performance of UC decisions. The key thrust of our approach is to leverage regression models, along with their empirical residuals, to set up and solve sample average approximation problems. Not only do we prove that our approach satisfies the requisite conditions for asymptotic optimality and consistency laid out in (Kannan et al., 2022), but we also assess its performance on several case studies conducted using real-world data collected in California ISO and New York ISO grids. In conclusion, results show that the proposed approach can significantly improve OOS performance compared to alternative methods proposed in the literature under varying dataset sizes.

Yurdakul, Ogun↗
DOE OSTI2025

Data-Driven Kinetic Reaction Networks for Separation Chemistry

Understanding complex, multistep chemical reactions at the molecular level is a major challenge whose solution would greatly benefit the design and optimization of numerous chemical processes. The separation of rare-earth (4f) and actinide (5f) elements is an example where improving our chemical understanding is important for designing and optimizing new chemistries, even with a limited number of observations. Here, in this work, we leverage data-driven artificial intelligence and machine-learning approaches to develop kinetic reaction networks that describe the liquid–liquid extraction mechanism of uranium using N,N-di-2-ethylhexyl-isobutyramide (DEHiBA). Specifically, we compare and contrast the properties of two classes of models: (1) purely data-driven models that are regularized using chemistry-agnostic, L1 regression and (2) chemistry-informed models that are regularized using relative reaction energies provided by quantum mechanical calculations. We observe that purely data-driven models are unbiased, simple, and accurate in their predictions of experimental measurements when provided with sufficient data but are difficult to fully constrain and interpret. In contrast, chemistry-informed models exhibit significantly improved chemical interpretability and consistency, providing a detailed description of the separation process while achieving high accuracy through ensemble averaging. Overall, the dominant species predicted to be extracted into the organic phase is UO 2 (NO 3 ) 2 (DEHiBA) 2 , agreeing with experimental slope analysis, thermodynamic modeling, EXAFS, and crystal structures. This work demonstrates that leveraging the fundamental structure of the problem can lead to efficient learning schemes that provide both accurate predictions and chemical insights at a low computational cost.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Data-Driven Learning of Nonautonomous Systems

In this work, we present a numerical framework for recovering unknown nonautonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the nonautonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-Driven Learning of Non-Autonomous Systems

We present a numerical framework for recovering unknown non-autonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the non-autonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Extratropical Cloud Feedback Constrained by Cloud Sources and Sinks in Cyclones

Constraining cloud feedback in global climate models (GCMs) using observations is important for establishing accurate predictions of future climate. Uncertainty in shortwave cloud feedback (SW FB ) dominates uncertainty in total cloud feedback. Recent studies show a shift toward more positive extratropical SW FB in the latest generations of GCMs leading to the emergence of very high equilibrium climate sensitivity (ECS). In this study, we use precipitation efficiency and albedo susceptibility to constrain liquid water path (LWP) response to warming and SW FB in the Southern Ocean (SO; 50°–80°S). We analyze precipitation in extratropical cyclones (ECs) to learn about extratropical condensed water sink processes, combined with observations of clouds and moisture convergence, and use the analysis to better understand and constrain SW FB . We utilize a perturbed parameter ensemble (PPE) hosted in the Community Atmosphere Model, version 6 (CAM6), to provide a constraint on SW FB based on observations from Clouds and the Earth’s Radiant Energy System (CERES) and Multisensor Advanced Climatology of LWP (MAC-LWP). We apply Gaussian process regression to emulate the model response to all parameters perturbed in the PPE. Confronting the emulator output with observations provides a new estimated response of Earth to global warming. Furthermore, our new estimates of SO LWP reduce the PPE range by 66%–72%, which results in a shortwave cloud radiative effect estimated range that is 27%–34% less than the PPE range. Observations suggest a more positive SO SW FB than the Community Earth System Model, version 2 (CESM2), and consequently do not reject the high climate sensitivity GCMs emerging from the Coupled Model Intercomparison Project phase 6 (CMIP6).

Atmosphere↗
DOE OSTI2020

Influences of North Pacific Ocean Domain Extent on the Western U.S. Winter Hydroclimatology in Variable-Resolution CESM

Variable-resolution global climate models (VRGCMs) are a dynamical downscaling method that can reach spatiotemporal scales needed for regional climate assessments. Over the years, several users of VRGCMs have assumed where the location and extent of the refinement domain should be based on knowledge of the prevailing storm tracks and resolution dependence of important regional climate processes (e.g., atmospheric rivers [ARs] and orographic uplift), but the effect of high-resolution domain size and extent on the simulation of downstream hydroclimatic phenomena has not been systematically evaluated. Here, we use variable resolution in the Community Earth System Model (VR-CESM) to perform such a test. To do this, three VR-CESM grids were generated that span the entire, two thirds, and one third of the North Pacific and evaluated for a 30-year climatology using Atmospheric Model Intercomparison Project protocols. Simulations are compared with reanalysis products offshore (fifth-generation of the European Centre for Medium-Range Weather Forecasts atmospheric reanalysis [ERA5]) and onshore (Livneh, 2015, https://doi.org/10.1038/sdata.2015.42, and Parameter-elevation Regressions on Independent Slopes Model [PRISM]) of the western United States. The westward expansion of refinement domain influenced integrated vapor transport (IVT), which was generally high biased but minimally impacted AR characteristics. Due to slight differences in landfalling AR counts in the western United States, California winter precipitation generally improved with westward expansion of the refinement domain. Western U.S. mountain snowpack and surface temperatures were insensitive to refinement domain size and were more influenced by changes in topographic resolution and/or land surface model version. Here, given minimal dependence of simulated western U.S. hydroclimate on refinement domain size over the North Pacific, we advise future VR-CESM studies to focus grid resolution on better resolving land surface heterogeneity.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Real-time capable modeling of ICRF heating on NSTX and WEST via machine learning approaches

Abstract A real-time capable core Ion Cyclotron Range of Frequencies (ICRF) heating model on NSTX and WEST is developed. The model is based on two nonlinear regression algorithms, the random forest ensemble of decision trees and the multilayer perceptron neural network. The algorithms are trained on TORIC ICRF spectrum solver simulations of the expected flat-top operation scenarios in NSTX and WEST assuming Maxwellian plasmas. The surrogate models are shown to successfully capture the multi-species core ICRF power absorption predicted by the original model for the high harmonic fast wave and the ion cyclotron minority heating schemes while reducing the computational time by six orders of magnitude. Although these models can be expanded, the achieved regression scoring, computational efficiency and increased model robustness suggest these strategies can be implemented into integrated modeling frameworks for real-time control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Assessment of Machine Learning for Ultrasonic Nondestructive Evaluation of Alkali–Silica Reaction in Concrete

Alkali–silica reaction (ASR) is a type of material degradation in concrete structures that leads to concrete cracking and rebar corrosion, thereby reducing the material’s structural integrity and the overall structure’s lifetime and raising safety concerns. Ultrasonic nondestructive evaluation (NDE) has been proven to be a valuable technique for assessing concrete properties and monitoring ASR progression in concrete. However, the deployment and analysis of ultrasonic NDE and its data requires specialized expertise, often relying on the engineer’s subjective interpretation. With the surge in computational power, artificial intelligence (AI) and machine learning (ML) algorithms have become popular in automating NDE data analysis. Various industrial sectors are increasingly adopting ML algorithms for NDE data analysis with a growing emphasis on AI–assisted automation. Regulatory agencies are also preparing for this technological shift, anticipating corresponding revisions in standards. Thus, there is an urgent need to identify the capabilities and limitations of current ML technologies for the evaluation of concrete material properties and damage status. Furthermore, the effects of various factors on ML model performance must be thoroughly investigated. The study summarized herein evaluated the effectiveness of two ML models (i.e., support vector regression (SVR) and deep neural network (DNN)) in predicting concrete material damage induced by ASR based on the long-term ultrasonic monitoring data. Four distinct concrete specimens were cast with artificially induced ASR, and over a period exceeding 500 days, ultrasonic signals and expansion data were continuously collected. For the SVR model, wave velocity and 12 other wave features were extracted from the ultrasonic signals, with 6 out of 13 features selected as input for the model. Different combinations of training and testing datasets were designed to explore factors influencing prediction performance, including the range of data within training and testing sets, in addition to various signal preprocessing methodologies. These findings suggest the importance of using a training dataset with a broader data range compared with testing datasets for improved model performance alongside consistent signal preprocessing across datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Harnessing distributed GPU computing for generalizable graph convolutional networks in power grid reliability assessments

Although machine learning (ML) has emerged as a powerful tool for rapidly assessing grid contingencies, prior studies have largely considered a static grid topology in their analyses. This limits their application, since they need to be re-trained for every new topology. Here, this paper explores the development of generalizable graph convolutional network (GCN) models by pre-training them across a range of grid topologies and contingency types. We found that a GCN model with auto-regressive moving average (ARMA) layers with a line graph representation of the grid offered the best predictive performance in predicting voltage magnitudes (VM) and voltage angles (VA). We introduced the concept of phantom nodes to consider disparate grid topologies with a varying number of nodes and lines. For pre-training the GCN ARMA model across a variety of topologies, distributed graphics processing unit (GPU) computing afforded us significant training scalability. The predictive performance of this model on grid topologies that were part of the training data is substantially better than the direct current (DC) approximation. Although direct application of the pre-trained model to topologies that are not part of the grid is not particularly satisfactory, fine-tuning with small amounts of data from a specific topology of interest significantly improves predictive performance. In general, this paper highlights the feasibility of training large-scale GNN models to assess the reliability of power grids by considering a wide variety of grid topologies and contingency types. With the advent of foundational models in ML and the exponential increase in GPU computing clusters, generalizable ML models will significantly enhance how utilities manage power systems and make decisions in real-time or near-real-time.

24 - POWER TRANSMISSION AND DISTRIBUTION↗