Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “regression models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Subsampling of Parametric Models with Bifidelity Boosting

Least squares regression is a ubiquitous tool for building emulators (a.k.a. surrogate models) of problems across science and engineering for purposes such as design space exploration and uncertainty quantification. When the regression data are generated using an experimental design process (e.g., a quadrature grid) involving computationally expensive models, or when the data size is large, sketching techniques have shown promise at reducing the cost of the construction of the regression model while ensuring accuracy comparable to that of the full data. However, random sketching strategies, such as those based on leverage scores, lead to regression errors that are random and may exhibit large variability. To mitigate this issue, we present a novel boosting approach that leverages cheaper, lower-fidelity data of the problem at hand to identify the best sketch among a set of candidate sketches. This in turn specifies the sketch of the intended high-fidelity model and the associated data. We provide theoretical analyses of this bifidelity boosting (BFB) approach and discuss the conditions the low- and high-fidelity data must satisfy for a successful boosting. In doing so, we derive a bound on the residual norm of the BFB sketched solution relating it to its ideal, but computationally expensive, high-fidelity boosted counterpart. Finally, empirical results on both manufactured and PDE data corroborate the theoretical analyses and illustrate the efficacy of the BFB solution in reducing the regression error, as compared to the nonboosted solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Band gap predictions of double perovskite oxides using machine learning

Abstract The compositional and structural variety inherent to oxide perovskites spawn wide-ranging applications. In perovskites, the band gap E g , a key material parameter for these applications, can be optimally controlled by varying the composition. Here, we implement a hierarchical screening process in which two cross-validated and predictive machine learning models for band gap classification and regression, trained using exhaustive datasets that span 68 elements of the periodic table, are applied sequentially. The classification model separates wide band gap materials, with E g ≥ 0.5 eV, from materials which have zero or relatively small band gaps, namely E g < 0.5 eV, and the second regression model quantitatively predicts the gap value of the wide band gap compounds. The study down-selects 13,589 cubic oxide perovskite compositions that are predicted to be experimentally formable, thermodynamically stable, and have a wide band gap. Of these, a subset of 310 compounds, which are predicted to be stable and formable with a confidence greater than 90%, are identified for further investigation. Our models are methodically analyzed via performance metrics and inter-dependence of model features to gain physical insight into the band gap prediction problem. Design maps to identify the variation of band gap with substitution of different elements are also presented.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Modelling stellar activity with Gaussian process regression networks

ABSTRACT Stellar photospheric activity is known to limit the detection and characterization of extrasolar planets. In particular, the study of Earth-like planets around Sun-like stars requires data analysis methods that can accurately model the stellar activity phenomena affecting radial velocity (RV) measurements. Gaussian Process Regression Networks (GPRNs) offer a principled approach to the analysis of simultaneous time series, combining the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian Processes. Using HARPS-N solar spectroscopic observations encompassing three years, we demonstrate that this framework is capable of jointly modelling RV data and traditional stellar activity indicators. Although we consider only the simplest GPRN configuration, we are able to describe the behaviour of solar RV data at least as accurately as previously published methods. We confirm the correlation between the RV and stellar activity time series reaches a maximum at separations of a few days, and find evidence of non-stationary behaviour in the time series, associated with an approaching solar activity minimum.

Camacho, J. D. (ORCID:0000000151215560)↗
DOE OSTI2025

Explaining word embeddings with perfect fidelity: a case study in predicting research impact

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

Coarse-grained models↗
DOE OSTI2021

Machine learning elastic constants of multi-component alloys

The present manuscript explores application of machine learning methods for determining elastic constants and other derived mechanical properties of multi-component alloys. Here, a number of machine learning models, including linear regression, neural network and random forest based models, are trained and tested on a dataset of binary alloys generated using density functional theory (DFT) calculations and spanning over a large number of elemental species in the periodic table. Starting with a wide range of simple and easily accessible compositionally-averaged elemental features, a correlation-based feature selection strategy was used to systematically down-select a set of most relevant features towards the prediction of the elasticity tensor components. The true predictive performance and the associated uncertainties of the models were established by testing on unseen data and bootstrapping, respectively. A single and pair-wise feature partial dependence analysis was performed to visualize the average property trends in the multi-dimensional feature space in order to further understand the achieved predictive performance. The utility of the trained model is further demonstrated by obtaining sufficiently accurate yet highly efficient approximations for bulk modulus, Young’s modulus, shear modulus and Poisson’s ratio for alloys beyond the binary space (i.e., two-component alloys) on which the model was originally trained. More importantly, we test and validate the predictive performance of the developed model directly against the experimentally measured elastic constants of technologically relevant multi-component alloys (such as, Ni- and Ti-based alloys). Finally, utility of such a data-enabled route is demonstrated by predicting the possible range of various elastic properties for vast composition space available within the five component Ni-Cr-Fe-Mo-W alloy system in a high-throughput manner.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Machine learning and deep learning for mineralogy interpretation and CO 2 saturation estimation in geological carbon Storage: A case study in the Illinois Basin

Carbon capture and storage (CCS) is a promising approach to simultaneously maintaining energy security and reducing carbon dioxide (CO 2 ) emissions under the current energy portfolio that is dominated by fossil fuel energy. Pre-injection formation characterization and post-injection CO 2 monitoring are two critical tasks to guarantee storage efficiency in CCS. The CCS projects in the Illinois Basin, the first large-scale CO 2 injection into saline aquifers in the United States, employed conventional and the latest pulsed neutron logging (PNL) tools for mineralogy interpretation and CO 2 saturation estimation, which provide valuable references for future CCS projects. Because of the inherent fuzziness of petrophysical measurements and complex subsurface heterogeneity, interpreting well-logging data is time-consuming, and its accuracy can be user-biased. In recent years, data-driven methods have been widely used to capture the non-linear patterns between input features and interpretation results. This work applied and evaluated four commonly used machine learning (ML) models, including ridge regression (RR), random forest (RF), gradient boosting regression (GBR), support vector regression (SVR), and one deep learning (DL) model, the artificial neural network (ANN). We optimized the hyperparameters of the four ML models and the DL model using the simulated annealing algorithm and the grid search strategy, respectively. The input features of the mineralogy interpretation models were eleven conventional well-logging parameters, and the label data (i.e., ground truth) were the porosity and volumetric fractions of six minerals, including quartz, feldspar, dolomite, calcite, clay, and iron minerals. The results demonstrated that the GBR and RF models were superior in predicting volumetric fractions of minerals and porosity; label data with low coefficient of variation (CV) values tended to yield better performance. For CO 2 saturation estimation, the RF was the best-performing model, followed by SVR, ANN, GBR, and RR. Furthermore, we conducted feature importance ranking using the permutation importance algorithm and found that the formation sigma and well pressure were the most important features in this study. In conclusion, the study of CCS projects in the Illinois Basin bridges the gap between the limited knowledge and understanding of geological carbon storage and the increasing demand for reliable, cost-effective, and sustainable energy solutions.

58 GEOSCIENCES↗
DOE OSTI2024

Uncertainty Quantification and Sensitivity Analysis of Non-Nuclear Advanced Controls Testbed Reactor Mockup

The research presented in this report describes our progress in applying stochastic methods and uncertainty quantification, parametric study, and variance-based sensitivity analysis (also known as Sobol sensitivity analysis) to a full-core model of a nuclear thermal propulsion (NTP) system simulated with Griffin, with the goal of developing a reduced order (surrogate) model which can be rapidly sampled while perturbing multiple input parameters. In this NTP system, reactivity and power feedback affect the rotation of control drums, which are controlled by a hybrid proportional, integral and derivative (PID) controller, actuated by the power demand and reactivity feedback from the numerical model. This model uses reactor kinetic feedback (mean generation time and $\beta$ from a transient Griffin simulation executed with the improved quasi-static method to provide the kinetic parameters) as inputs to functions which control the CD rotation angle. Using a number of stochastic method approaches, we developed a dual purpose training-surrogate model of the NTP system using polynomial regression. The trained model can be rapidly sampled while simultaneously perturbing various input parameters of the model, such as coefficients on the PID control, or temperature (directly affect the neutron cross section). The surrogate model delivers accurate results orders-of-magnitude faster (minutes, not days) than the base model. Once the base model has been trained, distributions of the uncertain parameters can be changed at will to investigate the effects of perturbing multiple inputs and their effect on the output. For example, coefficients used in the PID control system may vary due to some physical interference, or there may be uncertainty in the temperature of the neutron cross sections in various regions of the reactor. A distribution can be placed on these parameters and operational boundaries can be determined. The goal of this work is to support development of an advanced control system to operate CDs in a functioning NTP system.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deterministic symbolic regression with derivative information: General methodology and application to equations of state

Symbolic regression methods simultaneously determine the model functional form and the regression parameter values by generating expression trees. Symbolic regression can capture the complexity of real–world phenomena but the use of deterministic optimization for symbolic regression has been limited due to the complexity of the search space of existing formulations. Herein we present a novel deterministic mixed–integer nonlinear programming formulation for symbolic regression that incorporates derivative constraints through auxiliary expression trees. By applying the chain rule to mathematical operations, binary expression trees are capable of representing the calculation of first and second derivatives. We apply this formulation to illustrative examples using derivative information to show increased model discrimination capability. In addition, we perform a case study of a thermodynamic equation of state to gain insight on valid functional forms with thermodynamics–based constraints on the first and second derivatives.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Identification of Life Models for Li-Ion Batteries Using Penalized Regression and Bilevel Optimization

Reduced-order physics-based life models are extremely useful for rapidly predicting battery state-of-health and for simulating battery lifetime in arbitrary aging conditions. However, identification of well-parameterized models is difficult. This is because, for maximum usefulness in predicting lifetime under a variety of conditions, aging test data exhibits many degradation mechanisms, which all need to be accurately modeled. However, because aging tests are time-consuming and expensive, especially for large-format batteries, a minimum of tests are conducted while probing many stress factors. Building a well-parameterized model is then very challenging: an under-parameterized model will neglect critical degradation modes, and an over-parameterized model will extrapolate poorly to new testing conditions. To complicate this matter, the functional form of the model for any individual degradation rate can be very difficult to identify. In this work, the statistical tools of penalized regression and bilevel optimization are used to help identify both the functional forms of and optimize the parameters of reduced-order life models, accelerating identification of robust models. Model robustness is demonstrated through traditional statistics methods of cross-validation and sensitivity analysis, uncertainty quantification through bootstrap resampling and Monte-Carlo simulation, and simulation of real-world use cases.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Development of a Filtered CFD-DEM Drag Model with Multiscale Markers Using an Artificial Neural Network and Nonlinear Regression

Here, the accuracy of coarse-grained Euler-Lagrangian simulations of fluidized beds heavily depends on the mesoscale drag models to account for the influences of the unresolved sub-grid structures. Traditional filtered drag models are regressed with mesoscale markers such as voidage and slip velocities. In this research, a filtered drag was regressed with both mesoscale and macro-scale markers using fine grid Computational Fluid Dynamics - Discrete Element Method (CFD-DEM) simulations. The traditional non-linear regression method was compared with machine learning regression using an Artificial Neural Network (ANN) implemented in PyTorch and coupled with MFiX. The new drag showed higher accuracy than the Wen-Yu drag and another filtered drag derived from the two-fluid model. The nonlinear regression shows slightly better results than ANN regression in cases with similar R 2 values. The utilization of the gas inlet velocity as an additional macro-scale marker reduced the errors by up to 55.3% in the tested cases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

A Tutorial on Bayesian analysis of linear shock compression data

Gas gun and other shock compression experiments often produce shock wave velocity measurements that are linearly associated with particle velocity. Traditionally, this empirical relationship is quantified with a single Hugoniot curve that is estimated using least squares regression. However, for downstream modeling and simulation tasks, it is often more useful to have multiple Hugoniot curves in the pressure–volume plane that are consistent with the data. We employ Bayesian uncertainty quantification methods as a framework for propagating measurement uncertainty through to model parameters and predictions. Specifically, this Tutorial shows how to sample multiple Hugoniot curves in the pressure–volume plane that are consistent with the shock wave-particle velocity measurements in a two-step Bayesian approach. First, we obtain an analytical expression for the posterior distribution of the linear model parameters using Bayesian linear regression. Second, we propagate samples from the posterior distribution through the Rankine–Hugoniot equations to yield Hugoniot curves in the pressure–volume plane. The procedure is demonstrated with publicly available data on argon, copper, and nickel, and compared against bootstrapping and linear regression. The Bayesian procedure is shown to be interpretable, computationally inexpensive, and less sensitive than an alternative bootstrapping approach to the removal of the point in the copper dataset that has the largest particle velocity. As a Tutorial on Bayesian methodology for the shock compression community, we provide several derivations and explanations that make this paper self-contained, and make all code and data available at github.com/llnl/BALSCD.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Data-driven electrolyte design for lithium metal anodes

Improving Coulombic efficiency (CE) is key to the adoption of high energy density lithium metal batteries. Liquid electrolyte engineering has emerged as a promising strategy for improving the CE of lithium metal batteries, but its complexity renders the performance prediction and design of electrolytes challenging. Here, we develop machine learning (ML) models that assist and accelerate the design of high-performance electrolytes. Using the elemental composition of electrolytes as the features of our models, we apply linear regression, random forest, and bagging models to identify the critical features for predicting CE. Our models reveal that a reduction in the solvent oxygen content is critical for superior CE. We use the ML models to design electrolyte formulations with fluorine-free solvents that achieve a high CE of 99.70%. This work highlights the promise of data-driven approaches that can accelerate the design of high-performance electrolytes for lithium metal batteries.

25 ENERGY STORAGE↗
DOE OSTI2024

Data-Driven Digital Twin for Reliability Assessment of DC/DC Buck Converter

In commercial applications, the operation of DC/DC converters significantly impacts overall system performance and long-term reliability. This study introduces a data-driven digital twin (DT) approach for estimating critical degradation parameters of DC/DC BUCK converter under steady-state condition. Initially, a circuit-level MATLAB/Simulink digital model (DM C ) is refined against a hardware prototype’s switching model dataset using offline particle swarm optimization. The optimized digital model’s steady-state response is then verified with its average model response while varying the duty and load. Subsequently, degradation profiles are imposed on the inductor, capacitor, MOSFET in the DMC. A large dataset is generated from this model, allowing training, validation, and testing of machine learning (ML) models for component health regression tasks. The proposed method employs random forest ML models, achieving impressive regression results with a squared R value as high as 0.99978 and a root mean square error of 4.2× 10 –6 . The method is further validated on a medium power level DC/DC BUCK prototype with varying load conditions, and includes the analysis of MOSFET’s on-resistance under degradation conditions. This data-driven DT method shows promise for identifying parasitic degradation and ohmic loss parameters, enhancing converter reliability assessments in a non-invasive, generalized, and computationally efficient manner.

14 SOLAR ENERGY↗
DOE OSTI2025

Beyond pinball loss: Quantile methods for calibrated uncertainty quantification

Amongthemanywaysofquantifying uncertainty in a regression setting, specifying the full quantile function is attractive, as quantiles are amenable to interpretation and evaluation. A model that predicts the true conditional quantiles for each input, at all quantile levels, presents a correct and efficient representation of the underlying uncertainty. To achieve this, many current quantile-based methods focus on optimizing the pinball loss. However, this loss restricts the scope of applicable regression models, limits the ability to target many desirable properties (e.g. calibration, sharpness, centered intervals), and may produce poor conditional quantiles. In this work, we develop new quantile methods that address these shortcomings. In particular, we propose methods that can apply to any class of regression model, select an explicit balance between calibration and sharpness, optimize for calibration of centered intervals, and produce more accurate conditional quantiles. We provide a thorough experimental evaluation of our methods, which includes a high dimensional uncertainty quantification task in nuclear fusion.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Monitoring Noble Gases (Xe and Kr) and Aerosols (Cs and Rb) in a Molten Salt Reactor Surrogate Off-Gas Stream Using Laser-Induced Breakdown Spectroscopy (LIBS)

In this study with surrogate materials we show that laser-induced breakdown spectroscopy (LIBS) is a robust tool with promising capability toward monitoring gaseous (Xe and Kr) and aerosol (Cs and Rb) species in an off-gas stream from a molten salt reactor (MSR). MSRs will continually evolve fission products into the cover gas flowing across the reactor headspace. The cover gas entrains Xe and Kr gases, along with aerosol particles, before passing into an off-gas treatment system. Univariate models of Xe and Kr peaks showed a strong correlation to concentration indicated by their coefficients of determination of 0.983 and 0.997, respectively. Multivariate models were built for all four analytes using partial least squares regression coupled with preprocessing steps including normalization, trimming, and/or genetic algorithm derived filters. The models were evaluated by predicting the concentrations of the analytes in four validation samples, in which all calibration models were successfully validated at a confidence interval of 99.9%. Finally, pressure controllers were used to regulate the mass flow rate of Kr flowing into the measurement cell in sinusoidal and stepwise waveforms to test the real-time monitoring capabilities of the regression models. Both univariate and partial least squares Kr models were able to successfully quantify the gas concentration in the real-time evaluation. The root mean squared error of prediction (RMSEP) values for these real-time tests were calculated to be 0.051, 0.060, and 0.121 mol% demonstrating the measurement systems’ capability to perform online monitoring with acceptable accuracy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

A new method for predicting hurricane rapid intensification based on co-occurring environmental parameters

Abstract Tropical cyclones (TCs) that undergo Rapid Intensification (RI) can pose serious socioeconomic threats and can potentially result in major damaging impacts along coastal areas. Considering the complexity of various physical mechanisms that play a role in RI and its relatively low probability of occurrence, predicting RI remains a major operational challenge. In this study, we propose a simple deterministic binary classification model based on the co-occurrence of environmental parameters (MCE) to predict an RI event. More specifically, the model determines the possibility of RI based on a simple count of the number of environmental predictors deemed favorable and unfavorable. We compare our model results to logistic regression (LR) and decision tree (DT) models, well-trained using the same set of environmental predictors. Results reveal that at an RI threshold of 30 kt, the MCE exhibits a critical success index score of 0.233 which is 14% higher than DT and LR model performances. When tested at multiple RI thresholds, the MCE displays relatively higher skill scores across multiple metrics. By simultaneously evaluating the favorability of predictors, the MCE is able to comparatively reduce the number of false alarms predicted when certain predictors are unfavorable toward RI. Interpreting these model results to gain a physical understanding of how co-occurring environmental parameters can affect RI, we highlight future directions for using models based on the MCE approach to understand and predict TC RI as well as other meteorological extremes.

54 ENVIRONMENTAL SCIENCES↗