Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “regression models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Application of a Machine Learning Algorithm in Generating an Evapotranspiration Data Product From Coupled Thermal Infrared and Microwave Satellite Observations

Land surface evapotranspiration (ET) is one of the main energy sources for atmospheric dynamics and a critical component of the local, regional, and global water cycles. Consequently, accurate measurement or estimation of ET is one of the most active topics in hydro-climatology research. With massive and spatially distributed observational data sets of land surface properties and environmental conditions being collected from the ground, airborne or space-borne platforms daily over the past few decades, many research teams have started to use big data science to advance the ET estimation methods. The Geostationary satellite Evapotranspiration and Drought (GET-D) product system was developed at the National Oceanic and Atmospheric Administration (NOAA) in 2016 to generate daily ET and drought maps operationally. The primary inputs of the current GET-D system are the thermal infrared (TIR) observations from NOAA GOES satellite series. Because of the cloud contamination to the TIR observations, the spatial coverage of the daily GET-D ET product has been severely impacted. Based on the most recent advances, we have tested a machine learning algorithm to estimate all-weather land surface temperature (LST) from TIR and microwave (MW) combined satellite observations. With the regression tree machine learning approach, we can combine the high accuracy and high spatial resolution of GOES TIR data with the better spatial coverage of passive microwave observations and LST simulations from a land surface model (LSM). The regression tree model combines the three LST data sources for both clear and cloudy days, which enables the GET-D system to derive an all-weather ET product. This paper reports how the all-weather LST and ET are generated in the upgraded GET-D system and provides an evaluation of these LST and ET estimates with ground measurements. The results demonstrate that the regression tree machine learning method is feasible and effective for generating daily ET under all weather conditions with satisfactory accuracy from the big volume of satellite observations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

bayesian_tensor_regression

We plan to release the code used to perform the experiment described in our upcoming publication, entitled “Bayesian Tensor Modeling for Distribution-on-Distribution Regression.” This code a Bayesian regression model with a multi-way Dirichlet prior to tensor input distributions. All code to be released implements a new model that is intended for open-source publication.

Murph, Alexander C. [Los Alamos National Lab]↗
DOE OSTI2024

Subsampling of Parametric Models with Bifidelity Boosting

Least squares regression is a ubiquitous tool for building emulators (a.k.a. surrogate models) of problems across science and engineering for purposes such as design space exploration and uncertainty quantification. When the regression data are generated using an experimental design process (e.g., a quadrature grid) involving computationally expensive models, or when the data size is large, sketching techniques have shown promise at reducing the cost of the construction of the regression model while ensuring accuracy comparable to that of the full data. However, random sketching strategies, such as those based on leverage scores, lead to regression errors that are random and may exhibit large variability. To mitigate this issue, we present a novel boosting approach that leverages cheaper, lower-fidelity data of the problem at hand to identify the best sketch among a set of candidate sketches. This in turn specifies the sketch of the intended high-fidelity model and the associated data. We provide theoretical analyses of this bifidelity boosting (BFB) approach and discuss the conditions the low- and high-fidelity data must satisfy for a successful boosting. In doing so, we derive a bound on the residual norm of the BFB sketched solution relating it to its ideal, but computationally expensive, high-fidelity boosted counterpart. Finally, empirical results on both manufactured and PDE data corroborate the theoretical analyses and illustrate the efficacy of the BFB solution in reducing the regression error, as compared to the nonboosted solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Band gap predictions of double perovskite oxides using machine learning

Abstract The compositional and structural variety inherent to oxide perovskites spawn wide-ranging applications. In perovskites, the band gap E g , a key material parameter for these applications, can be optimally controlled by varying the composition. Here, we implement a hierarchical screening process in which two cross-validated and predictive machine learning models for band gap classification and regression, trained using exhaustive datasets that span 68 elements of the periodic table, are applied sequentially. The classification model separates wide band gap materials, with E g ≥ 0.5 eV, from materials which have zero or relatively small band gaps, namely E g < 0.5 eV, and the second regression model quantitatively predicts the gap value of the wide band gap compounds. The study down-selects 13,589 cubic oxide perovskite compositions that are predicted to be experimentally formable, thermodynamically stable, and have a wide band gap. Of these, a subset of 310 compounds, which are predicted to be stable and formable with a confidence greater than 90%, are identified for further investigation. Our models are methodically analyzed via performance metrics and inter-dependence of model features to gain physical insight into the band gap prediction problem. Design maps to identify the variation of band gap with substitution of different elements are also presented.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Modelling stellar activity with Gaussian process regression networks

ABSTRACT Stellar photospheric activity is known to limit the detection and characterization of extrasolar planets. In particular, the study of Earth-like planets around Sun-like stars requires data analysis methods that can accurately model the stellar activity phenomena affecting radial velocity (RV) measurements. Gaussian Process Regression Networks (GPRNs) offer a principled approach to the analysis of simultaneous time series, combining the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian Processes. Using HARPS-N solar spectroscopic observations encompassing three years, we demonstrate that this framework is capable of jointly modelling RV data and traditional stellar activity indicators. Although we consider only the simplest GPRN configuration, we are able to describe the behaviour of solar RV data at least as accurately as previously published methods. We confirm the correlation between the RV and stellar activity time series reaches a maximum at separations of a few days, and find evidence of non-stationary behaviour in the time series, associated with an approaching solar activity minimum.

Camacho, J. D. (ORCID:0000000151215560)↗
DOE OSTI2025

Explaining word embeddings with perfect fidelity: a case study in predicting research impact

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

Coarse-grained models↗
DOE OSTI2021

Machine learning elastic constants of multi-component alloys

The present manuscript explores application of machine learning methods for determining elastic constants and other derived mechanical properties of multi-component alloys. Here, a number of machine learning models, including linear regression, neural network and random forest based models, are trained and tested on a dataset of binary alloys generated using density functional theory (DFT) calculations and spanning over a large number of elemental species in the periodic table. Starting with a wide range of simple and easily accessible compositionally-averaged elemental features, a correlation-based feature selection strategy was used to systematically down-select a set of most relevant features towards the prediction of the elasticity tensor components. The true predictive performance and the associated uncertainties of the models were established by testing on unseen data and bootstrapping, respectively. A single and pair-wise feature partial dependence analysis was performed to visualize the average property trends in the multi-dimensional feature space in order to further understand the achieved predictive performance. The utility of the trained model is further demonstrated by obtaining sufficiently accurate yet highly efficient approximations for bulk modulus, Young’s modulus, shear modulus and Poisson’s ratio for alloys beyond the binary space (i.e., two-component alloys) on which the model was originally trained. More importantly, we test and validate the predictive performance of the developed model directly against the experimentally measured elastic constants of technologically relevant multi-component alloys (such as, Ni- and Ti-based alloys). Finally, utility of such a data-enabled route is demonstrated by predicting the possible range of various elastic properties for vast composition space available within the five component Ni-Cr-Fe-Mo-W alloy system in a high-throughput manner.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Machine learning and deep learning for mineralogy interpretation and CO 2 saturation estimation in geological carbon Storage: A case study in the Illinois Basin

Carbon capture and storage (CCS) is a promising approach to simultaneously maintaining energy security and reducing carbon dioxide (CO 2 ) emissions under the current energy portfolio that is dominated by fossil fuel energy. Pre-injection formation characterization and post-injection CO 2 monitoring are two critical tasks to guarantee storage efficiency in CCS. The CCS projects in the Illinois Basin, the first large-scale CO 2 injection into saline aquifers in the United States, employed conventional and the latest pulsed neutron logging (PNL) tools for mineralogy interpretation and CO 2 saturation estimation, which provide valuable references for future CCS projects. Because of the inherent fuzziness of petrophysical measurements and complex subsurface heterogeneity, interpreting well-logging data is time-consuming, and its accuracy can be user-biased. In recent years, data-driven methods have been widely used to capture the non-linear patterns between input features and interpretation results. This work applied and evaluated four commonly used machine learning (ML) models, including ridge regression (RR), random forest (RF), gradient boosting regression (GBR), support vector regression (SVR), and one deep learning (DL) model, the artificial neural network (ANN). We optimized the hyperparameters of the four ML models and the DL model using the simulated annealing algorithm and the grid search strategy, respectively. The input features of the mineralogy interpretation models were eleven conventional well-logging parameters, and the label data (i.e., ground truth) were the porosity and volumetric fractions of six minerals, including quartz, feldspar, dolomite, calcite, clay, and iron minerals. The results demonstrated that the GBR and RF models were superior in predicting volumetric fractions of minerals and porosity; label data with low coefficient of variation (CV) values tended to yield better performance. For CO 2 saturation estimation, the RF was the best-performing model, followed by SVR, ANN, GBR, and RR. Furthermore, we conducted feature importance ranking using the permutation importance algorithm and found that the formation sigma and well pressure were the most important features in this study. In conclusion, the study of CCS projects in the Illinois Basin bridges the gap between the limited knowledge and understanding of geological carbon storage and the increasing demand for reliable, cost-effective, and sustainable energy solutions.

58 GEOSCIENCES↗
DOE OSTI2024

Uncertainty Quantification and Sensitivity Analysis of Non-Nuclear Advanced Controls Testbed Reactor Mockup

The research presented in this report describes our progress in applying stochastic methods and uncertainty quantification, parametric study, and variance-based sensitivity analysis (also known as Sobol sensitivity analysis) to a full-core model of a nuclear thermal propulsion (NTP) system simulated with Griffin, with the goal of developing a reduced order (surrogate) model which can be rapidly sampled while perturbing multiple input parameters. In this NTP system, reactivity and power feedback affect the rotation of control drums, which are controlled by a hybrid proportional, integral and derivative (PID) controller, actuated by the power demand and reactivity feedback from the numerical model. This model uses reactor kinetic feedback (mean generation time and $\beta$ from a transient Griffin simulation executed with the improved quasi-static method to provide the kinetic parameters) as inputs to functions which control the CD rotation angle. Using a number of stochastic method approaches, we developed a dual purpose training-surrogate model of the NTP system using polynomial regression. The trained model can be rapidly sampled while simultaneously perturbing various input parameters of the model, such as coefficients on the PID control, or temperature (directly affect the neutron cross section). The surrogate model delivers accurate results orders-of-magnitude faster (minutes, not days) than the base model. Once the base model has been trained, distributions of the uncertain parameters can be changed at will to investigate the effects of perturbing multiple inputs and their effect on the output. For example, coefficients used in the PID control system may vary due to some physical interference, or there may be uncertainty in the temperature of the neutron cross sections in various regions of the reactor. A distribution can be placed on these parameters and operational boundaries can be determined. The goal of this work is to support development of an advanced control system to operate CDs in a functioning NTP system.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS1983

Determination of airplane model structure from flight data using splines and stepwise regression

A procedure for the determination of airplane model structure from flight data is presented. The model is based on a polynomial spline representation of the aerodynamic coefficients, and the procedure is implemented by use of a stepwise regression. First, a form of the aerodynamic force and moment coefficients amenable to the utilization of splines is developed. Next, expressions for the splines in one and two variables are introduced. Then the steps in the determination of an aerodynamic model structure and the estimation of parameters are discussed briefly. The focus is on the application to flight data of the techniques developed.

Klein, V.↗
DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deterministic symbolic regression with derivative information: General methodology and application to equations of state

Symbolic regression methods simultaneously determine the model functional form and the regression parameter values by generating expression trees. Symbolic regression can capture the complexity of real–world phenomena but the use of deterministic optimization for symbolic regression has been limited due to the complexity of the search space of existing formulations. Herein we present a novel deterministic mixed–integer nonlinear programming formulation for symbolic regression that incorporates derivative constraints through auxiliary expression trees. By applying the chain rule to mathematical operations, binary expression trees are capable of representing the calculation of first and second derivatives. We apply this formulation to illustrative examples using derivative information to show increased model discrimination capability. In addition, we perform a case study of a thermodynamic equation of state to gain insight on valid functional forms with thermodynamics–based constraints on the first and second derivatives.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Identification of Life Models for Li-Ion Batteries Using Penalized Regression and Bilevel Optimization

Reduced-order physics-based life models are extremely useful for rapidly predicting battery state-of-health and for simulating battery lifetime in arbitrary aging conditions. However, identification of well-parameterized models is difficult. This is because, for maximum usefulness in predicting lifetime under a variety of conditions, aging test data exhibits many degradation mechanisms, which all need to be accurately modeled. However, because aging tests are time-consuming and expensive, especially for large-format batteries, a minimum of tests are conducted while probing many stress factors. Building a well-parameterized model is then very challenging: an under-parameterized model will neglect critical degradation modes, and an over-parameterized model will extrapolate poorly to new testing conditions. To complicate this matter, the functional form of the model for any individual degradation rate can be very difficult to identify. In this work, the statistical tools of penalized regression and bilevel optimization are used to help identify both the functional forms of and optimize the parameters of reduced-order life models, accelerating identification of robust models. Model robustness is demonstrated through traditional statistics methods of cross-validation and sensitivity analysis, uncertainty quantification through bootstrap resampling and Monte-Carlo simulation, and simulation of real-world use cases.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Development of a Filtered CFD-DEM Drag Model with Multiscale Markers Using an Artificial Neural Network and Nonlinear Regression

Here, the accuracy of coarse-grained Euler-Lagrangian simulations of fluidized beds heavily depends on the mesoscale drag models to account for the influences of the unresolved sub-grid structures. Traditional filtered drag models are regressed with mesoscale markers such as voidage and slip velocities. In this research, a filtered drag was regressed with both mesoscale and macro-scale markers using fine grid Computational Fluid Dynamics - Discrete Element Method (CFD-DEM) simulations. The traditional non-linear regression method was compared with machine learning regression using an Artificial Neural Network (ANN) implemented in PyTorch and coupled with MFiX. The new drag showed higher accuracy than the Wen-Yu drag and another filtered drag derived from the two-fluid model. The nonlinear regression shows slightly better results than ANN regression in cases with similar R 2 values. The utilization of the gas inlet velocity as an additional macro-scale marker reduced the errors by up to 55.3% in the tested cases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1979

SCI model structure determination program (OSR) user's guide

The computer program, OSR (Optimal Subset Regression) which estimates models for rotorcraft body and rotor force and moment coefficients is described. The technique used is based on the subset regression algorithm. Given time histories of aerodynamic coefficients, aerodynamic variables, and control inputs, the program computes correlation between various time histories. The model structure determination is based on these correlations. Inputs and outputs of the program are given.

Source record↗
NASA NTRS2017

Probabilistic Forecasting of Surface Ozone with a Novel Statistical Approach

The recent change in the Environmental Protection Agency's surface ozone regulation, lowering the surface ozone daily maximum 8-h average (MDA8) exceedance threshold from 75 to 70 ppbv, poses significant challenges to U.S. air quality (AQ) forecasters responsible for ozone MDA8 forecasts. The forecasters, supplied by only a few AQ model products, end up relying heavily on self-developed tools. To help U.S. AQ forecasters, this study explores a surface ozone MDA8 forecasting tool that is based solely on statistical methods and standard meteorological variables from the numerical weather prediction (NWP) models. The model combines the self-organizing map (SOM), which is a clustering technique, with a step wise weighted quadratic regression using meteorological variables as predictors for ozone MDA8. The SOM method identifies different weather regimes, to distinguish between various modes of ozone variability, and groups them according to similarity. In this way, when a regression is developed for a specific regime, data from the other regimes are also used, with weights that are based on their similarity to this specific regime. This approach, regression in SOM (REGiS), yields a distinct model for each regime taking into account both the training cases for that regime and other similar training cases. To produce probabilistic MDA8 ozone forecasts, REGiS weighs and combines all of the developed regression models on the basis of the weather patterns predicted by an NWP model. REGiS is evaluated over the San Joaquin Valley in California and the northeastern plains of Colorado. The results suggest that the model performs best when trained and adjusted separately for an individual AQ station and its corresponding meteorological site.

Ozone variables↗
NASA NTRS1992

Improvement of observer/Kalman filter identification (OKID) by residual whitening

This paper presents a time-domain method to identify a state space model of a linear system and its corresponding observer/Kalman filter from a given set of general input-output data. The identified filter has the properties that its residual is minimized in the least squares sense, orthogonal to the time-shifted versions of itself, and to the given input-output data sequence. The connection between the state space model and a particular auto-regressive moving average description of a linear system is made in terms of the Kalman filter and a deadbeat gain matrix. The procedure first identifies the Markov parameters of an observer system, from which a state space model of the system and the filter gain are computed. The developed procedure is shown to improve results obtained by an existing observer/Kalman filter identification method, which is based on an auto-regressive model without the moving average terms. Numerical and experimental results are presented to illustrate the proposed method.

Phan, Minh↗