Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

J-PLUS: Support vector regression to measure stellar parameters

Stellar parameters are among the most important characteristics in studies of stars which, in traditional methods, are based on atmosphere models. However, time, cost, and brightness limits restrain the efficiency of spectral observations. The Javalambre Photometric Local Universe Survey (J-PLUS) is an observational campaign that aims to obtain photometry in 12 bands. Owing to its characteristics, J-PLUS data have become a valuable resource for studies of stars. Machine learning provides powerful tools for efficiently analyzing large data sets, such as the one from J-PLUS, and enables us to expand the research domain to stellar parameters. The main goal of this study is to construct a support vector regression (SVR) algorithm to estimate stellar parameters of the stars in the first data release of the J-PLUS observational campaign. The training data for the parameter's regressions are featured with 12-waveband photometry from J-PLUS and are crossidentified with spectrum-based catalogs. These catalogs are from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, the Apache Point Observatory Galactic Evolution Experiment, and the Sloan Extension for Galactic Understanding and Exploration. We then label them with the stellar effective temperature, the surface gravity, and the metallicity. Ten percent of the sample is held out to apply a blind test. We develop a new method, a multi-model approach, in order to fully take into account, the uncertainties of both the magnitudes and the stellar parameters. The method utilizes more than 200 models to apply the uncertainty analysis. We present a catalog of 2 493 424 stars with the root mean square error of 160 K in the effective temperature regression, 0.35 in the surface gravity regression, and 0.25 in the metallicity regression. We also discuss the advantages of this multi-model approach and compare it to other machine-learning methods.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Sparse regression for plasma physics

Many scientific problems can be formulated as sparse regression, i.e., regression onto a set of parameters when there is a desire or expectation that some of the parameters are exactly zero or do not substantially contribute. This includes many problems in signal and image processing, system identification, optimization, and parameter estimation methods such as Gaussian process regression. Sparsity facilitates exploring high-dimensional spaces while finding parsimonious and interpretable solutions. In the present work, we illustrate some of the important ways in which sparse regression appears in plasma physics and point out recent contributions and remaining challenges to solving these problems in this field. Further, a brief review is provided for the optimization problem and the state-of-the-art solvers, especially for constrained and high-dimensional sparse regression.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Three-Stage Adjusted Regression Forecasting for Software Defect Prediction

In this paper, a three-stage adjusted regression forecasting model is proposed to forecast the local regression model [6]. This is a growth curve approximation model that predicts the parameters for a future linear model based on a sliding window of previous linear models. The three stages of the model are as follows: • Initial fit: train regression models on a sliding window of the date and record model coefficients. • Prediction: fit new regression models to the coefficient lists and predict the value of the next coefficient. • Error correction: correct coefficient prediction error using the residual of the last point of the coefficient list and moving average. The resulting model from the multi-stage process is a forecast of the local regression model that represents the future window of data and is referred to as the predicted line. Results suggest the three-stage model demonstrates better prediction capability compared to existing solutions.

Pritchard, Shadow↗
DOE OSTI2023

Large-Scale Inference of Multivariate Regression for Heavy-Tailed and Asymmetric Data

Large-scale multivariate regression is a fundamental statistical tool with a wide range of applications. Here, this study considers the problem of simultaneously testing a large number of general linear hypotheses, encompassing covariate-effect analysis, analysis of variance, and model comparisons. The challenge that accompanies a large number of tests is the ubiquitous presence of heavy-tailed and/or highly skewed measurement noise, which is the main reason for the failure of conventional least squares-based methods. For large-scale multivariate regression, we develop a set of robust inference methods to explore data features such as heavy tailedness and skewness, which are not visible to least squares methods. The new testing procedure is based on the data-adaptive Huber regression and a new covariance estimator of regression estimates. Under mild conditions, we show that our methods produce consistent estimates of the false discovery proportion. Extensive numerical experiments and an empirical study on quantitative linguistics demonstrate the advantage of the proposed method over many state-of-the-art methods when the data are generated from heavy-tailed and/or skewed distributions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

SRBench++: Principled Benchmarking of Symbolic Regression With Domain-Expert Interpretation

Symbolic regression searches for analytic expressions that accurately describe studied phenomena. The main promise of this approach is that it may return an interpretable model that can be insightful to users, while maintaining high accuracy. The current standard for benchmarking these algorithms is SRBench, which evaluates methods on hundreds of datasets that are a mix of real-world and simulated processes spanning multiple domains. At present, the ability of SRBench to evaluate interpretability is limited to measuring the size of expressions on real-world data, and the exactness of model forms on synthetic data. In practice, model size is only one of many factors used by subject experts to determine how interpretable a model truly is. Furthermore, SRBench does not characterize algorithm performance on specific, challenging sub-tasks of regression such as feature selection and evasion of local minima. In this work, we propose and evaluate an approach to benchmarking SR algorithms that addresses these limitations of SRBench by 1) incorporating expert evaluations of interpretability on a domain-specific task, and 2) evaluating algorithms over distinct properties of data science tasks. We evaluate 12 modern symbolic regression algorithms on these benchmarks and present an in-depth analysis of the results, discuss current challenges of symbolic regression algorithms and highlight possible improvements for the benchmark itself.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Evaluating county-level lung cancer incidence from environmental radiation exposure, PM 2.5 , and other exposures with regression and machine learning models

Characterizing the interplay between exposures shaping the human exposome is vital for uncovering the etiology of complex diseases. For example, cancer risk is modified by a range of multifactorial external environmental exposures. Environmental, socioeconomic, and lifestyle factors all shape lung cancer risk. However, epidemiological studies of radon aimed at identifying populations at high risk for lung cancer often fail to consider multiple exposures simultaneously. For example, moderating factors, such as PM 2.5 , may affect the transport of radon progeny to lung tissue. This ecological analysis leveraged a population-level dataset from the National Cancer Institute’s Surveillance, Epidemiology, and End-Results data (2013–17) to simultaneously investigate the effect of multiple sources of low-dose radiation (gross γ activity and indoor radon) and PM 2.5 on lung cancer incidence rates in the USA. County-level factors (environmental, sociodemographic, lifestyle) were controlled for, and Poisson regression and random forest models were used to assess the association between radon exposure and lung and bronchus cancer incidence rates. Tree-based machine learning (ML) method perform better than traditional regression: Poisson regression: 6.29/7.13 (mean absolute percentage error, MAPE), 12.70/12.77 (root mean square error, RMSE); Poisson random forest regression: 1.22/1.16 (MAPE), 8.01/8.15 (RMSE). The effect of PM 2.5 increased with the concentration of environmental radon, thereby confirming findings from previous studies that investigated the possible synergistic effect of radon and PM 2.5 on health outcomes. In summary, the results demonstrated (1) a need to consider multiple environmental exposures when assessing radon exposure’s association with lung cancer risk, thereby highlighting (1) the importance of an exposomics framework and (2) that employing ML models may capture the complex interplay between environmental exposures and health, as in the case of indoor radon exposure and lung cancer incidence.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
DOE OSTI2023

Bayesian projection pursuit regression

In projection pursuit regression (PPR), a univariate response variable is approximated by the sum of $M$ “ridge functions,” which are flexible functions of one-dimensional projections of a multivariate input variable. Traditionally, optimization routines are used to choose the projection directions and ridge functions via a sequential algorithm, and $M$ is typically chosen via cross-validation. Here, we introduce a novel Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To infer appropriate projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case ($M$=1), called the Bayesian Single Index Model; and use a Reversible Jump Markov chain Monte Carlo algorithm to infer the number of ridge functions $M$. We evaluate the predictive ability of our model in 20 simulated scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Finally, we generalize this methodology and demonstrate the ability to accurately model multivariate response variables. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Classification and regression models of audio and vibration signals for machine state monitoring in precision machining systems

Here we present a data-driven method for monitoring machine status in manufacturing processes. Audio and vibration data from precision machining are used for inference in two operating scenarios: (a) variable machine health states (anomaly detection); and (b) settings of machine operation (state estimation). Audio and vibration signals are first processed through Fast Fourier Transform and Principal Component Analysis to extract transformed and informative features. These features are then used in the training of classification and regression models for machine state monitoring. Specifically, three classifiers (K-nearest neighbors, convolutional neural networks and support vector machines) and two regressors (support vector regression and neural network regression) were explored, in terms of their accuracy in machine state prediction. It is shown that the audio and vibration signals are sufficiently rich in information about the machine that 100% state classification accuracy could be accomplished. Data fusion was also explored, showing overall superior accuracy of data-driven regression models.

42 ENGINEERING↗
DOE OSTI2020

A regression algorithm for accelerated lattice QCD that exploits sparse inference on the D-Wave quantum annealer

We propose a regression algorithm that utilizes a learned dictionary optimized for sparse inference on a D-Wave quantum annealer. In this regression algorithm, we concatenate the independent and dependent variables as a combined vector, and encode the high-order correlations between them into a dictionary optimized for sparse reconstruction. On a test dataset, the dependent variable is initialized to its average value and then a sparse reconstruction of the combined vector is obtained in which the dependent variable is typically shifted closer to its true value, as in a standard inpainting or denoising task. Here, a quantum annealer, which can presumably exploit a fully entangled initial state to better explore the complex energy landscape, is used to solve the highly non-convex sparse coding optimization problem. The regression algorithm is demonstrated for a lattice quantum chromodynamics simulation data using a D-Wave 2000Q quantum annealer and good prediction performance is achieved. The regression test is performed using six different values for the number of fully connected logical qubits, between 20 and 64. The scaling results indicate that a larger number of qubits gives better prediction accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Provably Accurate Randomized Sampling Algorithm for Logistic Regression

In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When the number of observations greatly exceeds the number of predictor variables, we present a simple, randomized sampling-based algorithm for logistic regression problem that guarantees high-quality approximations to both the estimated probabilities and the overall discrepancy of the model. Our analysis builds upon two simple structural conditions that boil down to randomized matrix multiplication, a fundamental and well-understood primitive of randomized numerical linear algebra. We analyze the properties of estimated probabilities of logistic regression when leverage scores are used to sample observations, and prove that accurate approximations can be achieved with a sample whose size is much smaller than the total number of observations. To further validate our theoretical findings, we conduct comprehensive empirical evaluations. Overall, our work sheds light on the potential of using randomized sampling approaches to efficiently approximate the estimated probabilities in logistic regression, offering a practical and computationally efficient solution for large-scale datasets.

Chowdhury, Agniva↗
DOE OSTI2026

Regression Convolutional Neural Network for Energy Estimation in NOvA

Regression Convolutional Neural Network for Energy Estimation in NOvA" Abstract: "NOvA (NuMI Off-Axis $\nu_e$ Appearance) is a long baseline neutrino experiment designed to measure neutrino oscillations over a distance of 810 km. NOvA employs a near and far detector to observe $\nu_\mu$ disappearance and $\nu_e$ appearance of neutrinos produced by the NuMI beam at Fermilab. Energy reconstruction is critical for precise measurements of neutrino oscillation parameters and cross sections, which are functions of neutrino energy. Energy estimation remains difficult due to the complexity of detector response and final state particle kinematics. We present a regression-based convolutional neural network (CNN) method that reconstructs neutrino and lepton energies based on raw pixel inputs for NOvA. The trained model is able to reconstruct event energy for different interaction modes and complex final states containing leptons and hadrons. Studies of regression CNN networks show improved energy resolution and reduced sensitivity to calibration scale uncertainties relative to traditional kinematics-based energy reconstruction techniques. The results demonstrate the potential of the regression CNN method for neutrino physics analyses by improving on standard kinematics-based reconstruction.

Zhao, Larry [UC, Irvine (main)]↗
DOE OSTI2021

Fast and Non-Destructive Determination of Water Content in Ionic Liquids at Varying Temperatures by Raman Spectroscopy and Multivariate Regression Analysis

Imidazolium acetate ionic liquids (ILs) have been utilized as promising solvents in many applications that involve varying water content and temperature. These experimental variables affect the anion-cation intermolecular interactions, which in turn influence the performance of the ILs in these applications. Here, this paper shows Raman spectroscopy can be used as an operando method to measure water content in IL solvents when simultaneous temperature changes may occur. The Raman spectra of 1-alkyl-3-methylimidazolium acetate ILs (alkyl chain length n = 2, 4, 6, 8) with varying water content (from 0.028 to 0.899 water mole fraction) and temperature (from 78.1 K to 423.1 K) were measured. Increasing the water content or decreasing the temperature of the tested ILs weakens the anion-cation intermolecular interactions. The water content of these ILs can be quantified even in conditions when the temperature is changing using Raman spectroscopy combined with multivariate regression analysis, including principal component regression (PCR), partial-least-squares regression (PLSR), and artificial neural networks (ANNs). The ANN model combined with partial-least-squares (PLS) achieves the highest prediction accuracy of water content in ILs at varying temperatures (RMSECV = 0.017, R 2 CV = 99.1%, RMSEP = 0.019, R 2 P = 98.8%, RPD = 8.93). Raman spectroscopy provides a potential fast non-destructive operando method to monitor the water content of ILs even in applications when the temperature may be simultaneously altered; this information can lead to the optimized use of these ILs in many applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Poisson-response Tensor-on-Tensor Regression and Applications

We introduce Poisson-response tensor-on-tensor regression (PToTR), a novel regression framework designed to handle tensor responses composed element-wise of random Poisson-distributed counts. Tensors, or multi-dimensional arrays, composed of counts are common data in fields such as inter national relations, social networks, epidemiology, and medical imaging, where events occur across multiple dimensions like time, location, and dyads. PToTR accommodates such tensor responses alongside tensor covariates, providing a versatile tool for multi dimensional data analysis. We propose algorithms for maximum likelihood estimation under a canonical polyadic (CP) structure on the regression coefficient tensor that satisfy the positivity of Poisson parameters and then provide an initial theoretical error analysis for PToTR estimators. We also demonstrate the utility of PToTR through three concrete applications: longitudinal data analysis of the Integrated Crisis Early Warning System database, positron emission tomography (PET) image reconstruction, and change-point detection of communication patterns in longitudinal dyadic data. These applications highlight the versatility of PToTR in addressing complex, structured count data across various domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Nonlinear Logistic Regression Mixture Experiment Modeling for Binary Data Using Dimensionally Reduced Components

This article presents and illustrates an approach using nonlinear logistic regression for modeling binary response data from a mixture experiment when the components can be partitioned into groups used to form dimensionally-reduced pseudocomponents (DRPs). A DRP is a linear combination of the components in a group, where the linear combinations over all groups are normalized so that the DRP proportions sum to unity. Nonlinear logistic regression is required because, after normalization of the linear combinations, the model expressed in terms of the DRPs is nonlinear in the parameters that specify the linear combinations. A method for obtaining nonparametric tolerance limits on the probability of a “success” for the binary response variable using a bootstrap approach is also presented. Having three DRPs enables viewing data and modeling results on a ternary plot even though there may be many more than three mixture components. A real database, involving whether or not nepheline crystals form in simulated nuclear waste glass after cooling, is used to illustrate the nonlinear logistic regression modeling and nonparametric tolerance limit approaches when there are three DRPs.

Waste glass, Nepheline, Mixture experiment, Logist↗
DOE OSTI2023

Discovering a reaction–diffusion model for Alzheimer’s disease by combining PINNs with symbolic regression

Misfolded tau proteins play a critical role in the progression and pathology of Alzheimer's disease. Recent studies suggest that the spatio-temporal pattern of misfolded tau follows a reaction-diffusion type equation. However, the precise mathematical model and parameters that characterize the progression of misfolded protein across the brain remain incompletely understood. Here, we use deep learning and artificial intelligence to discover a mathematical model for the progression of Alzheimer's disease using longitudinal tau positron emission tomography from the Alzheimer's Disease Neuroimaging Initiative database. Specifically, we integrate physics informed neural networks (PINNs) and symbolic regression to discover a reaction-diffusion type partial differential equation for tau protein misfolding and spreading. First, we demonstrate the potential of our model and parameter discovery on synthetic data. Then, we apply our method to discover the best model and parameters to explain tau imaging data from 46 individuals who are likely to develop Alzheimer's disease and 30 healthy controls. Our symbolic regression discovers different misfolding models f(c) for two groups, with a faster misfolding for the Alzheimer's group, f(c) = 0.23c 3 – 1.34c 2 + 1.11c, than for the healthy control group, f(c) = –c 3 + 0.62c 2 + 0.39c. Our results suggest that PINNs, supplemented by symbolic regression, can discover a reaction-diffusion type model to explain misfolded tau protein concentrations in Alzheimer's disease. Furthermore, we expect our study to be the starting point for a more holistic analysis to provide image-based technologies for early diagnosis, and ideally early treatment of neurodegeneration in Alzheimer's disease and possibly other misfolding-protein based neurodegenerative disorders.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Probabilistic Power Consumption Modeling for Commercial Buildings Using Logistic Regression Markov Chain

The total energy consumed by buildings takes up to 40% of U.S. energy use, in which a large portion is contributed by commercial buildings. Building performance optimization is desirable but requires accurate building models with uncertainties taken into account. This paper proposes a novel probabilistic modeling method using Logistic Regression Markov Chain (LRMC). The LRMC model enhances the performance of traditional Markov Chain (MC) models by adopting time-variant transition matrices calibrated using logistic regression with exogenous inputs. Compared with existing building models, the proposed model produces accurate multi-step modeling results with full probability distribution. The proposed probabilistic building model is tested using actual commercial building measurements and modeling performance is evaluated with two probabilisitc metrics. The results show that the LRMC model has higher accuracy than traditional MC model and Logistic Regression (LR) model in that it yields lower error scores under both evaluation metrics.

Building modeling↗
DOE OSTI2026

Extracting the Breakout Distance from the ECOT Trajectories: Gaussian Process Regression Approach

Enhanced Corner Turning (ECOT) experiments provide an important metric of performance of high explosive (HE) formulations. The breakout distance is a single scalar value that characterizes the corner turning efficiency of an HE. Extracting the breakout distance from the raw ECOT results, whether experimental or simulated, is a conceptually straightforward procedure which, however, is non-unique, especially in the presence of noise. More specifically, this procedure involves numerical smoothing and selecting particular values for parameters of this smoothing introduces human bias. In this work, we propose to use the Gaussian process regression to analyze ECOT results. This analysis involves the effective smoothing of the data, thus allowing for accurate extraction of the breakout distance. Most importantly, the parameters of this smoothing can be inferred from the ECOT data itself, rendering the approach effectively parameter-free and thus diminishing the human bias. An additional benefit of the Gaussian process regression, being a statistical inference method, is that not just the value of the breakout distance, but also its confidence interval can be extracted from the data. This report introduces the Gaussian process regression, as applied to ECOT, and demonstrates its usefulness by extracting the breakout distances for a selection of experimental and simulated data.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗