Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “simple linear regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A Provably Accurate Randomized Sampling Algorithm for Logistic Regression

In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When the number of observations greatly exceeds the number of predictor variables, we present a simple, randomized sampling-based algorithm for logistic regression problem that guarantees high-quality approximations to both the estimated probabilities and the overall discrepancy of the model. Our analysis builds upon two simple structural conditions that boil down to randomized matrix multiplication, a fundamental and well-understood primitive of randomized numerical linear algebra. We analyze the properties of estimated probabilities of logistic regression when leverage scores are used to sample observations, and prove that accurate approximations can be achieved with a sample whose size is much smaller than the total number of observations. To further validate our theoretical findings, we conduct comprehensive empirical evaluations. Overall, our work sheds light on the potential of using randomized sampling approaches to efficiently approximate the estimated probabilities in logistic regression, offering a practical and computationally efficient solution for large-scale datasets.

Chowdhury, Agniva↗
DOE OSTI2026

Physicochemical and Performance Characterization of Six Commercial Organic Solvent Nanofiltration Membranes

This work introduces a novel, gradient-free metamaterial design method based on Gaussian process regression to represent the density field of a unit cell. The dimension of the design space is determined by the covariance matrix dimension in the Gaussian process regression. We propose compressing this matrix using an autoencoder, enabling the decoder to generate the density field and effectively reduce the originally large design space to a lower-dimensional subspace. In this compressed space, we employ an active learning method, Bayesian Adaptive Direct Search (BADS), for efficient exploration of the design space. We demonstrate that for simple 2D designs aimed at maximizing unit cell stiffness, our method yields results comparable to those of standard topology optimization. Furthermore, we extend our approach to various mechanical problems, from linear elasticity to hyperelastic large deformation and elasto-plasticity under finite deformation, to 3D metamaterial design. This illustrates the method’s versatility and effectiveness across a range of applications.

Wu, Haoran↗
DOE OSTI2023

When less is more: How increasing the complexity of machine learning strategies for geothermal energy assessments may not lead toward better estimates

Previous moderate- and high-temperature geothermal resource assessments of the western United States utilized data-driven methods and expert decisions to estimate resource favorability. Although expert decisions can add confidence to the modeling process by ensuring reasonable models are employed, expert decisions also introduce human and, thereby, model bias. This bias can present a source of error that reduces the predictive performance of the models and confidence in the resulting resource estimates. Our study aims to develop robust data-driven methods with the goals of reducing bias and improving predictive ability. We present and compare nine favorability maps for geothermal resources in the western United States using data from the U.S. Geological Survey's 2008 geothermal resource assessment. Two favorability maps are created using the expert decision-dependent methods from the 2008 assessment (i.e., weight-of-evidence and logistic regression). With the same data, we then create six different favorability maps using logistic regression (without underlying expert decisions), XGBoost, and support-vector machines paired with two training strategies. The training strategies are customized to address the inherent challenges of applying machine learning to the geothermal training data, which have no negative examples and severe class imbalance. We also create another favorability map using an artificial neural network. We demonstrate that modern machine learning approaches can improve upon systems built with expert decisions. We also find that XGBoost, a non-linear algorithm, produces greater agreement with the 2008 results than linear logistic regression without expert decisions, because the expert decisions in the 2008 assessment rendered the otherwise linear approaches non-linear despite the fact that the 2008 assessment used only linear methods. The F1 scores for all approaches appear low (F1 score < 0.10), do not improve with increasing model complexity, and, therefore, indicate the fundamental limitations of the input features (i.e., training data). Until improved feature data are incorporated into the assessment process, simple non-linear algorithms (e.g., XGBoost) perform equally well or better than more complex methods (e.g., artificial neural networks) and remain easier to interpret.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Accurate parameterization of the kinetic energy functional

The absence of a reliable formulation of kinetic energy density functional has hindered the development of orbital free density functional theory. Using the data-aided learning paradigm, we propose a simple prescription to accurately model the kinetic energy density of any system. Our method relies on a dictionary of functional forms for local and nonlocal contributions which have been proposed in the literature and the appropriate coefficients are calculated via a linear regression framework. To model the nonlocal contributions, we explore two new nonlocal functionals - a functional that captures fluctuations in electronic density and a functional that incorporates gradient information. Since, the analytical functional forms of the kernels present in these nonlocal terms are not known from theory, we propose a basis function expansion to model these seemingly difficult nonlocal quantities. This allows us to easily reconstruct kernels for any system using only a few structures. The proposed method is able to learn kinetic energy densities and total kinetic energies of molecular and periodic systems, such as H 2 , LiH, LiF and a one-dimensional chain of 8 hydrogens using data from Kohn-Sham density functional theory calculations for only a few structures. For the ease of reproduction, codes used to generate the models are provided in the supporting materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Parameter inference from event ensembles and the top-quark mass

One of the key tasks of any particle collider is measurement. In practice, this is often done by fitting data to a simulation, which depends on many parameters. Sometimes, when the effects of varying different parameters are highly correlated, a large ensemble of data may be needed to resolve parameter-space degeneracies. An important example is measuring the top-quark mass, where other physical and unphysical parameters in the simulation must be profiled when fitting the top-quark mass parameter. We compare four different methodologies for top-quark mass measurement: a classical histogram fit similar to one commonly used in experiment augmented by soft-drop jet grooming; a 2D profile likelihood fit with a nuisance parameter; a machine-learning method called DCTR; and a linear regression approach, either using a least-squares fit or with a dense linearly-activated neural network. Despite the fact that individual events are totally uncorrelated, we find that the linear regression methods work most effectively when we input an ensemble of events sorted by mass, rather than training them on individual events. Although all methods provide robust extraction of the top-quark mass parameter, the linear network does marginally best and is remarkably simple. For the top study, we conclude that the Monte-Carlo-based uncertainty on current extractions of the top-quark mass from LHC data can be reduced significantly (by perhaps a factor of 2) using networks trained on sorted event ensembles. More generally, machine learning from ensembles for parameter estimation has broad potential for collider physics measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Machine learning elastic constants of multi-component alloys

The present manuscript explores application of machine learning methods for determining elastic constants and other derived mechanical properties of multi-component alloys. Here, a number of machine learning models, including linear regression, neural network and random forest based models, are trained and tested on a dataset of binary alloys generated using density functional theory (DFT) calculations and spanning over a large number of elemental species in the periodic table. Starting with a wide range of simple and easily accessible compositionally-averaged elemental features, a correlation-based feature selection strategy was used to systematically down-select a set of most relevant features towards the prediction of the elasticity tensor components. The true predictive performance and the associated uncertainties of the models were established by testing on unseen data and bootstrapping, respectively. A single and pair-wise feature partial dependence analysis was performed to visualize the average property trends in the multi-dimensional feature space in order to further understand the achieved predictive performance. The utility of the trained model is further demonstrated by obtaining sufficiently accurate yet highly efficient approximations for bulk modulus, Young’s modulus, shear modulus and Poisson’s ratio for alloys beyond the binary space (i.e., two-component alloys) on which the model was originally trained. More importantly, we test and validate the predictive performance of the developed model directly against the experimentally measured elastic constants of technologically relevant multi-component alloys (such as, Ni- and Ti-based alloys). Finally, utility of such a data-enabled route is demonstrated by predicting the possible range of various elastic properties for vast composition space available within the five component Ni-Cr-Fe-Mo-W alloy system in a high-throughput manner.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Association Between Injection and Microseismicity in Geothermal Fields With Multiple Wells: Data-Driven Modeling of Rotokawa, New Zealand, and Húsmúli, Iceland

Understanding injection-induced microseismicity in geothermal systems can provide insight into reservoir connectedness. However, fault and reservoir complexity are difficult to represent in simple analytical models, which makes it difficult to discern clear relationships from incidental associations. Here, we have used data-driven models to study how fluid injection and microseismicity are related in the Rotokawa (New Zealand) and Hellisheiði (Iceland) geothermal fields. We tested two classes of model: (a) lagged linear regression of seismicity rate as a function of well injection rates; and (b) systematic extraction of injection time series features that are then evaluated for associations with the seismicity. These models allowed us to determine which wells had the greatest correlation with microseismicity and to explain this association in a reservoir context. Finally, exploring different data types and transformations, we were unable to establish a link between rapid changes in injection rate and seismicity spikes, as suggested by some theoretical models.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Temporal covariance structure of multi-spectral phenotypes and their predictive ability for end-of-season traits in maize

Vegetation indices (VIs) derived from multi-spectral imaging (MSI) platforms can be used to study properties of crop canopy, providing non-destructive phenotypes that could be used to better understand growth curves throughout the growing season. To investigate the amount of variation present in several VIs and their relationship with important end-of-season traits, genetic and residual (co)variances for VIs, grain yield and moisture were estimated using data collected from maize hybrid trials. The VIs considered were Normalized Difference Vegetation Index (NDVI), Green NDVI, Red Edge NDVI, Soil-Adjusted Vegetation Index, Enhanced Vegetation Index and simple Ratio of Near Infrared to Red (Red) reflectance. Genetic correlations of VIs with grain yield and moisture were used to fit multi-trait models for prediction of end-of-season traits and evaluated using within site/year cross-validation. To explore alternatives to fitting multiple phenotypes from MSI, random regression models with linear splines were fit using data collected in 2016 and 2017. Heritability estimates ranging from (0.10 to 0.82) were observed, indicating that there exists considerable amount of genetic variation in these VIs. Furthermore, strong genetic and residual correlations of the VIs, NDVI and NDRE, with grain yield and moisture were found. Considerable increases in prediction accuracy were observed from the multi-trait model when using NDVI and NDRE as a secondary trait. Finally, random regression with a linear spline function shows potential to be used as an alternative to mixed models to fit VIs from multiple time points.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

pnnl/simple-building-calculator

The Simple Building Calculator is a web application that estimates annual energy use using regression models that have been fit to simulation results for the DOE Commercial Prototype Building Models. The tool is a single-page application that will allow a user to rapidly analyze the impact of energy efficiency measures and design options on building performance. As it uses linear regression models rather than more complicated machine learning models or physics based simulation, the tool can deploy easily and run client-side.

Xu, Weili↗
DOE OSTI2021

Removing the Effects of Tropical Dynamics from North Pacific Climate Variability

Teleconnections from the Tropics energize variations of the North Pacific climate, but detailed diagnosis of this relationship has proven difficult. Simple univariate methods, such as regression on El Niño-Southern Oscillation (ENSO) indices, may be inadequate since the key dynamical processes involved -- including ENSO diversity in the Tropics, re-emergence of mixed layer thermal anomalies, and oceanic Rossby wave propagation in the North Pacific -- have a variety of overlapping spatial and temporal scales. Here we use a multivariate Linear Inverse Model to quantify tropical and extra-tropical multi-scale dynamical contributions to North Pacific variability, in both observations and CMIP6 models. In observations, we find that the Tropics are responsible for almost half of the seasonal variance, and almost three quarters of the decadal variance, along the North American coast and within the subtropical front region northwest of Hawaii. SST anomalies that are generated by local dynamics within the Northeast Pacific have much shorter time scales, consistent with transient weather forcing by Aleutian low anomalies. Variability within the Kuroshio-Oyashio Extension (KOE) region is considerably less impacted by the Tropics, on all time scales. Consequently, without tropical forcing the dominant pattern of North Pacific variability would be a KOE pattern, rather than the Pacific Decadal Oscillation (PDO). In contrast to observations, most CMIP6 historical simulations produce North Pacific variability that maximizes in the KOE region, with amplitude significantly higher than observed. Correspondingly, the simulated North Pacific in all CMIP6 models is shown to be relatively insensitive to the Tropics, with a dominant spatial pattern generally resembling the KOE pattern, not the PDO.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Versatile recognition of graphene layers from optical images under controlled illumination through green channel correlation method

In this study, a simple yet versatile method is proposed for identifying the number of exfoliated graphene layers transferred on an oxide substrate from optical images, utilizing a limited number of input images for training, paired with a more traditional number of a few thousand well-published Github images for testing and predicting. Two thresholding approaches, namely the standard deviation-based approach and the linear regression-based approach, were employed in this study. The method specifically leverages the red, green, and blue color channels of image pixels and creates a correlation between the green channel of the background and the green channel of the various layers of graphene. This method proves to be a feasible alternative to deep learning-based graphene recognition and traditional microscopic analysis. The proposed methodology performs well under conditions where the effect of surrounding light on the graphene-on-oxide sample is minimum and allows rapid identification of the various graphene layers. Here, the study additionally addresses the functionality of the proposed methodology with nonhomogeneous lighting conditions, showcasing successful prediction of graphene layers from images that are lower in quality compared to typically published in literature. In all, the proposed methodology opens up the possibility for the non-destructive identification of graphene layers from optical images by utilizing a new and versatile method that is quick, inexpensive, and works well with fewer images that are not necessarily of high quality.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Data-Driven Supervised Dimension Reduction for Scientific Discovery (LDRD QTI Report)

This report summarizes the findings of a four months FY24 Advanced Science & Technology (AS&T) LDRD Quick Targeted Investigation (QTI) project focused on the exploration of supervised dimension reduction approaches based on autoencoders. Autoencoders have been extensively employed in literature for unsupervised learning tasks, however, their use for supervised regression tasks, which are common within scientific applications, has been limited. Motivated by linear dimension reduction strategies like Active Subspaces and Adaptive Basis, we explored the possibility of employing autoencoders to discover a non-linear manifold able to represent the original function in fewer dimensions. In this report, we discuss a neural network architecture and we perform a numerical campaign on several problems ranging from simple two-dimensional functions to a model problem for magnetohydrodynamics in five dimensions. In our preliminary results, we show that the proposed approach is found to be superior to linear dimension reduction strategies in representing the target function even with a single latent variable.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Novel method for accurately estimating membrane transport properties and mass transfer coefficients in reverse osmosis

Here, we present a simple and robust method to simultaneously characterize the water and salt permeability (A, B) of reverse osmosis (RO) membranes and mass transfer coefficient (k) in membrane modules. The proposed methodology comprises a set of RO experiments performed at different operating pressures or stages. The measured water and salt fluxes in each stage are simultaneously fitted to the RO transport equations by performing a non-linear regression, using A, B, and k as regression parameters. We first perform a systematic accuracy analysis of the proposed method across the full operational range of RO. The assessment shows that the method accuracy is substantially higher than current methods and increases with number of experimental stages and driving forces. This assessment is used to inform the design of an experimental protocol that minimizes errors in estimated A, B, and k. We then evaluate two commercial RO membranes following the new protocol. For both membranes, A and B parameters decrease by 17% and 15% from the dilute solution to seawater concentrations, whereas the k parameter remains constant. Our study demonstrates that the proposed method, informed by data-driven experimental designs, provides a new approach for accurately characterizing transport phenomenon in membrane processes with feeds of less than 100 g/L total dissolved solids.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Operational-based annual energy production uncertainty: are its components actually uncorrelated?

Calculations of annual energy production (AEP) from a wind power plant – whether based on preconstruction or operational data – are critical for wind plant financial transactions. The uncertainty in the AEP calculation is especially important in quantifying risk and is a key factor in determining financing terms. A popular industry practice is to assume that different uncertainty components within an AEP calculation are uncorrelated and can therefore be combined as the sum of their squares. We assess the practical validity of this assumption for operational-based uncertainty by performing operational AEP estimates for more than 470 wind plants in the United States, mostly in simple terrain. We apply a Monte Carlo approach to quantify uncertainty in five categories: revenue meter data, wind speed data, regression relationship between density-corrected wind speed (from reanalysis data) and measured wind power, length of long-term-correction data set, and future interannual variability. We identify correlations between categories by comparing the results across all 470 wind plants. We observe a positive correlation between interannual variability and the linearized long-term correction; a negative correlation between wind resource interannual variability and linear regression; and a positive correlation between reference wind speed uncertainty and linear regression. Then, we contrast total operational AEP uncertainty values calculated by omitting and considering correlations between the uncertainty components. We quantify that ignoring these correlations leads to an underestimation of total AEP uncertainty of, on average, 0.1% and as large as 0.5% for specific sites. Although these are not large increases, these would still impact wind plant financing rates; further, we expect these values to increase for wind plants in complex terrain. Based on these results, we conclude that correlations between the identified uncertainty components should be considered when computing the total AEP uncertainty.

17 WIND ENERGY↗
DOE OSTI2022

Error quantification of the Arrhenius blending rule for viscosity of hydrocarbon mixtures

Six hundred and seventy-five measurements of dynamic viscosity and density have been used to assess the prediction error of the Arrhenius blending rule for kinematic viscosity of hydrocarbon mixtures. Major trends within the data show that mixture complexity–binary to hundreds of components—and temperature are more important determinants of prediction error than differences in molecular size or hydrogen saturation between the components of the mixtures. Over the range evaluated, no correlation between prediction error and mole fractions was observed, suggesting the log of viscosity truly is linear in mole fraction, as indicated by the Arrhenius blending rule. Mixture complexity and temperature also impact molar volume and its prediction. However, a linear regression between the two model errors explains less than 20% of the observed variation, indicating that mixture viscosity and/or molar volume are not linear with respect to temperature and/or mixture complexity. Extensive discussion of the intermolecular forces and the geometric arrangement of molecules and vacancies in liquids, which ultimately determines its viscosity, is brought into context with the implicit approximations behind the Arrhenius blending rule. The complexity of this physics is not compatible with a simple algebraic correction to the model. However, sufficient data is now available to determine confidence intervals around the prediction of fuel viscosity based on its component mole fractions and viscosities. At -40°C, when all identified components are pure molecules the modeling error is 13.2% of the predicted (nominal) viscosity times the root mean square of the component mole fractions.

10 SYNTHETIC FUELS↗
DOE OSTI2024

Predicting seismic amplitudes with machine learning

The accurate estimation of seismic wave amplitude is vital to precisely determine the yield, magnitude, and event discrimination possible for a given network – a critical element in nuclear explosion monitoring. This task is complicated by several factors, including but not limited to radiation pattern, scattering effects, and crustal variations, which can lead to the attenuation or amplification of amplitude along a given raypath. In this report, we explore the novel application of machine learning to the task of seismic amplitude estimation by training a simple Artificial Neural Network (ANN) on an S-wave amplitude dataset from Lai et al. (2019). Attributes from this dataset used as input to the ANN included event-station distances, station locations (latitude, longitude), event locations (latitude, longitude), event depths, event magnitudes, radiation patterns, signal-to noise ratio (SNR) measurements (average-amplitude, peak-to-trough, maximum peak), and signal periods. We find that the trained ANN predicts S-wave amplitudes with a modest tendency toward underestimating the actual values, as indicated by a linear regression between predicted and actual data (slope: 0.892, intercept: -0.651). These results suggest that an ANN can perform this task, with potential for significant improvements through improved datasets, architectures, and parameter tuning.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2022

How Generalizable is a Machine-Learning Approach for Modeling Hub-Height Turbulence Intensity?

Hub-height turbulence intensity is essential for a variety of wind energy applications. However, simulating it is a challenging task. Simple analytical models have been proposed in the literature, but they all come with significant limitations. Even state-of-the-art numerical weather prediction models, such as the Weather Research and Forecasting model, currently struggle to predict hub-height turbulence intensity. Here, we propose a machine-learning-based approach to predict hub-height turbulence intensity from other hub-height and ground-level atmospheric measurements, using observations from the Perdigao field campaign and the Southern Great Plains atmospheric observatory. We consider a random forest regression model, which we validate first at the site used for training and then under a more robust round-robin approach, and compare its performance to a multivariate linear regression. The random forest successfully outperforms the linear regression in modeling hub-height turbulence intensity, with a normalized root-mean-square error as low as 0.014 when using 30-minute average data. In order to achieve such low root-mean-square error values, the knowledge of hub-height turbulence kinetic energy (which can instead be modeled in the Weather Research and Forecasting model) is needed. Interestingly, we find that the performance of the random forest generalizes well when considering a round-robin validation (i.e., when the algorithm is trained at one site such as Perdigao or Southern Great Plains) and then applied to model hub-height turbulence intensity at the other location.

17 WIND ENERGY↗
DOE OSTI2020

A Comfort Model Simplification for Tight Integration with Grid Service Optimizations

Localized generation, storage, and intelligent power electronics are increasingly being integrated into building. The excess storage, thermal, and generation capacity can be used to develop new markets for ancillary services such as peak reduction, voltage and frequency support, or load flattening. Widespread participation in these markets will make the power grid more resilient. However, widespread participation in these markets will require a simple and intuitive control over the excess capacity by the building occupants. Most importantly for adopting these technologies, participation in these markets should not require sacrificing occupant comfort. Many of the proposed control and optimization technologies for supplying the ancillary services do not explicitly incorporate occupant comfort into their models and when they do, generally utilize a rigid constraint on the indoor air temperature set a priori. This reduces flexibility in using excess thermal capacity. It also cannot take into account uncertainties in local factors that affect thermal comfort such as clothing insulation or relative humidity. In this paper we present and justify some assumptions that simplify the standard predictors of thermal comfort. We also develop a regression model of the thermal comfort that is linear with respect to the indoor air temperature. Next we develop the concept of thermal comfort variation that eliminates the dependence of the thermal comfort on uncertain thermal comfort factors. Then, we present a quadratic program utilizing the thermal comfort variation in both the objective function and as a constraint. Finally, we show the results of a time-of-use cost optimization that utilizes the simplified thermal comfort model.

Melin, Alexander↗