Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Linear Regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Geographical Insights into Suicide Mortality Through Spatial Machine Learning

Suicide mortality is a leading cause of death in the United States, with an upward trend that emphasizes its significance as a public health issue. Previous research has employed global models like ordinary least squares (OLS) regression and local models such as geographically weighted regression (GWR). While local models are useful for analyzing spatial variations in suicide mortality, they share limitations with traditional global models, particularly about their inability to handle multi-collinearity and non-linear relationships. Machine learning approaches, like random forests (RF), can address some of these limitations but often fail to account for spatial variability. This gap highlights the need for spatial ML models specifically designed to tackle suicide mortality. This research seeks to fill this void by using a geographically weighted random forest model (GWRF) to examine the associations between county-level suicide mortality in the U.S. from 2010 to 2020 and various social and environmental determinants of health. A key aspect of our methodology is disciplined feature selection, which reduces the pool of explanatory variables by about 90%. This refinement enhances the explanatory power of both global (R2 improved from 0.59 to 0.67) and local (R2 improved from 0.64 to 0.67) RF models while reducing their run times. An analysis of the importance scores for these selected features reveals that the drivers of suicide mortality vary by context. Thus, to effectively address regional disparities and inform targeted public health interventions, a holistic approach that incorporates multiple county-level characteristics is essential.

Lebakula, Viswadeep [ORNL] (ORCID:0000000152935914↗
DOE OSTI2023

Integrating Reservoirs into the Dissolved Organic Matter Versus Primary Production Paradigm: How Does Chlorophyll-$a$ Change Across Dissolved Organic Carbon Concentrations in Reservoirs?

Primary production in freshwater ecosystems is largely a function of light and nutrient availability, both of which have been changing in many lakes and reservoirs in response to anthropogenic pressures. Recent studies focusing on natural lakes have found a hump-shaped response of primary production (sometimes measured as chlorophyll-$a$) to dissolved organic matter (DOM, measured as dissolved organic carbon, DOC), which has both light-absorbing chromophoric properties and DOM-bound nutrients. We used the United States National Lakes Assessment dataset to integrate reservoirs into this paradigm in comparison with natural lakes and assessed the relative differences in the predicted response’s model structure, regression parameter values, and drivers of the chlorophyll-$a$ residuals. We found that chlorophyll-$a$ in reservoirs exhibited a hump-shaped response to DOC, while natural lakes from this dataset were better fit with a linear response, differing from previous studies focused on boreal lakes. Despite this, reservoirs had a greater maximum chlorophyll-a response compared to natural lakes in this study (45.5 versus 33.8 μg L -1 ), which occurred at a lower DOC concentration threshold (18.3 versus 26.4 mg L -1 ) when compared using quadratic models. Reservoirs had lower median light:nutrient values compared to natural lakes, and greater median surface area and total phosphorus (TP), that can all influence the light environment and the peak chlorophyll-a responses. In both reservoirs and natural lakes, chlorophyll-$a$ residuals were most strongly influenced by TP, where TP < 25-30 µg L -1 suppressed chlorophyll-a residuals and higher TP amplified them. Light:nutrient values were somewhat important predictors, and patterns with chlorophyll-$a$ residuals supported previous work showing low light:nutrient values amplified chlorophyll-$a$ responses and higher values suppressed them. In conclusion, quantifying the shape of the response of primary production to DOM quantity and quality as well as the drivers of the residuals, namely TP for lakes and reservoirs in this dataset, will be important for understanding the effects that changes in water quality may have on primary production and freshwater ecosystem processes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

MIONet: Learning Multiple-Input Operators via Tensor Product

As an emerging paradigm in scientific machine learning, neural operators aim to learn operators, via neural networks, that map between infinite-dimensional function spaces. Several neural operators have been recently developed. However, all the existing neural operators are only designed to learn operators defined on a single Banach space; i.e., the input of the operator is a single function. Here, for the first time, we study the operator regression via neural networks for multiple-input operators defined on the product of Banach spaces. We first prove a universal approximation theorem of continuous multiple-input operators. We also provide a detailed theoretical analysis including the approximation error, which provides guidance for the design of the network architecture. Based on our theory and a low-rank approximation, we propose a novel neural operator, MIONet, to learn multiple-input operators. MIONet consists of several branch nets for encoding the input functions and a trunk net for encoding the domain of the output function. Here, we demonstrate that MIONet can learn solution operators involving systems governed by ordinary and partial differential equations. In our computational examples, we also show that we can endow MIONet with prior knowledge of the underlying system, such as linearity and periodicity, to further improve accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Data Driven Optimization Framework for Volt-age Regulation in Distribution Systems

This letter proposes a data-driven optimization framework for voltage regulation problems to address the challenge of model inaccuracy and parameter varying. To achieve online voltage optimization, the recursive kernel regression and interior point methods are integrated. The IEEE 123-Bus system and EPRI Ckt5 feeder are selected to validate the effectiveness of the proposed data-driven optimization framework. The proposed method is also compared with a linear function based method.

Hong, Tianqi↗
DOE OSTI2022

Heuristic Computational Model for Predicting Lignin Solubility in Tailored Organic Solvents

Lignin is a random heteropolymer that has been extensively studied as a renewable source of aromatic precursors for high-value chemicals, biofuels, and bioplastics. A key challenge in lignin valorization is the structural and compositional heterogeneity of lignin feedstocks. Solvent-based approaches are commonly used to fractionate lignin to reduce this heterogeneity, but solvent selection can be challenging due to variability in lignin composition. In this work, we developed computational methods to predict good and poor organic solvents as a function of lignin composition. We analyzed 28 different linear pentamer structures, 18 from known libraries and 10 hypothetical polymers, and calculated their activity coefficients in 50 different organic solvents by using the conductor-like screening model for realistic solvents. We used these data to train a regression model that enabled the extensive investigation of the impact of solvent and monolignol compositions on predicted lignin solubility. The exhaustive exploration of solubility trends using model predictions revealed sets of solvents, identified using Kamlet–Taft parameters, that are predicted to promote lignin dissolution regardless of lignin composition. We further identified solvents expected to selectively isolate lignin fractions enriched in certain subunits. Furthermore, these results establish heuristic guidelines for solvent selection that can be used to tailor fractionation processes for lignin feedstocks of distinct composition or to design new processes that isolate fractions with higher proportions of selected subunits.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

A Survey of Constrained Gaussian Process: Approaches and Implementation Challenges

Gaussian process regression is a popular Bayesian framework for surrogate modeling of expensive data sources. As part of a larger effort in scientific machine learning, many recent works have incorporated physical constraints or other a priori information within Gaussian process regression to supplement limited data and regularize the behavior of the model. We provide an overview and survey of several classes of Gaussian process constraints, including positivity or bound constraints, monotonicity and convexity constraints, differential equation constraints provided by linear PDEs, and boundary condition constraints. We compare the strategies behind each approach as well as the differences in implementation, concluding with a discussion of the computational challenges introduced by constraints.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

[Re] Drivers of evapotranspiration from boreal wildfires

Computational reproducibility is a difficult challenge across science. I attempted to use R 3.6.1 to reproduce linear model fits, done originally using v2.6.0 for a 2009 paper on the drivers of large-scale forest evapotranspiration after wildfire. Model outputs were largely identical, aside from minor formatting changes, except for one–out of 12 total– regression in which the median residual value changed very slightly (in the sixth decimal place). I suggest that this essentially successful reproducibility is due to the relative simplicity of the script, its use of only base R functions, and R’s historically conservative approach to breaking changes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Data-Driven Mean-Corrected Recursive Estimation-Based Optimal DER Dispatch for Distribution System Voltage Control

Recent advances in smart inverters offer opportunities to mitigate adverse grid impacts caused by high penetrations of distributed photovoltaics (PV) in distribution grids, such as voltage violations. Here, this paper proposes a novel measurement-driven optimal power flow (OPF)-based distributed energy resource management system (DERMS) voltage regulation via recursive sensitivity estimation informed coordinated control of distributed PV inverters. The proposed approach leverages available grid and controllable DER measurements, eliminating reliance on system model information while being adaptive and robust to volatile operating conditions. A mean-corrected recursive ridge regression (MCRRR) algorithm is proposed for sensitivity estimation, continuously refining the sensitivity model through a closed-form solution. It effectively manages varying grid operating conditions, such as changes in power injections and topology reconfiguration, to facilitate a time-varying update of the Load Sensitivity Factors (LSF). The proposed approach is formulated as a linear programming (LP) problem and is thus scalable to larger-scale distribution systems. Its effectiveness and efficiency are demonstrated on a realistic distribution feeder with high PV penetrations in Southern California, USA.

14 SOLAR ENERGY↗
DOE OSTI2024

Modern deep neural networks for Direct Normal Irradiance forecasting: A classification approach

The escalating energy demand and the adverse environmental impacts of fossil-fuel use necessitate a shift towards cleaner and renewable alternatives. Concentrated Solar Power (CSP) technology emerges as a promising solution, offering a carbon-free alternative for power generation. The efficiency and profitability of CSP depend on the Direct Normal Irradiance (DNI) component of solar radiation; hence, accurate DNI forecasting can help optimize CSP plants’ operations and performance. The unpredictable nature of weather phenomena, particularly cloud cover, introduces uncertainty into DNI projections. Existing DNI forecasting models use meteorological factors, which are both challenging to estimate numerically over short prediction windows and expensive to model through data at a sufficiently high spatial and temporal resolution. This research addresses the challenge by presenting a novel approach that formulates DNI prediction as a multi-class classification problem, departing from conventional regression-based methods. The primary objective of this classification framework is to identify optimal periods aligning with specific operational thresholds for CSP plants, contributing to enhanced dispatch optimization strategies. We model the DNI classification problem using four advanced deep neural networks – rectified linear unit (ReLU) networks, 1D residual networks (ResNets), bidirectional long short-term memory (BiLSTM) networks, and transformers – achieving accuracies up to 93.5% without requiring meteorological parameters.

14 SOLAR ENERGY↗
DOE OSTI2021

Detection of the Metabolic Response to Drought Stress Using Hyperspectral Reflectance

Drought is the most important limitation on crop yield. Understanding and detecting drought stress in crops is vital for improving water use efficiency through effective breeding and management. Leaf reflectance spectroscopy offers a rapid, non-destructive alternative to traditional techniques for measuring plant traits involved in a drought response. We measured drought stress in six glasshouse-grown agronomic species using physiological, biochemical and spectral data. In contrast to physiological traits, leaf metabolite concentrations revealed drought stress before it was visible to the naked eye. We used full-spectrum leaf reflectance data to predict metabolite concentrations using partial least-squares regression, with validation R2 values of 0.49 to 0.87. We show for the first time that spectroscopy may be used for the quantitative estimation of proline and abscisic acid, demonstrating the first use of hyperspectral data to detect a phytohormone. We used linear discriminant analysis and partial least squares discriminant analysis to differentiate between droughted and watered plants based on measured traits (accuracy: 71%) and raw spectral data (66%). Finally, we validated our glasshouse-developed models in an independent field trial. We demonstrate that spectroscopy can detect drought stress via underlying biochemical changes, before visual differences occur, representing a powerful advance for measuring limitations on yield.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Learning-Based Demand Response in Grid Interactive Buildings via Gaussian Processes

This paper presents a predictive controller for a grid-interactive multi-zone building where the temperature dynamics are learned via Gaussian Process (GP) regression. We investigate the development of a learning-based predictive control with two main objectives: (i) continuously learn the temperature dynamics of the building based on data; and, (ii) use the learned dynamics to solve a multi-objective predictive control problem to guarantee occupants' comfort and energy efficiency during normal conditions and demand response events. We leverage the probabilistic non-parametric properties of GPs to estimate the (unknown) non-linear temperature dynamics of the building and to incorporate the uncertainty of those predictions in a multi-objective optimization problem. The GP-based predictive control is solved via a zero-order primal-dual projected-gradient algorithm. We evaluate numerically the performance of the proposed controller using a five-zone commercial building.

demand response↗
DOE OSTI2023

Tensor Decompositions for Count Data that Leverage Stochastic and Deterministic Optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the global maximum likelihood estimator from local minima. Simultaneously, a recent trend in theoretical computer science and numerical linear algebra leverages randomization to solve very large, hard problems. The typical approach is to use randomization for a fast approximation and determinism for refinement to yield effective algorithms with theoretical guarantees. Two popular algorithms for Poisson CPD reflect that emergent dichotomy: CP Alternating Poisson Regression is a deterministic algorithm and Generalized Canonical Polyadic decomposition makes use of stochastic algorithms in several variants. This work extends recent work to develop two new methods that leverage randomized and deterministic algorithms for improved accuracy and performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Permutation-adapted complete and independent basis for atomic cluster expansion descriptors

Atomic cluster expansion (ACE) methods provide a systematic way to describe particle local environments of arbitrary body order. For practical applications it is often required that the basis of cluster functions be symmetrized with respect to rotations and permutations. Existing methodologies yield sets of symmetrized functions that are over-complete. These methodologies thus require an additional numerical procedure, such as singular value decomposition (SVD), to eliminate redundant functions. In this work, it is shown that analytical linear relationships for subsets of cluster functions may be derived using recursion and permutation properties of generalized Wigner symbols. From these relationships, subsets (blocks) of cluster functions can be selected such that, within each block, functions are guaranteed to be linearly independent. It is conjectured that this block-wise independent set of permutation-adapted rotation and permutation invariant (PA-RPI) functions forms a complete, independent basis for ACE. Along with the first analytical proofs of block-wise linear dependence of ACE cluster functions and other theoretical arguments, numerical results are offered to demonstrate this. The utility of the method is demonstrated in the development of an ACE interatomic potential for tantalum. Using the new basis functions in combination with Bayesian compressive sensing sparse regression, some high degree descriptors are observed to persist and help achieve high-accuracy models.

Angular momentum↗
DOE OSTI2022

Sampling-based Sublinear Low-rank Matrix Arithmetic Framework for Dequantizing Quantum Machine Learning

We present an algorithmic framework for quantum-inspired classical algorithms on close-to-low-rank matrices, generalizing the series of results started by Tang’s breakthrough quantum-inspired algorithm for recommendation systems [STOC’19]. Motivated by quantum linear algebra algorithms and the quantum singular value transformation (SVT) framework of Gilyén et al. [STOC’19], we develop classical algorithms for SVT that run in time independent of input dimension, under suitable quantum-inspired sampling assumptions. Our results give compelling evidence that in the corresponding QRAM data structure input model, quantum SVT does not yield exponential quantum speedups. Since the quantum SVT framework generalizes essentially all known techniques for quantum linear algebra, our results, combined with sampling lemmas from previous work, suffice to generalize all prior results about dequantizing quantum machine learning algorithms. In particular, our classical SVT framework recovers and often improves the dequantization results on recommendation systems, principal component analysis, supervised clustering, support vector machines, low-rank regression, and semidefinite program solving. We also give additional dequantization results on low-rank Hamiltonian simulation and discriminant analysis. Our improvements come from identifying the key feature of the quantum-inspired input model that is at the core of all prior quantum-inspired results: ℓ 2 -norm sampling can approximate matrix products in time independent of their dimension. We reduce all our main results to this fact, making our exposition concise, self-contained, and intuitive.

Computer Science↗
DOE OSTI2023

Multilevel-in-width training for deep neural network regression

A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. Here, we draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Parsimonious Potential Energy Surface Expansions Using Dictionary Learning with Multipass Greedy Selection

Potential energy surfaces fit with basis set expansions have been shown to provide accurate representations of electronic energies and have enabled a variety of high-accuracy dynamics, kinetics, and spectroscopy applications. The number of terms in these expansions scales poorly with system size, a drawback that challenges their use for systems with more than similar to 10 atoms. A solution is presented here using dictionary learning. Subsets of the full set of conventional basis functions are optimized using a newly developed multipass greedy regression method inspired by forward and backward selection methods from the statistics, signal processing, and machine learning literatures. Here, the optimized representations have accuracies comparable to the full set but are 1 or more orders of magnitude smaller, and notably, the number of terms in the optimized multipass greedy expansions scales approximately linearly with the number of atoms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Unveiling the effect of composition on nuclear waste immobilization glasses’ durability by nonparametric machine learning

Abstract Ensuring the long-term chemical durability of glasses is critical for nuclear waste immobilization operations. Durable glasses usually undergo qualification for disposal based on their response to standardized tests such as the product consistency test or the vapor hydration test (VHT). The VHT uses elevated temperature and water vapor to accelerate glass alteration and the formation of secondary phases. Understanding the relationship between glass composition and VHT response is of fundamental and practical interest. However, this relationship is complex, non-linear, and sometimes fairly variable, posing challenges in identifying the distinct effect of individual oxides on VHT response. Here, we leverage a dataset comprising 654 Hanford low-activity waste (LAW) glasses across a wide compositional envelope and employ various machine learning techniques to explore this relationship. We find that Gaussian process regression (GPR), a nonparametric regression method, yields the highest predictive accuracy. By utilizing the trained model, we discern the influence of each oxide on the glasses’ VHT response. Moreover, we discuss the trade-off between underfitting and overfitting for extrapolating the material performance in the context of sparse and heterogeneous datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Exploring limitations in the induced polarization versus surface conductivity relationship in the case of wetland soils

Recent induced polarization studies suggest that the real part of surface conductivity (σ' surf ) scales linearly with the imaginary conductivity (σ'' = σ'' surf ) or normalized chargeability (Mn) for a range of soil types. The coefficients of this relationship l and l_Mn ( l = σ''/σ' surf or l_Mn = Mn/σ' surf ) allow the separation of the surface and electrolytic conductivities from the bulk conductivity. However, the dependence of these constants on varying soil physicochemical properties, including under unsaturated conditions, is yet to be assessed. Using estimates of σ' surf from 18 undisturbed soil samples from a restored wetland and σ'' measured over a frequency range of 0.01 Hz to 10 kHz, the σ' surf and σ'' were compared with the laboratory measurements of soil properties. Also, l and l_Mn were calculated for each soil sample and regressed them against the soil properties. We find an apparent dependence of l on soil texture, bulk density, organic matter, and moisture contents, with coefficients of determination (R 2 ) ranging from 0.5 to 0.65 at low frequencies (e.g., 1 Hz) but not at high frequencies (e.g., 936 Hz). This dependence of l on soil texture results from the insensitivity of σ'' at low frequency to σ' surf and, by implication, to the soil properties controlling σ' surf . In contrast, l_Mn indicates no correlation with the soil properties because Mn is linearly correlated with σ' surf and correlated with the soil properties controlling σ' surf . Finally, our results call for caution on the application of σ'' at a single frequency as a proxy of σ' surf because σ'' is not necessarily correlated with σ' surf across all soil types. Although using l_Mn derived from multifrequency measurements overcomes this limitation, field acquisition of spectral information (e.g., up to 1000 Hz) remains a challenge.

54 ENVIRONMENTAL SCIENCES↗