Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “nonlinear regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1979

A regression technique for evaluation and quantification for water quality parameters from remote sensing data

The objective of this paper is to define optical physics and/or environmental conditions under which the linear multiple-regression should be applicable. An investigation of the signal-response equations is conducted and the concept is tested by application to actual remote sensing data from a laboratory experiment performed under controlled conditions. Investigation of the signal-response equations shows that the exact solution for a number of optical physics conditions is of the same form as a linearized multiple-regression equation, even if nonlinear contributions from surface reflections, atmospheric constituents, or other water pollutants are included. Limitations on achieving this type of solution are defined.

Whitlock, C. H.↗
NASA NTRS1979

A regression technique for evaluation and quantification for water quality parameters from remote sensing data

The paper attempts to define optical physics and/or environmental conditions under which the linear multiple-regression should be applicable. It is reported that investigation of the signal response shows that the exact solution for a number of optical physics conditions is of the same form as a linearized multiple-regression equation, even if nonlinear contributions from surface reflections, atmospheric constituents, or other water pollutants are included. Limitations on achieving this type of solution are defined. Laboratory data are used to demonstrate that the technique is applicable to water mixtures which contain constituents with both linear and nonlinear radiance gradients. Finally, it is concluded that instrument noise, ground-truth placement, and time lapse between remote sensor overpass and water sample operations are serious barriers to successful use of the technique.

Whitlock, C. H.↗
NASA NTRS1981

Estimation of airplane stability and control derivatives at high angles of attack

A technique for extracting approximate aircraft stability and control derivatives over a broad range of angle of attack from a single large amplitude maneuver which encompasses that range is presented. The results from the application of this technique to flight data are shown to agree well with derivatives extracted from several maneuvers which are more restricted in range of angle of attack. The technique consists of dividing the data from the large maneuver into bins that are a function of angle of attack. The binned data is then analyzed with a modified stepwise regression. The identification of nonlinear terms in the aerodynamic model is shown to discern regions of rapid change in aircraft behavior.

Batterson, J. G.↗
NASA NTRS1990

Constrained optimization using design of experiment surfaces

An algorithm for solving constrained optimization problems is presented. First, design of experiment techniques are used to survey the design space. After evaluating the objective and constraint functions, as specified by Taguchi orthogonal arrays, analytical models of these functions are generated using a least-squares regression analysis. Next, a nonlinear programming package is used to optimize the analytical model. Based on the optimization information, the design space is reduced so as to close in around the minimum, and the entire procedure is repeated until convergence. An important feature of the algorithm is that function gradients are not required; therefore, for problems in which gradients would have to be estimated using finite-differences the number of function evaluations required for the optimization is significantly reduced, when compared with traditional nonlinear programming techniques. In addition, there is no requirement that the gradients must be smooth and continuous.

Bolt, Marvin Vance↗
NASA NTRS1995

Parameter identification and modeling of longitudinal aerodynamics

Using a comprehensive flight test database and a parameter identification software program produced at NASA Ames Research Center, a math model of the longitudinal aerodynamics of the Harrier aircraft was formulated. The identification program employed the equation error method using multiple linear regression to estimate the nonlinear parameters. The formulated math model structure adhered closely to aerodynamic and stability/control theory, particularly with regard to compressibility and dynamic manoeuvring. Validation was accomplished by using a three degree-of-freedom nonlinear flight simulator with pilot inputs from flight test data. The simulation models agreed quite well with the measured states. It is important to note that the flight test data used for the validation of the model was not used in the model identification.

Aksteter, J. W.↗
DOE OSTI2019

Development and Calibration of an Online Energy Model for AHU Fan

The model development is necessary for the study of the energy consumption of Heating, Ventilation, and Air Conditioning (HVAC) systems. To predict the HVAC energy consumption accurately, one needs to model the individual HVAC components either from the measured data or based on the knowledge of the underlying physical phenomenon. Online model characterization is critical for improving the performance of real-time model-based fault detection and diagnosis (FDD) strategies. For HVAC control, models can be used to optimize the supervisory and local feedback control strategies to improve the energy consumption efficiency, or for providing ancillary services to the grid. It has been reported that, fans in HVAC systems of commercial buildings alone can provide substantial frequency regulation service, with little change in the indoor environment. In this paper, a real-time data-driven Air Handling Unit (AHU) fan model was developed based on recursive multi regression model. A generic nonlinear polynomial model has been studied to cover scenarios with different combinations of measurement variables, variable orders as well as different training and prediction horizons. Typical measurements including static pressure, mass flow rate, and damper positions are utilized as inputs to model the power consumption of the fan. The developed models have been validated both with simulation data from EnergyPlus-Dymola co-simulation model and with field measurement data for small to medium commercial buildings. The validation results show that the online model proposed can provide an effective prediction of the AHU fan power consumption.

Dong, Jin↗
DOE OSTI2025

Projection-based multifidelity linear regression for data-scarce applications

Surrogate modeling for systems with high-dimensional quantities of interest remains challenging, particularly when training data are costly to acquire. This work develops multifidelity methods for multiple-input multiple-output linear regression targeting data-limited applications with high-dimensional outputs. Multifidelity methods integrate many inexpensive low-fidelity model evaluations with limited, costly high-fidelity evaluations. We introduce two projection-based multifidelity linear regression approaches with linear and nonlinear features that leverage principal component basis vectors for dimensionality reduction and combine multifidelity data through: (i) a direct data augmentation using low-fidelity data, and (ii) a data augmentation incorporating explicit linear corrections between low-fidelity and high-fidelity data. The data augmentation approaches combine high-fidelity and low-fidelity data into a unified training set and train the linear regression model through weighted least squares with fidelity-specific weights. We introduce a proximity-based weighting scheme with automatic weight selection strategy through cross-validation. Here, the proposed multifidelity linear regression methods are demonstrated on approximating the surface pressure field of a hypersonic vehicle in flight and the temperature field on an aircraft disc braking system. In an ultra low-data regime of no more than twelve high-fidelity samples, multifidelity linear regression achieves approximately 2% – 12% improvement in median accuracy and a higher R 2 score relative to single-fidelity methods at comparable computational cost.

data augmentation↗
DOE OSTI2022

Nonlinear gyrokinetic predictions of SPARC burning plasma profiles enabled by surrogate modeling

Multi-channel, nonlinear predictions of core temperature and density profiles are performed for the SPARC tokamak accounting for both kinetic neoclassical and fully nonlinear gyro-kinetic turbulent fluxes. A series of flux-tube, nonlinear, electromagnetic simulations using the CGYRO code with six gyrokinetic species are coupled to a nonlinear optimizer using Gaussian process regression techniques. The simultaneous evolution of energy sources, including alpha heat, radiation, and energy exchange, coupled with these high fidelity models and techniques, leads to a converged solution in electron temperature, ion temperature and electron density channels with a minimal number of expensive gyrokinetic simulations without compromising accuracy.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2017

Formation enthalpies for transition metal alloys using machine learning

The enthalpy of formation is an important thermodynamic property. Developing fast and accurate methods for its prediction is of practical interest in a variety of applications. Material informatics techniques based on machine learning have recently been introduced in the literature as an inexpensive means of exploiting materials data, and can be used to examine a variety of thermodynamics properties. Here we investigate the use of such machine learning tools for predicting the formation enthalpies of binary intermetallic compounds that contain at least one transition metal. We consider certain easily available properties of the constituting elements complemented by some basic properties of the compounds, to predict the formation enthalpies. We show how choosing these properties (input features) based on a literature study (using prior physics knowledge) seems to outperform machine learning based feature selection methods such as sensitivity analysis and LASSO (least absolute shrinkage and selection operator) based methods. A nonlinear kernel based support vector regression method is employed to perform the predictions. The predictive ability of our model is illustrated via several experiments on a dataset containing 648 binary alloys. We train and validate the model using the formation enthalpies calculated using a model by Miedema, which is a popular semiempirical model used for the prediction of formation enthalpies of metal alloys.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Probabilistic Data Association for Orbital-Element Estimation Using Multistage Expectation–Maximization

Tracking space objects is important for managing space traffic and predicting collisions, but is difficult in part due to data association and orbit model uncertainty. Expectation–maximization (EM) is a commonly used tracking method that has not been widely considered for tracking space objects. The technique consists of iteratively computing data association probabilities with a set of current element estimates, and updating estimates of the elements by solving a nonlinear weighted least-squares regression problem where the weights are the data association probabilities. In this paper, we demonstrate the use of EM for probabilistic data association and orbital-element estimation by applying the technique to simulated data from two angles-only tracking scenarios. In both scenarios, EM provides correct data associations and accurate maximum likelihood estimates of orbital elements. One scenario considers tracking a single object in clutter and quantifies the improvement of the orbital-element estimates and data associations as the detection probability increases. However, standard application of EM requires knowing the number of objects or may fail when a large number of objects are present. To address these issues, this paper employs a multistage version of EM that is applicable when there are a large and possibly unknown number of objects.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Testing the reliability of interpretable neural networks in geoscience using the Madden–Julian oscillation

Abstract. We test the reliability of two neural network interpretation techniques, backward optimization and layerwise relevance propagation, within geoscientific applications by applying them to a commonly studied geophysical phenomenon, the Madden–Julian oscillation. The Madden–Julian oscillation is a multi-scale pattern within the tropical atmosphere that has been extensively studied over the past decades, which makes it an ideal test case to ensure the interpretability methods can recover the current state of knowledge regarding its spatial structure. The neural networks can, indeed, reproduce the current state of knowledge and can also provide new insights into the seasonality of the Madden–Julian oscillation and its relationships with atmospheric state variables. The neural network identifies the phase of the Madden–Julian oscillation twice as accurately as a linear regression approach, which means that nonlinearities used by the neural network are important to the structure of the Madden–Julian oscillation. Interpretations of the neural network show that it accurately captures the spatial structures of the Madden–Julian oscillation, suggest that the nonlinearities of the Madden–Julian oscillation are manifested through the uniqueness of each event, and offer physically meaningful insights into its relationship with atmospheric state variables. We also use the interpretations to identify the seasonality of the Madden–Julian oscillation and find that the conventionally defined extended seasons should be shifted later by 1 month. More generally, this study suggests that neural networks can be reliably interpreted for geoscientific applications and may thereby serve as a dependable method for testing geoscientific hypotheses.

58 GEOSCIENCES↗
NASA NTRS1992

The European Southern Observatory-MIDAS table file system

The new and substantially upgraded version of the Table File System in MIDAS is presented as a scientific database system. MIDAS applications for performing database operations on tables are discussed, for instance, the exchange of the data to and from the TFS, the selection of objects, the uncertainty joins across tables, and the graphical representation of data. This upgraded version of the TFS is a full implementation of the binary table extension of the FITS format; in addition, it also supports arrays of strings. Different storage strategies for optimal access of very large data sets are implemented and are addressed in detail. As a simple relational database, the TFS may be used for the management of personal data files. This opens the way to intelligent pipeline processing of large amounts of data. One of the key features of the Table File System is to provide also an extensive set of tools for the analysis of the final results of a reduction process. Column operations using standard and special mathematical functions as well as statistical distributions can be carried out; commands for linear regression and model fitting using nonlinear least square methods and user-defined functions are available. Finally, statistical tests of hypothesis and multivariate methods can also operate on tables.

Peron, M.↗
DOE OSTI2023

Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristics

Here, in this paper, we consider the problem of learning a regression function without assuming its functional form. This problem is referred to as symbolic regression. An expression tree is typically used to represent a solution function, which is determined by assigning operators and operands to the nodes. Cozad and Sahinidis propose a nonconvex mixed-integer nonlinear program (MINLP), in which binary variables are used to assign operators and nonlinear expressions are used to propagate data values through nonlinear operators, such as square, square root, and exponential. We extend this formulation by adding new cuts that improve the solution of this challenging MINLP. We also propose a heuristic that iteratively builds an expression tree by solving a restricted MINLP. We perform computational experiments and compare our approach with a mixed-integer program–based method and a neural network–based method from the literature.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Machine‐learning‐based predictive control of nonlinear processes. Part II: Computational implementation

Abstract Machine learning is receiving more attention in classical engineering fields, and in particular, recurrent neural networks (RNNs) coupled with ensemble regression tools have demonstrated the capability of modeling nonlinear dynamic processes. In Part I of this two‐article series, the Lyapunov‐based model predictive control (LMPC) method using a single RNN model and an ensemble of RNN models, respectively, was rigorously developed for a general class of nonlinear systems. In the present article, computational implementation issues of this new control method ranging from training of the RNN models, ensemble regression of the RNN models, and parallel computation for accelerating the real‐time model calculations are addressed. Furthermore, a chemical reactor example is used to demonstrate the implementation and effectiveness of these machine‐learning tools in LMPC as well as compare them with standard state‐space model identification tools.

Wu, Zhe↗
DOE OSTI2017

Gradient-based Optimization for Regression in the Functional Tensor-Train Format

We consider the task of low-multilinear-rank functional regression, i.e., learning a low-rank parametric representation of functions from scattered real-valued data. Our first contribution is the development and analysis of gradient-based optimization procedures, including stochastic gradient descent and quasi-Newton methods, for learning the parameters of a functional tensor-train (FT) that yields improved accuracy over standard alternating least squares methods. The functional tensor-train uses the tensor-train (TT) representation of low-rank arrays as an ansatz for a class of low-multilinear-rank functions. The FT is represented by a set of matrix-valued functions that contain a set of univariate functions, and the regression task is to learn the parameters of these univariate functions. Our second contribution demonstrates that using nonlinearly parameterized univariate functions, e.g., symmetric kernels with moving centers, within each core can outperform the standard approach of using a linear expansion of basis functions. Our final contributions are new rank adaptation and group-sparsity regularization procedures to minimize overfitting. We use several benchmark problems to demonstrate at least an order of magnitude lower accuracy with gradient-based optimization methods than ALS in the low-sample number regime. We also demonstrate an order of magnitude reduction in accuracy on a test problem resulting from nonlinear parameterizations over linear parameterizations. Finally we compare regression performance with 22 other nonparametric and parametric regression methods on 10 real-world data sets. We achieve top-five accuracy for seven of the data sets and best accuracy for two of the data sets. These rankings are the best amongst parametric models and competetive with the best non-parametric methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A data-driven approach to construct a quantitative relationship between microstructural features of fatigue cracks and contact acoustic nonlinearity

This study demonstrates the feasibility of a data-driven approach to construct a quantitative relationship between nonlinear acoustic parameters and microstructural features of contact interfaces. The near-surface nonlinearity is measured using dynamic acousto-elastic testing (DAET) with a surface wave probe, while the microstructural features are extracted from scanning electron microscopy (SEM) images of fatigue cracks. Four aluminum alloy samples, each having a fatigue crack are prepared. Six local nonlinearity parameters are measured at different locations along the crack propagation direction. A total of 40 local measurements are acquired. A principal component analysis (PCA) reveals that all six nonlinearity parameters are correlated and hence can be replaced by one principal component (PC). Fifteen crack micro-geometrical features at each measurement point were extracted from the SEM images. Regression analysis is used to relate the PC of the nonlinearity parameters to the microstructural features at the crack interface. We compare three regression models that take variable selection into account: stepwise multiple linear regression (MLR), stepwise principal component regression (PCR), and least absolute shrinkage and selection operator (LASSO). Despite having different principles, the three predictive models identify two features as the most significant in predicting the interface nonlinearity: the crack aperture (opening) distribution and the distance to the crack tip. The differences between the three models and the physical interpretation of the data-driven predictions are discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Effects of Omitting Non-confounding Predictors From General Relative-Risk Models for Binary Outcomes

The effects, in terms of bias and precision, of omitting non-confounding predictive covariates from generalized linear models have been well studied, and it is known that such omission results in attenuation bias but increased precision with logistic regression. However, many epidemiologic risk analyses utilize alternative models that are not based on a linear predictor, and the effect of omitting non-confounding predictive covariates from such models has not been characterized. We employed simulation to study the effects on risk estimation of omitting non-confounding predictive covariates from an excess relative risk (ERR) model and a general additive-multiplicative relative-risk mixture model for binary outcome data in a case-control setting. We also compared the results to the effects with ordinary logistic regression. For these commonly employed alternative relative-risk models, the bias was similar to that with logistic regression when the risk was small. More generally, the bias and standard error of the risk-parameter estimates demonstrated patterns that are similar to those with logistic regression, but with greater magnitude depending on the true value of the risk. The magnitude of bias and standard error had little relation to study size or underlying disease prevalence. Prior conclusions regarding omitted covariates in logistic regression models can be qualitatively applied to the ERR and the general additive-multiplicative relative-risk mixture model without substantial change. Quantitatively, however, these alternative models may have slightly greater omitted-covariate bias, depending on the magnitude of the true risk being estimated.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

A comprehensive and fair comparison of two neural operators (with practical extensions) based on $\mathrm{FAIR}$ data

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

42 ENGINEERING↗