SEARCH · Engineering Papers

Results for “Sparse regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1976

A LANDSAT study of ephemeral and perennial rangeland vegetation and soils

The author has identified the following significant results. Several methods of computer processing were applied to LANDSAT data for mapping vegetation characteristics of perennial rangeland in Montana and ephemeral rangeland in Arizona. The choice of optimal processing technique was dependent on prescribed mapping and site condition. Single channel level slicing and ratioing of channels were used for simple enhancement. Predictive models for mapping percent vegetation cover based on data from field spectra and LANDSAT data were generated by multiple linear regression of six unique LANDSAT spectral ratios. Ratio gating logic and maximum likelihood classification were applied successfully to recognize plant communities in Montana. Maximum likelihood classification did little to improve recognition of terrain features when compared to a single channel density slice in sparsely vegetated Arizona. LANDSAT was found to be more sensitive to differences between plant communities based on percentages of vigorous vegetation than to actual physical or spectral differences among plant species.

Bentley, R. G., Jr.↗

DOE OSTI2024

Unveiling the effect of composition on nuclear waste immobilization glasses’ durability by nonparametric machine learning

Abstract Ensuring the long-term chemical durability of glasses is critical for nuclear waste immobilization operations. Durable glasses usually undergo qualification for disposal based on their response to standardized tests such as the product consistency test or the vapor hydration test (VHT). The VHT uses elevated temperature and water vapor to accelerate glass alteration and the formation of secondary phases. Understanding the relationship between glass composition and VHT response is of fundamental and practical interest. However, this relationship is complex, non-linear, and sometimes fairly variable, posing challenges in identifying the distinct effect of individual oxides on VHT response. Here, we leverage a dataset comprising 654 Hanford low-activity waste (LAW) glasses across a wide compositional envelope and employ various machine learning techniques to explore this relationship. We find that Gaussian process regression (GPR), a nonparametric regression method, yields the highest predictive accuracy. By utilizing the trained model, we discern the influence of each oxide on the glasses’ VHT response. Moreover, we discuss the trade-off between underfitting and overfitting for extrapolating the material performance in the context of sparse and heterogeneous datasets.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Operator learning for predicting multiscale bubble growth dynamics

We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS1989

Regression and ratio estimators to integrate AVHRR and MSS data

Regression and ratio estimators are used to integrate AVHRR-Global Area Coverage (GAC) and Landsat MSS digital data to estimate forest area in the continental United States. Forestlands are enumerated for the 48 contiguous states using five different AVHRR-GAC data sets. Results indicated that the GAC and MSS forest estimates were not highly correlated. Although the ratio of means and linear regression corrections were, on the average, closer to national U.S. Forest Service forest area estimates, these correction procedures did not consistently improve GAC estimates of forest area. GAC forest area estimates tended to be high in densely forested regions such as the northeast and low in sparsely forested areas.

Nelson, Ross↗

NASA NTRS1997

Global Analysis of Empirical Relationships Between Annual Climate and Seasonality of NDVI

This study describes the use of satellite data to calibrate a new climate-vegetation greenness function for global change studies. We examined statistical relationships between annual climate indexes (temperature, precipitation, and surface radiation) and seasonal attributes of the AVHRR Normalized Difference Vegetation Index (NDVI) time series for the mid-1980s in order to refine our empirical understanding of intraannual patterns and global abiotic controls on natural vegetation dynamics. Multiple linear regression results using global l(sup o) gridded data sets suggest that three climate indexes: growing degree days, annual precipitation total, and an annual moisture index together can account to 70-80 percent of the variation in the NDVI seasonal extremes (maximum and minimum values) for the calibration year 1984. Inclusion of the same climate index values from the previous year explained no significant additional portion of the global scale variation in NDVI seasonal extremes. The monthly timing of NDVI extremes was closely associated with seasonal patterns in maximum and minimum temperature and rainfall, with lag times of 1 to 2 months. We separated well-drained areas from l(sup o) grid cells mapped as greater than 25 percent inundated coverage for estimation of both the magnitude and timing of seasonal NDVI maximum values. Predicted monthly NDVI, derived from our climate-based regression equations and Fourier smoothing algorithms, shows good agreement with observed NDVI at a series of ecosystem test locations from around the globe. Regions in which NDVI seasonal extremes were not accurately predicted are mainly high latitude ecosystems and other remote locations where climate station data are sparse.

Potter, C. S.↗

NASA NTRS2007

Validation of the Pulmonary Function System for Use on the International Space Station

Aerobic deconditioning occurs during long duration space flight despite the use of exercise countermeasures (Convertino, 1996). As a part of International Space Station (ISS) medical operations, periodic tests designed to estimate aerobic capacity are performed to track changes in aerobic fitness and to determine the effectiveness of exercise countermeasures. These tests are performed prior to, during, and after missions of greater than 30 days in duration. Crewmembers selected for missions aboard the ISS perform a graded exercise test on a cycle ergometer approximately 270 days prior to their scheduled launch date in order to measure peak oxygen consumption (VO2PK) and peak heart rate (HRpk). Approximately 30 to 45 days prior to launch, crewmembers perform a submaximal cycle ergometer test at work rates set to elicit 25, 50 and 75% of their pre-flight VO2PK. This test, known as the Periodic Fitness Evaluation (PFE), serves as a baseline measure to which subsequent in-and post-flight exercise tests are compared. While onboard the ISS, crewmembers are normally scheduled to perform the PFE beginning with flight day (FD) 14 and every 30 days thereafter. The PFE is also conducted 5 and 30 days following flight. Using PFE data, aerobic fitness is estimated by quantifying the VO2 vs. HR relationship using linear regression and calculating the VO2 that would occur at the crewmember s previously measured HRpk. Currently, for data collected during flight, this technique assumes that the pre- vs. in-flight oxygen consumption per given cycle workload is similar. However, the validity of this assumption is based upon a sparse amount of data collected during the Skylab era (Michel, et al. 1977). The method of using heart rate and cycle ergometer work rates has been used to estimate aerobic fitness in normal gravity (Astrand and Ryhming, 1954; Lee, 1993). Due to spaceflight induced physiological alterations, such as shifts in extracellular fluid (e.g. plasma) volume, this method may not be valid during space flight. In addition, the ergometer onboard ISS is vibration-isolated and moves with the astronaut s application of force into the pedals. The effect of this movement on the VO2 of cycle exercise on ISS has not been quantified.

McCleary, Frank A.↗

NASA NTRS2010

An Investigation of Widespread Ozone Damage to the Soybean Crop in the Upper Midwest Determined From Ground-Based and Satellite Measurements

Elevated concentrations of ground-level ozone (O3) are frequently measured over farmland regions in many parts of the world. While numerous experimental studies show that O3 can significantly decrease crop productivity, independent verifications of yield losses at current ambient O3 concentrations in rural locations are sparse. In this study, soybean crop yield data during a 5-year period over the Midwest of the United States were combined with ground and satellite O3 measurements to provide evidence that yield losses on the order of 10% could be estimated through the use of a multiple linear regression model. Yield loss trends based on both conventional ground-based instrumentation and satellite-derived tropospheric O3 measurements were statistically significant and were consistent with results obtained from open-top chamber experiments and an open-air experimental facility (SoyFACE, Soybean Free Air Concentration Enrichment) in central Illinois. Our analysis suggests that such losses are a relatively new phenomenon due to the increase in background tropospheric O3 levels over recent decades. Extrapolation of these findings supports previous studies that estimate the global economic loss to the farming community of more than $10 billion annually.

Fishman, Jack↗

NASA NTRS2022

MLtool: Universal Supervised Machine Learning Tool to Model Tabulated Data

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine learning↗

NASA NTRS2022

MLtool Python Code

Machine Learning↗

DOE OSTI2026

Machine learning inversion of interatomic force constants from single-crystal inelastic neutron scattering

Atomic vibrations govern many macroscopic properties of materials, but experiments to comprehensively probe them remain challenging. Inelastic neutron scattering (INS) is a powerful technique to map phonon dispersions in crystals, especially when leveraging modern time-of-flight (ToF) spectrometers with large detectors. However, efficiently and robustly extracting interatomic force constants (FCs) parameterizing phonon dynamics from experimental spectra remains a bottleneck due to the complexity and high dimensionality of ToF INS datasets. Here, we present a machine learning approach for the direct inversion of FCs from single-crystal INS measurements. The framework leverages synthetic training data generated using universal machine-learned force fields and an efficient physics-based forward model. We benchmark two neural architectures–one emphasizing structured latent representation learning and the other direct, supervised spectral regression–across simulated datasets for two materials under idealized and noisy conditions. The latent-representation model is subsequently applied to experimental single-crystal INS data on germanium. The model is shown to reproduce FCs derived from both first-principles simulations and from iterative optimization, and furthermore achieves reliable inference even from sparse, single-orientation measurements representing short data acquisitions. Analysis of the learned latent space reveals semantically continuous and physically interpretable encodings that support strong cross-domain generalization. By bridging theoretical and experimental domains, we establish a path toward rapid inversion of experimental spectra and data-driven interpretation of temperature-dependent lattice dynamics.

42 ENGINEERING↗

DOE OSTI2021

Efficient Data Compression for 3D Sparse TPC via Bicephalous Convolutional Autoencoder

Real-time data collection and analysis in large experimental facilities present a great challenge across multiple domains, including high energy physics, nuclear physics, and cosmology. To address this, machine learning (ML)-based methods for real-time data compression have drawn significant attention. However, unlike natural image data, such as CIFAR and ImageNet that are relatively small-sized and continuous, scientific data often come in as three-dimensional 3D data volumes at high rates with high sparsity (many zeros) and non-Gaussian value distribution. This makes direct application of popular ML compression methods, as well as conventional data compression methods, suboptimal. To address these obstacles, this work introduces a dual-head autoencoder to resolve sparsity and regression simultaneously, called Bicephalous Convolutional AutoEncoder (BCAE). This method shows advantages both in compression fidelity and ratio compared to traditional data compression methods, such as MGARD, SZ, and ZFP. To achieve similar fidelity, the best performer among the traditional methods can reach only half the compression ratio of BCAE. Moreover, a thorough ablation study of the BCAE method shows that a dedicated segmentation decoder improves the reconstruction.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1997

Global Analysis of Empirical Relationships Between Annual Climate and Seasonality of NDVI

This paper describes the use of satellite data to calibrate a new climate-vegetation greenness relationship for global change studies. We examined statistical relationships between annual climate indexes (temperature, precipitation, and surface radiation) and seasonal attributes If the AVHRR Normalized Difference Vegetation Index (NDVI) time series for the mid-1980's in order to refine our understanding of intra-annual patterns and global abiotic controls on natural vegetation dynamics. Multiple linear regression results using global 1o gridded data sets suggest that three climate indexes: degree days (growing/chilling), annual precipitation total, and an annual moisture index together can account to 70-80 percent of the geographic variation in the NDVI seasonal extremes (maximum and minimum values) for the calibration year 1984. Inclusion of the same annual climate index values from the previous year explains no substantial additional portion of the global scale variation in NDVI seasonal extremes. The monthly timing of NDVI extremes is closely associated with seasonal patterns in maximum and minimum temperature and rainfall, with lag times of 1 to 2 months. We separated well-drained areas from lo grid cells mapped as greater than 25 percent inundated coverage for estimation of both the magnitude and timing of seasonal NDVI maximum values. Predicted monthly NDVI, derived from our climate-based regression equations and Fourier smoothing algorithms, shows good agreement with observed NDVI for several different years at a series of ecosystem test locations from around the globe. Regions in which NDVI seasonal extremes are not accurately predicted are mainly high latitude zones, mixed and disturbed vegetation types, and other remote locations where climate station data are sparse.

Potter, C. S.↗

NASA NTRS1982

Tropical cyclone track and genesis forecasting using satellite microwave sounder data

Although many dynamical and statistical prediction schemes are available to forecasters, tropical cyclone track errors are still large. One primary difficulty is that tropical cyclones exist over the data-sparse tropical oceans. Satellite sounders, however, routinely provide numerous data over these areas. Mean layer temperatures from the Scanning Microwave Spectrometer on board the Nimbus 6 satellite are decomposed using empirical orthogonal functions, and the expansion coefficients are related to deviations from the persistence forecast location, to speed change, to direction change and to intensity change. The significance of the regression equations is tested by a null hypothesis of zero correlation coefficient. It appears that significant information about tropical cyclone motion exists in the satellite-estimated mean layer temperatures, especially at upper levels. A physical interpretation of the statistical results is offered, and a one-storm-out independent test is used to test the stability of the equations. Finally, some further work is suggested.

Kidder, S. Q.↗

DOE OSTI2025

A Markov chain Monte Carlo (MCMC) Bayesian inference approach to analyze apparent activation barriers and reaction orders from microreactor data

Statistical analysis of steady-state catalytic kinetic data is often limited by data sparsity due to the slow pace at which the data is collected. Data sparsity and limitations in statistical analysis make it difficult to differentiate between mechanistic models and catalytic sites. A Bayesian inference tool is reported for catalysis researchers to estimate error in the determination of reaction orders from steady state microreactor data. The benefits of a Bayesian inference approach are discussed, as an alternative to the more common frequentist approach. The approach incorporates prior knowledge of the system and the data collected to form an error estimate on reaction orders. We investigated the effects of three distinct data treatments—individual fitting of trials, pooled analysis, and constrained regression methods—on the precision and uncertainty of reaction order determinations. To assess the robustness of our findings, we conducted sensitivity analyses to evaluate the influence of Bayesian parameters on uncertainty estimation. Additionally, we utilized synthetic data to illustrate how data quality impacts the precision of uncertainty assessments. We show Bayesian analysis can obtain a more precise estimation of error with a sparse data set than a frequentist analysis. Finally, this work provides strong evidence that the adoption of Bayesian analysis of kinetic data may help researchers make more precise arguments as to the strength of their evidence for a particular mechanistic hypothesis, or in comparing across different catalysts.

42 ENGINEERING↗

DOE OSTI2020

A Satellite-Derived Upper-Ocean Stratification Data Set for the Tropical North Atlantic With Potential Applications for Hurricane Intensity Prediction

Upper-ocean stratification strongly impacts vertical mixing and the heat flux between the ocean and atmosphere, especially under extreme conditions of tropical cyclones (TCs). Knowledge of prestorm stratification is important for accurate TC intensity prediction. In situ observations of the tropical ocean have significantly increased in the past decade. However, they are still too sparse to resolve ocean stratification variability in near-real time and on small spatial scales. Here, in this study, based on long-term observations and an ocean reanalysis data set from 2004–2017, we investigate the possibility of retrieving upper-ocean stratification from sea surface temperature (SST), sea surface salinity (SSS), and sea surface height (SSH) using a simple regression method. It is found that more than 90% of the mean seasonal cycle and about 30% to 80% of temperature and salinity stratification anomalies can be reconstructed using surface data from either observations or an ocean reanalysis. Simple regression can be used with satellite observations to create a high-resolution, near-real-time-gridded ocean stratification data set that successfully reproduces both the large and mesoscale variability of ocean stratification. When used in a simple expression for TC-induced SST cooling, the satellite-derived stratification shows improvements over an ocean analysis in terms of variance explained of SST cooling, offering promise as a near-real-time indicator of the ocean's impact on TC intensification.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Nonlinear sparse Bayesian learning for physics-based models

This paper addresses the issue of overfitting while calibrating unknown parameters of over-parameterized physics-based models with noisy and incomplete observations. Here, a semi-analytical Bayesian framework of nonlinear sparse Bayesian learning (NSBL) is proposed to identify sparsity among model parameters during Bayesian inversion. NSBL offers significant advantages over machine learning algorithm of sparse Bayesian learning (SBL) for physics-based models, such as 1) the likelihood function or the posterior parameter distribution is not required to be Gaussian, and 2) prior parameter knowledge is incorporated into sparse learning (i.e. not all parameters are treated as questionable). NSBL employs the concept of automatic relevance determination (ARD) to facilitate sparsity among questionable parameters through parameterized prior distributions. The analytical tractability of NSBL is enabled by employing Gaussian ARD priors and by building a Gaussian mixture-model approximation of the posterior parameter distribution that excludes the contribution of ARD priors. Subsequently, type-II maximum likelihood is executed using Newton's method whereby the evidence and its gradient and Hessian information are computed in a semi-analytical fashion. We show numerically and analytically that SBL is a special case of NSBL for linear regression models. Subsequently, a linear regression example involving multimodality in both parameter posterior pdf and model evidence is considered to demonstrate the performance of NSBL in cases where SBL is inapplicable. Next, NSBL is applied to identify sparsity among the damping coefficients of a mass-spring-damper model of a shear building frame. These numerical studies demonstrate the robustness and efficiency of NSBL in alleviating overfitting during Bayesian inversion of nonlinear physics-based models.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1990

Estimation of multidimensional precipitation parameters by areal estimates of oceanic rainfall

The parameters of the multidimensional precipitation model proposed by Waymire et al. (1984) are estimated using the areal-averaged radar measurements of precipitation of the Global Atlantic Tropical Experiment (GATE) data set. The procedure followed was the fitting of the first- and second-order moments at different aggregation scales by nonlinear regression techniques. The numerical estimates of the parameters using different subsets of GATE information were reasonably stable, i.e., they were not affected by changes of the area-averaging size, temporal length of the records, and percentage of areal coverage of rainfall. This suggests that the estimation procedure is relatively robust and suitable to estimate the parameters of the multidimensional model in areas of sparse density of rain gages. The use of the space-time spectrum of rainfall to help in the determination of sampling errors due to intermittent visits of future space-borne low-altitude sensors of precipitation is also discussed.

Valdes, J. B.↗