Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “regression models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A Parametric, Data-Driven, Non-Intrusive Reduced-Order Model Framework for Crystal Plasticity Simulations of Voids

The influence of the internal structure at micrometer length scales on the deformation of polycrystalline materials can be effectively captured using crystal plasticity finite element methods (CPFEM). However, the complexity and nonlinearity of the deformation equations CPFEM solves demand significant computational power and resources to achieve accurate predictions, limiting its broader application. To address this challenge, we have identified a reduced-order representation of the complex data in order to establish a computationally efficient reduced-order models (ROM) and drastically reduce the computational expense of CPFEM. Specifically, in this work, we developed a parametric, data-driven, and non-intrusive ROM framework for CPFEM using proper orthogonal decomposition (POD) and sparse variational Gaussian process (SVGP) regression for single-crystal microstructures under tensile loading conditions. The developed protocol enables one to compress field into a latent/low-dimensional space described by principal component analysis (PCA) via the singular value decomposition (SVD) algorithm. As a result, the high-dimensional data are reduced to a significantly smaller amount of dimensions with POD bases and POD coefficients. Furthermore, we deployed an ensemble of SVGPs—extended from the classical Gaussian process (GP) regression for scalability and handling big data—in a massively parallel manner to train and predict latent POD coefficients using known POD bases from a set of previously obtained simulations results. Lastly, using the predicted POD coefficients, we reconstructed the full-field results and showed reasonable agreement compared with the true values obtained from running CPFEM. The developed framework is validated with a set of CPFEM simulations of a single embedded void in single-crystal aluminum alloy. While the framework is broadly applicable, this work specifically focuses on single-crystal microstructures, a single load case (e.g., tensile), and a specific void geometry (spherical).

Anisotropy↗
DOE OSTI2024

Tensor decompositions for count data that leverage stochastic and deterministic optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the maximum likelihood estimator (MLE) of the Poisson CPD model. Here, this work presents two new algorithms that extend state-of-the-art local methods for Poisson CPD. Hybrid GCP-CPAPR combines Generalized Canonical Decomposition (GCP) with stochastic optimization and CP Alternating Poisson Regression (CPAPR), a deterministic algorithm, to increase the probability of converging to the MLE over either method used alone. Restarted CPAPR with SVDrop uses a heuristic based on the singular values of the CPD model unfoldings to identify convergence toward optimizers that are not the MLE and restarts within the feasible domain of the optimization problem, thus reducing overall computational cost when using a multi-start strategy. We provide empirical evidence that indicates our approaches outperform existing methods with respect to converging to the Poisson CPD MLE.

CPAPR↗
DOE OSTI2021

Quantification of Raman-Interfering Polyoxoanions for Process Analysis: Comparison of Different Chemometric Models and a Demonstration on Real Hanford Waste

The Hanford site represents a complicated environmental remediation challenge, remaining from the production of nuclear weapons. Over 100 million gallons of liquid radioactive waste of unknown composition will be chemically processed and vitrified, but the varying chemical composition and highly radioactive nature of the waste preclude the implementation of more developed, offline technologies to determine the composition. The only practical approach to waste treatment will require the significant utilization of real-time, chemometric modeling approaches. Although chemometric approaches have been applied to the analysis of Hanford waste, the models developed were highly tank-specialized, and limited discussion was provided on how models fared with interfering signals. As the tank waste is largely composed of oxoanions, which tend to have interfering Raman spectra, the general question was posed as to what chemometric approach is best suited to accurately quantify analytes in the presence of interfering signals. This was carried out by examining the ability of classical least square (CLS), principal component regression (PCR), partial least square (PLS), and locally weighted regression (LWR) to quantify NO 3 – and CO 3 2– using their bands around 1050 cm –1 . Finally, for all samples, the PLS-based model was found to be the most efficient approach from a model building and application perspective.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Clarifying trust of materials property predictions using neural networks with distribution-specific uncertainty quantification

It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

36 MATERIALS SCIENCE↗
DOE OSTI2022

ASP (Adaptive Splines for Prediction) [SWR-22-66]

This software package provides functionality to fit piecewise cubic splines to model the relationship between a univariate X (independent variable) and a univariate Y (dependent variable). The output is akin to linear regression but has higher "capacity" in that it can learn more complicated relationships despite leveraging only a single predictor. It thus can learn very reasonable relationships between X and Y with minimal effort. The canonical intended use case is for predicting electric demand (Y) from temperature (X). There can be complicated, non-linear relationships between these two variables, but these can be reasonably approximated by the sum of: 1) an "aggregate" relationship between daily average temperature and daily average demand, and 2) intra-day patterns represented as hourly offsets from the daily average load. The spline functionality is used for learning a robust "aggregate" relationship without the inconvenience of manual feature engineering while still ensuring robust results. The model fitting workhorse is R's built-in smooth.spline. This package provides a suite of options for controlling how smooth.spline fits a spline to the data. In particular, it is designed to return a model with a single critical point (i.e., a single location along the domain/support where the first derivative is zero). This is to ensure that predicted changes in electric demand are always positive as temperatures become more extreme. This helps to prevent overfitting, particularly when the input data has few observations or has significant uncertainty from other sources.

Murphy, Sinnott↗
DOE OSTI2026

Arm and shoulder muscle segmentation in axial MRI with UNet deep learning model

Quantifying individual upper-limb muscle volumes from MRI provides key insight into muscle-specific strength, deficits, and adaptations. Manual delineation is the gold standard but time‑intensive, and the performance of current deep learning approaches, particularly for small or anatomically complex muscles, remains incompletely characterized. We evaluated a state‑of‑the‑art deep learning framework across the entire upper limb and analyzed factors governing segmentation performance, with attention to the forearm. Three previously published MRI datasets (1.5 T, 3D GRE T1‑weighted; total n = 39) spanning young, middle‑aged, and older adults were curated and quality‑checked, including expert manual segmentations for 31 muscles. Following multiclass mask reconstruction, we trained three 3D nnU‑Net multiclass models matched to the muscle subsets present across datasets, using five‑fold cross‑validation and a composite Dice Similarity Coefficient (DSC) + cross entropy loss. Segmentation accuracy was assessed with DSC. Performance varied across muscles (mean DSC = 0.806 ± 0.098), ranging from 0.920 (Deltoid) to 0.461 (Extensor pollicis brevis). In uncertainty‑weighted regressions, muscle volume was positively associated with DSC (R2 = 0.36, p < 0.001), whereas training segmentation count and muscle orientation showed negligible associations (R2 ≤ 0.06). A weighted mixed‑effects model identified volume as the strongest evaluated predictor, explaining 23.9% of variance in DSC; orientation and training count each contributed <1%, leaving 61.5% unexplained. These results indicate that deep learning–based segmentation can accurately quantify muscle volume for many upper‑limb muscles but remains constrained for small, low‑contrast forearm muscles.

Gillespie, Samuel↗
DOE OSTI2022

Leveraging Multiple Raman Excitation Wavelength Systems for Process Monitoring of Nuclear Waste Streams

Processing nuclear waste from sites such as Hanford is a significant environmental cleanup need while being a significant logistical challenge. Integration of process monitoring tools, that can provide in situ and real-time feedback about the process, can significantly alleviate needs to collect grab samples for process control and product characterization. Raman spectroscopy paired with chemometric analysis is one process monitoring tool that can provide chemical composition information on a large number of chemical targets in nuclear waste streams. However, methods to improve limits of detection as well as drop uncertainty in quantification are needed. Optimizing instrument specifications can achieve this, here this is demonstrated by comparing limits of detection for key analytes when using Raman systems with 671 nm, 532 nm, and 405 nm excitation wavelengths. Generally, limits of detection decease (allowing the measurement of lower salt concentrations) with decreasing wavelength. Similarly, data collection times and averaging were optimized. Finally, multiple chemometric modeling approaches were leveraged, including multiblock methods that combined data from all three Raman systems to simultaneously quantify targets with improved sensitivity.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2019

Spatial Predictions and Associated Uncertainty of Annual Soil Respiration at the Global Scale

Abstract Soil respiration (Rs), the soil‐to‐atmosphere CO 2 flux produced by microbes and plant roots, is a critical but uncertain component of the global carbon cycle. Our current understanding of the variability and dynamics is limited by the coarse spatial resolution of existing estimates. We predicted annual Rs and associated uncertainty across the world at 1‐km resolution using a quantile regression forest algorithm trained with observations from the global Soil Respiration Database spanning from 1961 to 2011. This model yielded a global annual Rs estimate of 87.9 Pg C/year with an associated global uncertainty of 18.6 (mean absolute error) and 40.4 (root mean square error) Pg C/year. The estimated annual heterotrophic respiration (Rh), derived from empirical relationships with Rs, was 49.7 Pg C/year over the same period. Predicted Rs rates and associated uncertainty varied widely across vegetation types, with the greatest predicted rates of Rs in evergreen broadleaf forests (accounting for 20.9% of global Rs). The greatest prediction uncertainties were in northern latitudes and arid to semiarid ecosystems, suggesting that these areas should be targeted in future measurement campaigns. This study provides predictions of Rs (and associated prediction uncertainty) at unprecedentedly high spatial resolution across the globe that could help constrain local‐to‐global process‐based models. Furthermore, it provides insights into the large variability of Rs and Rh across vegetation classes and identifies regions and vegetation types with poor model performance that should be prioritized for future data collection.

Warner, D. L.↗
DOE OSTI2021

Regional environmental controllers influence continental scale soil carbon stocks and future carbon dynamics

Abstract Understanding the influence of environmental factors on soil organic carbon (SOC) is critical for quantifying and reducing the uncertainty in carbon climate feedback projections under changing environmental conditions. We explored the effect of climatic variables, land cover types, topographic attributes, soil types and bedrock geology on SOC stocks of top 1 m depth across conterminous United States (US) ecoregions. Using 4559 soil profile observations and high-resolution data of environmental factors, we identified dominant environmental controllers of SOC stocks in 21 US ecoregions using geographically weighted regression. We used projected climatic data of SSP126 and SSP585 scenarios from GFDL-ESM 4 Earth System Model of Coupled Model Intercomparison Project phase 6 to predict SOC stock changes across continental US between 2030 and 2100. Both baseline and predicted changes in SOC stocks were compared with SOC stocks represented in GFDL-ESM4 projections. Among 56 environmental predictors, we found 12 as dominant controllers across all ecoregions. The adjusted geospatial model with the 12 environmental controllers showed an R 2 of 0.48 in testing dataset. Higher precipitation and lower temperatures were associated with higher levels of SOC stocks in majority of ecoregions. Changes in land cover types (vegetation properties) was important in drier ecosystem as North American deserts, whereas soil types and topography were more important in American prairies. Wetlands of the Everglades was highly sensitive to projected temperature changes. The SOC stocks did not change under SSP126 until 2100, however SOC stocks decreased up to 21% under SSP585. Our results, based on environmental controllers of SOC stocks, help to predict impacts of changing environmental conditions on SOC stocks more reliably and may reduce uncertainties found in both, geospatial and Earth System Models. In addition, the description of different environmental controllers for US ecoregions can help to describe the scope and importance of global and local models.

58 GEOSCIENCES↗
DOE OSTI2023

A Better Method to Calculate Fuel Burnup in Pebble Bed Reactors Using Machine Learning

Burnup measurement is an important step in material control and accountancy (MC&A) at nuclear reactors, and may be done by examining gamma spectra of fuel samples. Traditional approaches rely on known correlations to specific photopeaks (e.g. 137 Cs) and operate via a standard linear regression method. However, the quality of these regression methods is limited even in the best case, and is significantly poorer at short fuel cool-down times, due to the elevated radiation background by short life-time isotopes, and self-shielding effect of the fuel. For practical operation of pebble bed reactors (PBRs), quick measurements (in minutes) and short cooling times (in hours) are required from a safety and security perspective. We investigated the efficacy and performance of machine learning (ML) methods to predict the burnup of the pebble fuel from full gamma spectra (rather than specific discrete photopeaks) and found a full-spectrum ML approach to far outperform baseline regression predictions in all measurement and cooling conditions - including in operational-like measurement conditions. We also performed model and data ablation experiments to determine the relative performance impact of our ML methods' capacity to model data nonlinearities and the inherent additional information in full spectra. Applying our ML methods, we found a number of surprising results, including improved accuracy at shorter fuel cooling times (the opposite of the norm), remarkable robustness to spectrum compression (via rebinning), and competitive burnup predictions even when using background signal only (i.e. explicitly omitting known isotope photopeaks).

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2026

SPUS-Small-PDE-U-net-Solver

Small PDE U-Net Solver (SPUS) is a compact and efficient foundation model (FM) designed as a unified neural operator for solving a wide range of partial differentialequations (PDEs). SPUS leverages a lightweight residual U-Net-based architecture as a foundation model architecture. To enable effective learning in this minimalist framework, SPUS utilizes a simple yet powerful auto-regressive pretraining strategy which closely replicates the behavior of numerical solvers to learn the underlying physics. SPUS is designed to be pretrained on a diverse set of fluid dynamics PDEs from public benchmark datasets.

Siddik, Abu↗
DOE OSTI2024

Spectral Data Fusion From Handheld Laser-Induced Breakdown Spectroscopy (LIBS) and X-ray Fluorescence (XRF) Analyzers for Improved Detection of Cerium in a Simulated Dispersal Accident

Here, this work implements a mid-level data fusion methodology on spectral data from handheld X-ray fluorescence and laser-induced breakdown spectroscopy analyzers to quantify plutonium surrogate (CeO 2 ) contamination in soil samples for the first time. Spectral data from each analyzer were used independently to train supervised machine learning regressions to predict Ce concentration. Fused features from both data sets were then used to train the same models, comparing prediction performance by evaluating model precision and sensitivity. Fusing principal component scores from the two sensors yielded an order of magnitude improvement in precision and sensitivity of predictions made with an artificial neural network, compared to predictions made by models trained on independent sensor data. As a result, a boosted ensemble trained on the fused spectral features yielded an ideal predictor with root-mean-squared error on the order of 10 –6 and calculated limit of detection order 10 –5 wt %.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

GMLS-Nets: A Framework for Learning from Unstructured Data

Data fields sampled on irregularly spaced points arise in many applications in the sciences and engineering. For regular grids, Convolutional Neural Networks (CNNs) have been successfully used to gaining benefits from weight sharing and invariances. We generalize CNNs by introducing methods for data on unstructured point clouds based on Generalized Moving Least Squares (GMLS). GMLS is a non-parametric technique for estimating linear bounded functionals from scattered data, and has recently been used in the literature for solving partial differential equations. By parameterizing the GMLS estimator, we obtain learning methods for operators with unstructured stencils. In GMLS-Nets the necessary calculations are local, readily parallelizable, and the estimator is supported by a rigorous approximation theory. We show how the framework may be used for unstructured physical data sets to perform functional regression to identify associated differential operators and to regress quantities of interest. The results suggest the architectures to be an attractive foundation for data-driven model development in scientific machine learning applications

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Perspectives on the Calibration of CNN Energy Reconstruction in Highly Granular Calorimeters

We present a study which shows encouraging stability of the response linearity for a simulated high granularity calorimeter module reconstructed by a CNN model to miscalibration, bias, and noise effects. Our results also show an intuitive, quantifiable relationship between these factors and the calibration parameters. We trained a CNN model to reconstruct energy in the calorimeter module using simulated single-pion events; we then observed the response of the model under various miscalibration, bias, and noise conditions that affected the model input. From these data, we estimated linear response models to calibrate the CNN. We also quantified the relationship between these factors and the calibration parameters by regression analysis.

Akchurin, Nural↗
DOE OSTI2026

Neural Network‐Based Methods for Ocean Surface Wave Measurement Using Submarine Distributed Acoustic Sensing (DAS)

Two new data-driven models for estimating ocean surface waves from distributed acoustic sensing (DAS) submarine cable strain rate are developed using supervised machine learning on a 10-day data set collected offshore of Oliktok Point, Alaska. The new models were trained on target data from seafloor pressure moorings at three sites spaced evenly along 27.1 km of cable and were benchmarked against an empirical transfer function method previously used to estimate waves from DAS. A model which uses convolutional neural networks to transform 2-km frequency-wavenumber strain spectra to seafloor pressure spectra outperforms the benchmark in wave height prediction (RMSE of 0.15 vs. 0.41 m) and period prediction (0.29 vs. 0.37 s) when evaluated on a held-out test data set. When applied to a DAS data set collected on the same cable 2 years prior, the CNN-based model maintained similar significant wave height performance (RMSE = 0.23 m) relative to available satellite altimetry data. A two-hidden-layer, fully connected neural network which transforms 1-D strain spectra to seafloor pressure spectra also outperforms the benchmark in wave height prediction (RMSE of 0.19 vs. 0.41 m), but does not generalize as well to the prior data. Regression-based machine learning is useful for estimating waves from DAS data when the pressure-strain relationship varies temporally and spatially across different wave conditions. Models can be applied to DAS data to measure waves with higher spatial resolution and longer temporal coverage than traditional methods, which often measure waves only at a single point.

Davis, Jacob R. [Univ. of Washington, Seattle, WA ↗
DOE OSTI2024

Poleward Migration of the Latitude of Maximum Tropical Cyclone Intensity—Forced or Natural?

Abstract Past studies have shown a significant observed poleward trend in the latitude at which tropical cyclones reach their lifetime maximum intensity (LMI), especially in the northwest Pacific basin. Given the brevity of the historical record, it remains difficult to separate the forced trend from internal variability of the climate system. A recently developed tropical cyclone downscaling model is used to downscale the Community Earth System Model, version 2 (CESM2), preindustrial control simulation. It is found that the observed trend in the latitude at which tropical cyclones reach their LMI in the northwest Pacific is very unlikely to be caused by internal variability. The same downscaling model is then used to downscale CESM2 simulations under historical forcing. The resulting trend distribution shows a significant poleward migration of tropical cyclone LMI even after regressing out both natural variability and the part of the forced warming pattern that projects onto natural variability. The results indicate that the observed poleward migration of the latitude at which tropical cyclones reach their LMI in the northwest Pacific basin is likely to be, at least in part, forced. However, the magnitude of the projected poleward trend in climate models can be significantly modulated by the simulated spatial pattern of ocean warming. This highlights how discrepancies between models and observations, with regard to projected changes to the equatorial zonal sea surface temperature gradient under anthropogenic forcing, can lead to large uncertainties in projected changes to the LMI latitude of tropical cyclones. Significance Statement Observations in the northwest Pacific basin show that the latitude at which tropical cyclones are at their most intense has been trending northward in the recent half century. These changes are important since tropical cyclones could bring hazardous weather to coastal areas that are poorly equipped to handle them. Here, we show that natural variations in Earth’s climate are very unlikely to explain the observed poleward trend in the latitude that tropical cyclone reach their maximum intensity. We find that it is much more likely that the observed trend is forced by human-related emissions, though the spatial pattern of warming in response to greenhouse emissions can have significant impacts on the magnitude of the trend.

Lin, Jonathan↗
DOE OSTI2021

On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

Physics-informed neural networks (PINNs) are demonstrating remarkable promise in integrating physical models with gappy and noisy observational data, but they still struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. Here in this work we investigate this limitation through the lens of Neural Tangent Kernel (NTK) theory and elucidate how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK. Using this observation, we construct novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justify how such coordinate embedding layers can lead to robust and accurate PINN models. Numerical examples are presented for several challenging cases where conventional PINN models fail, including wave propagation and reaction–diffusion dynamics, illustrating how the proposed methods can be used to effectively tackle both forward and inverse problems involving partial differential equations with multi-scale behavior.

42 ENGINEERING↗