SEARCH · Engineering Papers

Results for “Constrained regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Designing Monte Carlo Simulation and an Optimal Machine Learning to Optimize and Model Space Missions

This paper investigates applying artificial intelligence (AI) algorithms to attitude control system of satellites to optimally tune the controller using high performance computing. This methodology is applied to the Virtual Telescope for X-ray Observation mission, which is a precise formation of two separate spacecraft observing multiple objects in the space in the X-ray domain. The mission is divided into phases based on the instrumentation and the mission goal. To reach an stable precise formation robust to stochastic slew and slew rate (i.e., Euler angles and angular velocities) in a minimal constrained time T , consumed energy of the attitude control system, denoted as E, and root-mean-square state error of attitude control system, denoted as e, are minimized. Monte-Carlo simulation is used for the sensitivity analysis of optimization and designing a controller. Deep neural networks (DNN), Gaussian processes (GP), and support vector regression (SVR) learn this optimization as a surrogate model, while their hyperparameters are optimized in a novel approach. THETA supercomputer at Argonne Leadership Computing Facility (ALCF) is used for optimizing the hyperparameters of DNN. The surrogate model meets the requirements of the mission, and it shows a better performance over the optimization and Monte-Carlo. The optimal DNN can satisfy the mission requirements e and T while reducing E for 90% compared to the other given methods.

42 ENGINEERING↗

DOE OSTI2023

Partial Least Squares, Experimental Design, and Near-Infrared Spectrophotometry for the Remote Quantification of Nitric Acid Concentration and Temperature

Near-infrared spectrophotometry and partial least squares regression (PLSR) were evaluated to create a pleasantly simple yet effective approach for measuring HNO3 concentration with varying temperature levels. A training set, which covered HNO3 concentrations (0.1–8 M) and temperature (10–40 °C), was selected using a D-optimal design to minimize the number of samples required in the calibration set for PLSR analysis. The top D-optimal-selected PLSR models had root mean squared error of prediction values of 1.4% for HNO 3 and 4.0% for temperature. The PLSR models built from spectra collected on static samples were validated against flow tests including HNO 3 concentration and temperature gradients to test abnormal conditions (e.g., bubbles) and the model performance between sample points in the factor space. Based on cross-validation and prediction modeling statistics, the designed near-infrared absorption approach can provide remote, quantitative analysis of HNO 3 concentration and temperature for production-oriented applications in facilities where laser safety challenges would inhibit the implementation of other optical techniques (e.g., Raman spectroscopy) and in which space, time, and/or resources are constrained. The experimental design approach effectively minimized the number of samples in the training set and maintained or improved PLSR model performance, which makes the described chemometric approach more amenable to nuclear field applications.

07 ISOTOPE AND RADIATION SOURCES↗

NASA NTRS2020

Modeling Whitecaps on Global Scale

Whitecaps play an important role in the surface-atmosphere interactions across the ocean. They are directly linked to the energy dissipation rate during wave breaking and transfer of heat, momentum, and gas/aerosol exchange from the wind to the sea surface. Although the first models of W were dependent only on wind speeds, a large number of diverse models based on wind and sea state which include wave height, wave age, friction velocity, and stability effects have been proposed since then. However, it is recognized that most of the proposed W models have strong systematic (e.g., zonal bias) and random errors when compared against observations. This is partly due to the differences in environmental conditions, measurement techniques, and geographical locations among these studies. But, some of these biases are linked to the inability of the proposed models to capture the variability in W in certain wind/wave regimes. Despite the knowledge of existing biases, W residual relationships from the models with wind and wave fields remain highly uncertain, with residual trends varying between the published studies. Here, we take advantage of the availability of relatively dense observations of W from WindSat microwave satellite retrievals in combination with the University of Miami wave model which was recently incorporated within the NASA GMAO/GEOS system (GEOS-UMWM). We use Windsat W retrievals to assess and constrain the previously published W models and understand the relationships of residuals from models in different wind/wave regimes. We link these unexplained residual variations to additional factors such as swell index, drag coefficient etc and add information to the existing whitecap models. Since Windsat retrievals cover wide range of environmental conditions, it helps to reduce the uncertainties associated with differences in measurement techniques. Regression of wind-wave fields against all Windsat data points (CTL) results in larger residuals for lower wave age and W is overestimated upto ~4% for wave age < 10 and underestimated by upto ~2% as wave age increases. We attest to this bias by considering two approaches. One is to perform regression separately for different stages of wave development such as developing sea, fully developed, and wind sea regimes thereby understanding the sensitivity of regression coefficients to sea state (EXP1). Another is to derive coefficients of W models in EXP1 as a function of additional wind/wave factors such as swell index, drag coefficient, and mean squared slope, deriving more nonlinear W models (EXP2). EXP2 provides reduction in Root Mean Squared Error (RMSE) by 0.1-0.3%. Sea surface drag has a stronger relationship with regression coefficients compared to swell index.These additional factors provide improved parameterizations in different wind and wave age regimes, with smaller unexplained/residual variations in W that has been a major concern in the W community.

Raman, Aishwarya↗

DOE OSTI2019

Reconstructing probability distributions with Gaussian processes

Modern cosmological analyses constrain physical parameters using Markov Chain Monte Carlo (MCMC) or similar sampling techniques. Oftentimes, these techniques are computationally expensive to run and require up to thousands of CPU hours to complete. Here we present a method for reconstructing the log-probability distributions of completed experiments from an existing chain (or any set of posterior samples). Here, the reconstruction is performed using Gaussian process regression for interpolating the log-probability. This allows for easy resampling, importance sampling, marginalization, testing different samplers, investigating chain convergence, and other operations. As an example use case, we reconstruct the posterior distribution of the most recent Planck 2018 analysis. We then resample the posterior, and generate a new chain with 40 times as many points in only 30 min. Our likelihood reconstruction tool is made publicly available online.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Machine Learning with Gradient-Based Optimization of Nuclear Waste Vitrification with Uncertainties and Constraints

Gekko is an optimization suite in Python that solves optimization problems involving mixed-integer, nonlinear, and differential equations. The purpose of this study is to integrate common Machine Learning (ML) algorithms such as Gaussian Process Regression (GPR), support vector regression (SVR), and artificial neural network (ANN) models into Gekko to solve data based optimization problems. Uncertainty quantification (UQ) is used alongside ML for better decision making. These methods include ensemble methods, model-specific methods, conformal predictions, and the delta method. An optimization problem involving nuclear waste vitrification is presented to demonstrate the benefit of ML in this field. ML models are compared against the current partial quadratic mixture (PQM) model in an optimization problem in Gekko. GPR with conformal uncertainty was chosen as the best substitute model as it had a lower mean squared error of 0.0025 compared to 0.018 and more confidently predicted a higher waste loading of 37.5 wt% compared to 34 wt%. The example problem shows that these tools can be used in similar industry settings where easier use and better performance is needed over classical approaches. Future works with these tools include expanding them with other regression models and UQ methods, and exploration into other optimization problems or dynamic control.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

Gaussian-process generative model for the QCD equation of state

We develop a generative model for the nuclear matter equation of state at zero net baryon density using the Gaussian process regression method. We impose first-principles theoretical constraints from lattice quantum chromodynamics and hadron resonance gas at high- and low-temperature regions, respectively. By allowing the trained Gaussian process regression model to vary freely near the phase transition region, we generate random smooth crossover equations of state with different speeds of sound that do not rely on specific parametrizations. Here, we explore a collection of experimental observable dependencies on the generated equations of state, which paves the groundwork for future Bayesian inference studies to use experimental measurements from relativistic heavy-ion collisions to constrain the nuclear matter equation of state.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2026

Regularization via f -Divergence: An Application to Multi-Oxide Spectroscopic Analysis

In this paper, we explore the application of convolutional neural networks (CNNs) for predicting the chemical composition of complex geologic samples in a simulated Martian atmospheric environment. Specifically, we aim to characterize oxide weight percentages (wt.%) of rock samples analyzed by remote Laser-Induced Breakdown Spectroscopy (LIBS), framing the problem as a multi-target regression task . Neural networks trained on LIBS spectra are prone to overfitting due to high spectral complexity, limited labeled data, and measurement noise. While regularization is critical for improving generalization, common methods (e.g., ℓ 2 regularization) impose constraints not directly tied to data distribution properties. We propose a novel regularization method based on a specific ƒ-divergence induced by a graph-based estimator, designed to constrain the distributional discrepancy between predictions and targets. This regularizer serves a dual purpose: (a) mitigating overfitting by enforcing a constraint on the distributional difference between predictions and noisy targets, and (b) acting as an auxiliary loss that penalizes large divergences. To enable backpropagation, we develop a differentiable approximation of this particular ƒ-divergence, making the method feasible for neural networks. Experiments on ChemCam and SuperCam LIBS calibration spectra show that mathematical equation-divergence regularization outperforms or matches standard regularization methods (ℓ 1 , ℓ 2 , dropout) and the classical baseline, partial least squares (PLS). Combining ƒ-divergence regularization with standard regularization yields further performance gains, indicating that distributional regularization is useful in this context giving a promising direction for robust model training in planetary science applications. Source code is publicly available at Klein and Li (2025), https://doi.org/10.11578/dc.20250530.7.

58 GEOSCIENCES↗

NASA NTRSDate not supplied

TPSAS-NF1676L-17943-DND

SAGE II used the solar occultation technique to measure atmospheric ozone for 21 years between 1984 and 2005 and has continued to play a key role in numerous international assessments. Traditionally, the determination of trends in ozone using SAGE II data has been done via time series analysis of monthly mean data within specific latitude bands. However, given the sparse sampling of SAGE II data, this will introduce temporal and/or spatial biases, particularly in the period after late 2000 when the instrument operated at half-duty cycle. Herein we present a new method for performing time series analysis of unevenly sampled data (in particular SAGE II); namely, we utilize the daily mean data and perform a simultaneous temporal and spatial analysis, making use of a Fourier series to constrain latitudinal variation. This methodology is applied to both the previous (v6.2) and current (v7.0) versions of the SAGE II data. Results are discussed that show the improvements to the data between versions 6.2 and 7.0 that make time series analysis produce reduced residuals as well as the preliminary results of the time series itself, which reveal some of the limitations in currently used proxies for the regression.

Robert P Damadeo↗

DOE OSTI2021

A comparison of Gaussian processes and neural networks for computer model emulation and calibration

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. Finally, as a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Statistical and Machine Learning Approaches to Analyzing Pipeline Incidents in the United States (2010–2024)

This study applies machine learning methods to analyze natural gas pipeline incidents in the United States using the Pipeline and Hazardous Materials Safety Administration (PHMSA) Gas Distribution Incident Dataset (2010–2024). The dataset includes over 600 variables describing incident characteristics, infrastructure attributes, and contributing factors associated with unintentional gas releases. The objective is to assess whether these features can reliably predict the underlying cause of pipeline failures. Multinomial logistic regression and Random Forest models were developed to classify incident causes, including excavation damage, corrosion, equipment failure, and natural forces. Results show that excavation damage is both the most frequent and most predictable cause, with models achieving strong performance for this category. However, when excavation damage is excluded, model accuracy declines significantly, with some models performing near random levels. Across all approaches, severe class imbalance and limited variability in key predictors constrain predictive performance. Pipeline age and diameter emerge as the most influential variables, but they provide insufficient discriminatory power to distinguish among less frequent failure types. These findings indicate that non-excavation-related incidents are rare, heterogeneous, and weakly represented in the dataset, limiting the effectiveness of machine learning classification. Overall, this study highlights the structural limitations of the PHMSA dataset for predictive modeling and underscores the need for improved data balance and feature enrichment. The results reinforce excavation damage prevention as the most impactful strategy for reducing pipeline incidents.

03 NATURAL GAS↗

DOE OSTI2026

Arm and shoulder muscle segmentation in axial MRI with UNet deep learning model

Quantifying individual upper-limb muscle volumes from MRI provides key insight into muscle-specific strength, deficits, and adaptations. Manual delineation is the gold standard but time‑intensive, and the performance of current deep learning approaches, particularly for small or anatomically complex muscles, remains incompletely characterized. We evaluated a state‑of‑the‑art deep learning framework across the entire upper limb and analyzed factors governing segmentation performance, with attention to the forearm. Three previously published MRI datasets (1.5 T, 3D GRE T1‑weighted; total n = 39) spanning young, middle‑aged, and older adults were curated and quality‑checked, including expert manual segmentations for 31 muscles. Following multiclass mask reconstruction, we trained three 3D nnU‑Net multiclass models matched to the muscle subsets present across datasets, using five‑fold cross‑validation and a composite Dice Similarity Coefficient (DSC) + cross entropy loss. Segmentation accuracy was assessed with DSC. Performance varied across muscles (mean DSC = 0.806 ± 0.098), ranging from 0.920 (Deltoid) to 0.461 (Extensor pollicis brevis). In uncertainty‑weighted regressions, muscle volume was positively associated with DSC (R2 = 0.36, p < 0.001), whereas training segmentation count and muscle orientation showed negligible associations (R2 ≤ 0.06). A weighted mixed‑effects model identified volume as the strongest evaluated predictor, explaining 23.9% of variance in DSC; orientation and training count each contributed <1%, leaving 61.5% unexplained. These results indicate that deep learning–based segmentation can accurately quantify muscle volume for many upper‑limb muscles but remains constrained for small, low‑contrast forearm muscles.

Gillespie, Samuel↗

DOE OSTI2020

Reionization history constraints from neural network based predictions of high-redshift quasar continua

ABSTRACT Observations of the early Universe suggest that reionization was complete by z ∼ 6, however, the exact history of this process is still unknown. One method for measuring the evolution of the neutral fraction throughout this epoch is via observing the Lyα damping wings of high-redshift quasars. In order to constrain the neutral fraction from quasar observations, one needs an accurate model of the quasar spectrum around Lyα, after the spectrum has been processed by its host galaxy but before it is altered by absorption and damping in the intervening intergalactic medium (IGM). In this paper, we present a novel machine learning approach, using artificial neural networks, to reconstruct quasar continua around Lyα. Our Quasar Spectra from Artificial Neural Network based predictive Regression Algorithm(QSANNdRA) improves the error in this reconstruction compared to the state-of-the-art principal component analysis (PCA) based model in the literature by 14.2 per cent on average, and provides an improvement of 6.1 per cent on average when compared to an extension thereof. In comparison with the extended PCA model, QSANNdRA further achieves an improvement of 22.1 per cent and 16.8 per cent when evaluated on low-redshift quasars most similar to the two high-redshift quasars under consideration, ULAS J1120+0641 at z = 7.0851 and ULAS J1342+0928 at z = 7.5413, respectively. Using our more accurate reconstructions of these two z > 7 quasars, we estimate the neutral fraction of the IGM using a homogeneous reionization model and find $\bar{x}_\mathrm{H\, \small{I}} = 0.25^{+0.05}_{-0.05}$ at z = 7.0851 and $\bar{x}_\mathrm{H\, \small{I}} = 0.60^{+0.11}_{-0.11}$ at z = 7.5413. Our results are consistent with the literature and favour a rapid end to reionization.

Ďurovčíková, Dominika↗

NASA NTRS2019

The Southern Ocean Sea Surface Temperature Response to Ozone Depletion: A Multi-Model Comparison

The effect of the Antarctic ozone hole extends downwards from the stratosphere, with clear signatures in surface weather patterns including a positive trend in the Southern Annular Mode (SAM). Several recent studies have used coupled climate models to investigate the impact of these changes on Southern Ocean sea surface temperature (SST), notably motivated by the observed cooling from the late 1970s. Here we examine the robustness of these model results through comparison of both previously published and new simulations. We focus on the calculation of â€˜climate response functions' (CRFs), transient responses to an instantaneous step-change in ozone concentrations. The CRF for most models consists of a rapid cooling of SST, followed by a slower warming trend. However, inter-model comparison reveals large uncertainties, such that even the sign of the impact of ozone depletion on historical SST, when reconstructed from the CRF, remains unconstrained. Comparison of these CRFs with SST responses to a hypothetical step-change in the SAM, inferred through lagged linear regression, shows broadly similar results. Causes of uncertainty are explored by examining relationships between model climatologies and their CRFs. The inter-model spread in CRFs can be reproduced by varying a single subgrid-scale mixing parameter within a single model. Antarctic sea-ice CRFs are also calculated: these do not generally exhibit the two-time-scale behavior of SST, suggesting a complex relationship between the two. Finally, by constraining model climatology-response relationships with observational values, we conclude that ozone depletion in unlikely to have been the primary driver of the observed SST cooling trend.

Climate response functions (CRF)↗

DOE OSTI2024

Frequency Nadir Constrained Unit Commitment for High Renewable Penetration Island Power Systems

The process of energy decarbonization in island power systems is accelerated due to the swift integration of inverter-based renewable energy resources (IBRs). The unique features of such systems, including rapid frequency changes resulting from potential generation outages or imbalances due to the unpredictability of renewable power, pose a significant challenge in maintaining the frequency nadir without external support. This paper presents a unit commitment (UC) model with data-driven frequency nadir constraints, including either frequency nadir or minimum inertia requirements, helping to limit frequency deviations after significant generator outages. The constraints are formulated using a linear regression model that takes advantage of real-world, year-long generation scheduling and dynamic simulation data. The efficacy of the proposed UC model is verified through a year-long simulation in an actual island power system using historical weather data. The alternative minimum inertia constraint, derived from actual system operation assumptions, is also evaluated. Findings demonstrate that the proposed frequency nadir constraint notably improves the system's frequency nadir under high photovoltaic (PV) penetration levels, albeit with a slight increase in generation costs, when compared to the alternative minimum inertia constraint.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2019

Biogeographic classification of streams using fish community– and trait–environment relationships

Abstract Aim Present a hybrid biogeographic and environmentally constrained clustering approach to classify ~853,000 stream reaches in the eastern United States. Examine the frequency of stream typologies in the landscape relative to anthropogenic stressors to identify potential stream conservation needs. Location Eastern United States. Methods Fish communities at 956 least‐disturbed sampling reaches were characterized using taxonomic and functional composition of native species. Environmental variables summarized within stream reaches included stream discharge, channel gradient, hydrologic regime, summer water temperature and boundaries of freshwater ecoregions. Multivariate regression trees were used to relate environmental variables to fish communities while simultaneously developing taxa‐ and trait‐specific biogeographic classifications. We then overlaid stream classifications with indices of anthropogenic disturbances to evaluate rarity and risk of complete loss of natural representation of stream types. Results Taxonomically based classes represented a combination of ecoregional boundaries and environmental gradients, whereas temperature gradients were most important in differentiating functional composition. An optimal taxonomy‐based classification contained 13 classes and explained 26.4% of community variation, and the optimal trait‐based classification contained 6 classes, explaining 25.2% of community variation. Overlaying class maps with an anthropogenic disturbance index revealed substantial variation in the severity of degradation among classes relative to their abundance in the landscape, especially rare classes (i.e., large rivers and cold headwater systems). Main conclusions We demonstrate the value of integrating local, regional and historical biogeographic factors with taxonomic and functional characteristics of freshwater biota to classify ecosystems based on mechanistic biota–environment relationships. This approach is important to addressing the challenge of developing stream classifications that reconcile the environmental drivers of community trait composition with the constraints of biogeographic regionality at large spatial scales. Furthermore, the spatially explicit and high‐resolution maps disseminated here can uniquely inform management decisions regarding anthropogenic threats to the biodiverse freshwaters of the eastern United States.

Troia, Matthew J.↗

DOE OSTI2026

Frequency-Nadir-Constrained Unit Commitment for Low-Inertia, High-IBR Island Power Systems [Slides]

14 SOLAR ENERGY↗

DOE OSTI2020

Probabilistic learning and updating of a digital twin for composite material systems

This paper presents an approach for characterizing and estimating statistical dependence between a large number of observables in a composite material system. Conditional regression is carried out using the estimated joint density function, permitting a systematic exploration of interdependence between fine scale and coarse observables that can be used for both prognosis and design of complex material systems. An example demonstrates the integration of experimental data with a computational database. The statistical approach is based on the probabilistic learning on manifolds recently developed by the authors. This approach leverages intrinsic structure detected through diffusion on graphs with projected stochastic differential equations to generate samples constrained to that structure.

42 ENGINEERING↗

DOE OSTI2019

Approximate Mixed-Integer Programming Solution with Machine Learning Technique and Linear Programming Relaxation

Security constrained unit commitment (SCUC) is solved in the day-ahead electricity market to determine electricity generation schedule for the following date. The associated mixed integer programming (MIP) problem is known computationally hard. Heuristics like rounding can be computationally efficient for solving large MIP problems however often produces lower quality solutions due to discrepancy between MIP solution and linear programming relaxation (LPR) solution, i.e., variables taking different discrete values between MIP and LPR solutions. In this work, we applied the machine learning-based algorithms—Classification and Regression Tree (CART) model and random forest (RF) model—to obtain an LPR-based approximation that close to an MIP solution without actually solving the MIP problem. By computing feature importance derived from the CART and RF models, we assessed the link between coefficients in the MIP model, the LPR solution, and the optimal solution of the MIP problem. We applied the CART and RF approaches to solve SCUC problems using day-ahead energy market cases from Midcontinent Independent System Operator. The results show that our method is computationally efficient and could achieve an approximation sufficiently close to the SCUC solution with <0.15% error and ~78% reduction in the discrepancies between LPR and MIP solutions.

Lin, Xinming↗