Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “regression models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments [https://arxiv.org/abs/2403.11872]. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative data sets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student t process regression. We apply Student t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via colouring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics and classic machine learning clustering algorithms.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

How reliable is distribution of relaxation times (DRT) analysis? A dual regression-classification perspective on DRT estimation, interpretation, and accuracy

The distribution of relaxation times (DRT) has gained increasing attention and adoption in recent years as a versatile method for analyzing electrochemical impedance spectroscopy (EIS) data obtained from complex devices like fuel cells, electrolyzers, and batteries. The DRT deconvolutes the impedance without a priori specification of a generative model, which is especially useful for interpretation and model selection when the governing principles of the system under study are not fully understood. However, DRT estimation is an ill-posed inversion problem that must be addressed with a subjective choice of regularization and tuning, which leaves substantial risk of misleading interpretations of EIS data. In this work, we suggest a new classification view of the DRT inversion to clarify DRT estimation and interpretation. We introduce a dual regression-classification framework that unifies the classification and regression views of the DRT inversion with wide-reaching implications for DRT analysis. The dual framework is employed to demonstrate a new kind of DRT inversion algorithm and develop novel evaluation metrics that capture previously ignored aspects of DRT accuracy. These approaches are applied to both synthetic data and experimental spectra collected from a protonic ceramic fuel cell and a lithium-ion battery to illustrate their broad utility. The dual inversion algorithm shows promising performance for accurate DRT estimation and autonomous model identification, while the dual evaluation approach produces metrics that meaningfully assess the strengths and risks of DRT algorithms. Here this work provides valuable insight for both practical application of the DRT to experimental data and further development of EIS analysis methods.

36 MATERIALS SCIENCE↗

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative datasets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student-t process regression. We apply Student-t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via coloring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics, and classic machine learning clustering algorithms.

97 MATHEMATICS AND COMPUTING↗

Suppressing the sample variance of DESI-like galaxy clustering with fast simulations

Ongoing and upcoming galaxy redshift surveys, such as the Dark Energy Spectroscopic Instrument (DESI) survey, will observe vast regions of sky and a wide range of redshifts. In order to model the observations and address various systematic uncertainties, N-body simulations are routinely adopted, however, the number of large simulations with sufficiently high mass resolution is usually limited by available computing time. Therefore, achieving a simulation volume with the effective statistical errors significantly smaller than those of the observations becomes prohibitively expensive. In this study, we apply the Convergence Acceleration by Regression and Pooling (CARPool) method to mitigate the sample variance of the DESI-like galaxy clustering in the AbacusSummit simulations, with the assistance of the quasi-N-body simulations FastPM. Based on the halo occupation distribution (HOD) models, we construct different FastPM galaxy catalogs, including the luminous red galaxies (LRGs), emission line galaxies (ELGs), and quasars, with their number densities and two-point clustering statistics well matched to those of AbacusSummit. We also employ the same initial conditions between AbacusSummit and FastPM to achieve high cross-correlation, as it is useful in effectively suppressing the variance. Our method of reducing noise in clustering is equivalent to performing a simulation with volume larger by a factor of 5 and 4 for LRGs and ELGs, respectively. We also mitigate the standard deviation of the LRG bispectrum with the triangular configurations k 2 = 2k 1 = 0.2 h Mpc -1 by a factor of 1.6. With smaller sample variance on galaxy clustering, we are able to constrain the baryon acoustic oscillations (BAO) scale parameters to higher precision. The CARPool method will be beneficial to better constrain the theoretical systematics of BAO, redshift space distortions (RSD) and primordial non-Gaussianity (NG).

79 ASTRONOMY AND ASTROPHYSICS↗

High-pressure ignition delay time measurements of a four-component gasoline surrogate and its high-level blends with ethanol and methyl acetate

Ignition delay times of a gasoline surrogate (iso-octane/toluene/n-heptane/1-hexene at 55/25/15/5% by liquid volume, developed in McCormick et al. 2017) with high-level bioblendstock-gasoline surrogate blends (50% and 85% biofuel by liquid volume of each ethanol and methyl acetate) were collected behind reflected shock waves. Post-reflected-shock temperatures ranged from 968 to 1361 K at pressures of about 4, 10, 25, and 50 atm. Data were collected for real fuel–air mixtures at fuel lean and stoichiometric conditions (φ = 0.5, 1.0) with focus on the more-practical, fuel-lean conditions. Ignition delay times were measured from OH* chemiluminescence around 307 nm using an endwall diagnostic. The RON and MON of the surrogate were 90.3 and 84.7, respectively. Ethanol was chosen due to its wide use in flex-fuel vehicles and availability, while also providing an increased octane rating. Methyl acetate was chosen for its especially high octane rating to investigate the effect of an extreme case. To validate the gasoline surrogate, the data are compared to real gasoline (RD387) and several gasoline surrogate experiments from the literature. Using the wide range of pressures studied, the gasoline surrogate’s pressure dependence was quantified to account for test-to-test variations using regression analysis. Similarly, a global correlation for gasoline and its surrogates was developed using all available data from the literature. Two modern chemical kinetics models targeting gasoline and its surrogates are compared to the ignition delay time measurements. These new high-pressure, high-bioblendstock concentration tests provide required chemical kinetic data for optimizing fuel and engine design.

42 ENGINEERING↗

Insights into Prismatic Loop Formation in Irradiated Fe–Cr Alloys from Hypothesis-Driven Active Learning and Causal Analysis

Neutron and electron irradiation experimental studies conducted on body-centered cubic Fe and Fe–Cr alloys have established two prismatic dislocation loop populations, which have Burgers vectors of either a/2$\langle$111$\rangle$ or a$\langle$100$\rangle$. Here, the loop formation depends on factors such as dose (D), dose rate (D rt ), temperature (T), chromium content (Cr%), and other alloying elements. Hence, it is important to understand how irradiation-induced dislocation loops evolve conditional upon the loop characteristics, such as loop density (DD), average loop size d̅, and irradiation parameters (D, D rt , T, and irradiation type), which is still an active area of research. To understand these complex structure–property relationships, machine learning (ML) is employed in a three-step approach. This includes imputing missing data with a k-nearest neighbor, generating functionalized features, and assessing feature importance with random forest classification and regression. Physics-based features are incorporated in a hypothesis-driven active learning scheme to overcome data unavailability challenges. Insights obtained from ML models (i) to categorize dislocation loop types, show the highest correlation with d̅; (ii) Log(DD), obtained through mathematical formulations involving D, Cr%, d̅, and T (e.g., Log(DD) ~ D + exp(-Cr%) + 1/d̅ and log(DD) ~ D + exp(-Cr%) + 1/T). Hypothesis-driven active learning is able to predict Log(DD) in which the experimental date is not known. Causal models verify cause–effect relationships for dislocation loop classification and irradiation factors in FeCr alloys.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Probing multi-dimensional composition spaces in search of strong metallic alloys

Refractory complex concentrated alloys (RCCA) offer exceptionally high-temperature strength compared to pure metals and dilute alloys, but predictive theory for RCCA design is lacking. We present large-scale molecular Dynamics (MD) simulations of crystal plasticity to explore alloy compositions for maximum mechanical strength, focusing on Fe-Ta-W and Nb-Ta-Mo-W alloy families modeled with Embedded Atom Model (EAM) and Spectral Neighbor Analysis Potentials (SNAP). To efficiently guide the search for strong alloy compositions, we employ iterative optimization using Gaussian process regression. Many simulated RCCA compositions exhibit pronounced cocktail strengthening, with strengths surpassing their strongest constituent metal, tungsten. Contrary to expectations, the highest strength is found on binary edges of the RCCA composition space. Detailed analyses of atomistic simulations reveal that, similar to pure BCC metals, plastic response in RCCA is primarily governed by screw dislocations. However, at large strains, dislocation multiplication and interactions (Taylor hardening) become the dominant mechanisms contributing to RCCA strength.

Materials science↗

Learning the Temporal Effect in Infrared Thermal Videos With Long Short-Term Memory for Quality Prediction in Resistance Spot Welding

With the advances of sensing technology, in-situ infrared thermal videos can be collected from Resistance Spot Welding (RSW) processes. Each video records the formulation process of a weld nugget. The nugget evolution creates a “temporal effect” across the frames, which can be leveraged for real-time, nondestructive evaluation (NDE) of the weld quality. Currently, quality prediction with imaging data mainly focuses on optical feature extraction with Convolutional Neural Network (CNN) but does not make the most of such temporal effect. In this study, pixels corresponding to critical locations on the weld nugget surface are extracted from a video to form multivariate time series (MTS). Multivariate Adaptive Regression Splines (MARS) is used in MTS processing to remove noisy signals related to uninformative frames. A Stacked Long Short-Term Memory (LSTM) model is developed to learn from the processed MTS and then predicts weld nugget size and thickness in real-time NDE. Results from a case study on RSW of Boron steel demonstrates the improvement in prediction accuracy and computational time with the proposed method, as compared to CNN-based weld quality prediction.

Guo, Shenghan↗

Interpretable Net Load Forecasting Using Smooth Multiperiodic Features

We consider the problem of forecasting net load over a horizon such as one day, using a trailing window of past net load values as well as date and time. We focus on three variations on this problem: point forecasts, marginal quantile forecasts, and generating conditional samples of the future value. We propose a method that relies on linear regression using some custom engineered time-based features to capture multiple periodicities, such as daily, weekly, and seasonal, and their interactions. Our proposed models are readily interpretable, and rely on efficient and reliable convex optimization [1] to fit. We illustrate our method on four years worth of hourly net load data, comparing predictions made with various subsets of the features.

Ogut, Mehmet G↗

Assessing heterogeneity of patient and health system delay among TB in a population with internal migrants in China

Backgrounds The diagnostic delay of tuberculosis (TB) contributes to further transmission and impedes the implementation of the End TB Strategy. Therefore, we aimed to describe the characteristics of patient delay, health system delay, and total delay among TB patients in Shanghai, identify areas at high risk for delay, and explore the potential factors of long delay at individual and spatial levels. Method The study included TB patients among migrants and residents in Shanghai between January 2010 and December 2018. Patient and health system delays exceeding 14 days and total delays exceeding 28 days were defined as long delays. Time trends of long delays were evaluated by Joinpoint regression. Multivariable logistic regression analysis was employed to analyze influencing factors of long delays. Spatial analysis of delays was conducted using ArcGIS, and the hierarchical Bayesian spatial model was utilized to explore associated spatial factors. Results Overall, 61,050 TB patients were notified during the study period. Median patient, health system, and total delays were 12 days (IQR: 3–26), 9 days (IQR: 4–18), and 27 days (IQR: 15–43), respectively. Migrants, females, older adults, symptomatic visits to TB-designated facilities, and pathogen-positive were associated with longer patient delays, while pathogen-negative, active case findings and symptomatic visits to non-TB-designated facilities were associated with long health system delays (LHD). Spatial analysis revealed Chongming Island was a hotspot for patient delay, while western areas of Shanghai, with a high proportion of internal migrants and industrial parks, were at high risk for LHD. The application of rapid molecular diagnostic methods was associated with reduced health system delays. Conclusion Despite a relatively shorter diagnostic delay of TB than in the other regions in China, there was vital social-demographic and spatial heterogeneity in the occurrence of long delays in Shanghai. While the active case finding and rapid molecular diagnosis reduced the delay, novel targeted interventions are still required to address the challenges of TB diagnosis among both migrants and residents in this urban setting.

Sun, Ruoyao↗

Verbal Learning and Memory Deficits across Neurological and Neuropsychiatric Disorders: Insights from an ENIGMA Mega Analysis

Deficits in memory performance have been linked to a wide range of neurological and neuropsychiatric conditions. While many studies have assessed the memory impacts of individual conditions, this study considers a broader perspective by evaluating how memory recall is differentially associated with nine common neuropsychiatric conditions using data drawn from 55 international studies, aggregating 15,883 unique participants aged 15–90. The effects of dementia, mild cognitive impairment, Parkinson’s disease, traumatic brain injury, stroke, depression, attention-deficit/hyperactivity disorder (ADHD), schizophrenia, and bipolar disorder on immediate, short-, and long-delay verbal learning and memory (VLM) scores were estimated relative to matched healthy individuals. Random forest models identified age, years of education, and site as important VLM covariates. A Bayesian harmonization approach was used to isolate and remove site effects. Regression estimated the adjusted association of each clinical group with VLM scores. Memory deficits were strongly associated with dementia and schizophrenia (p < 0.001), while neither depression nor ADHD showed consistent associations with VLM scores (p > 0.05). Differences associated with clinical conditions were larger for longer delayed recall duration items. By comparing VLM across clinical conditions, this study provides a foundation for enhanced diagnostic precision and offers new insights into disease management of comorbid disorders.

Neurosciences & Neurology↗

$\mathrm{SageNet}$: Fast Neural Network Emulation of the Stiff-amplified Gravitational Waves from Inflation

Accurate modeling of the inflationary gravitational waves (GWs) requires time-consuming, iterative numerical integrations of differential equations to take into account their backreaction on the expansion history. To improve computational efficiency while preserving accuracy, we present the Stiff-amplified Gravitational-wave Emulator Network (SageNet), a deep learning framework designed to replace conventional numerical solvers (code available at https://github.com/YifangLuo/SageNet). SageNet employs a long short-term memory architecture to emulate the present-day energy density spectrum of the inflationary GWs with possible stiff amplification, Ω GW (f). Trained on a data set of 25,689 numerically generated solutions, SageNet allows accurate reconstructions of Ω GW (f) and generalizes well to a wide range of cosmological parameters; 90.9% of the test emulations with randomly distributed parameters exhibit errors of under 4%. In addition, SageNet demonstrates its ability to learn and reproduce the artificial, adaptive sampling patterns in numerical calculations, which implement denser sampling of frequencies around changes in spectral indices in Ω GW (f). The dual capability of learning both physical and artificial features of the numerical GW spectra establishes SageNet as a robust alternative to exact numerical methods. Finally, our benchmark tests show that SageNet reduces the computation time from tens of seconds to milliseconds, achieving a speedup of ∼10 4 times over standard CPU-based numerical solvers with the potential for further acceleration on GPU hardware. These capabilities make SageNet a powerful tool for accelerating Bayesian inference procedures for extended cosmological models. In a broad sense, the SageNet framework offers a fast, accurate, and generalizable solution to modeling cosmological observables whose theoretical predictions demand costly differential equation solvers.

Astronomy data modeling↗

Uncertainty quantification and sensitivity analysis for SPERT III E-core reactivity measurement benchmarking

The Special Power Excursion Reactor Test (SPERT) III E-core experiment is important because it provides critical data on reactor behavior under significant reactivity insertions, which is essential for validating computer simulations and ensuring the safety of modern light water reactors. Its design similarities to contemporary reactors make it a valuable resource for understanding and mitigating extreme hazards in nuclear operations. The current study details the application of formal parametric uncertainty quantification and sensitivity analysis to a model of the SPERT-III E-core model for zero power reactivity benchmarking. Additionally, the reactivity impact from various modeling assumptions is quantified. Overall, an conservative estimate for an uncertainty in k$_{\text{eff}}$ of $\pm$1257 was observed. A less conservative, more realistic, uncertainty estimate of $\pm$1096 pcm can be justified by the potential for various parametric uncertainties to become negligible when sampled independently across the ~1400 pins in the core. The experimental results fall within both of these uncertainty bounds. Standardized regression coefficient as well as Sobol indices are used to identify the guide tube thicknesses as the primary contributors to the uncertainty in k$_{\text{eff}}$. Overall, this study provides information on how the uncertainties in input parameters and modeling methods impact simulated k$_{\text{eff}}$ values and can be used to aid model building efforts for future code validation with the SPERT-III E-core experiment.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

The Oceanic Barrier Layer in the Eastern Indian Ocean as a Predictor for Rainfall Over Indonesia and Australia

Barrier layers in the tropics trap heat in a shallow and stable near-surface layer and limit entrainment of cooler water from below. Both processes act to increase sea surface temperature and enhance atmospheric convection. The high resolution fully coupled pre-industrial Energy Exascale Earth System Model version 0 (E3SMv0) is used to investigate the relationship between barrier layers in the eastern Indian Ocean during the wet season with local atmospheric convection and remote rainfall. A partial least squares regression reveals a significant relationship between Australasian rainfall and the barrier layer thickness (BLT) west of Sumatra, occurring one month earlier. Furthermore, the largest positive regression coefficients are over northern Australia. The region west of Sumatra is strategically located where the East-Asian monsoon moisture flows toward northern Australia. Thickening of the west Sumatra BLT intensifies evaporation and local convection and amplifies the moisture transported to Australia acting to increase the terrestrial rainfall.

58 GEOSCIENCES↗

Predicting Drug Effects from High-dimensional Asymmetric Drug Data Sets using Graph Neural Networks: A Comprehensive Analysis of Multi-target Drug Effect Prediction

Graph neural networks (GNNs) have emerged as one of the most effective Machine learning (ML) techniques for drug effect prediction from drug molecular graphs. Despite having immense potential, GNN models lack performance when using data sets that contain high dimensional asymmetrically co-occurrent drug effects as targets with complex correlations between them. Training individual learning models for each drug effect and incorporating every prediction result for a wide spectrum of drug effects is beyond practicality. Such an implication provides a testbed to address this challenge as multi-target prediction problems, aiming to predict all drug effects at a time. We develop standard and hybrid graph neural networks (GNNs)to perform two separate tasks that are multi-regression for continuous values and multi-label classification for categorical values contained in our data sets. Since this step makes the target data even more sparse and introduces asymmetric label co-occurrence, the learning of multi-label classification models becomes difficult and heavily impacts the GNN's performance. To address these challenges, we propose a new data oversampling technique to improve multi-label classification performances on all the given imbalanced molecular graph data sets. Using the technique, we improve the data imbalance ratio of the drug effects better than before while protecting the data set's integrity. Finally, we evaluate multi-label classification performance using the best-performant hybrid GNN model on all the oversampled data sets obtained from the proposed oversampling technique. These results outperform those of other ML models including GNN models when they are trained on the original data sets or oversampled data sets using MLSMOTE (a well-known oversampling technique) in all evaluation metrics precision, recall, and F1 score by a significant margin.

Bose, Avishek [ORNL]↗