Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “generalizability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Identifying atmospheric rivers and their poleward latent heat transport with generalizable neural networks: ARCNNv1

Abstract. Atmospheric rivers (ARs) are extreme weather events that can alleviate drought or cause billions of US dollars in flood damage. By transporting significant amounts of latent energy towards the poles, they are crucial to maintaining the climate system's energy balance. Since there is no first-principle definition of an AR grounded in geophysical fluid mechanics, AR identification is currently performed by a multitude of expert-defined, threshold-based algorithms. The variety of AR detection algorithms has introduced uncertainty into the study of ARs, and the thresholds of the algorithms may not generalize to new climate datasets and resolutions. We train convolutional neural networks (CNNs) to detect ARs while representing this uncertainty; we name these models ARCNNs. To detect ARs without requiring new labeled data and labor-intensive AR detection campaigns, we present a semi-supervised learning framework based on image style transfer. This framework generalizes ARCNNs across climate datasets and input fields. Using idealized and realistic numerical models, together with observations, we assess the performance of the ARCNNs. We test the ARCNNs in an idealized simulation of a shallow-water fluid in which nearly all the tracer transport can be attributed to AR-like filamentary structures. In reanalysis and a high-resolution climate model, we use ARCNNs to calculate the contribution of ARs to meridional latent heat transport, and we demonstrate that this quantity varies considerably due to AR detection uncertainty.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Implications of stop-and-go traffic on training learning-based car-following control

Learning-based car-following control (LCC) of connected and autonomous vehicles (CAVs) is gaining significant attention with the advancement of computing power and data accessibility. While the flexibility and large model capacity of model-free architecture enable LCC to potentially outperform the model-based car-following (CF) model in improving traffic efficiency and mitigating congestion, the generalizability of LCC for traffic conditions different from the training environment/dataset is not well-understood. Herein, this study seeks to explore the impact of stop-and-go traffic in the training dataset on the generalizability of LCC. It uses the characteristics of lead vehicle trajectories to describe stop-and-go traffic, and links the theory of identifiability (i.e., obtaining a unique parameter estimation result using sensor measurements) to the generalizability of behavior cloning (BC) and policy-based deep reinforcement learning (DRL). Correspondingly, the study shows theoretically that: (i) stop-and-go traffic can enable the property of identifiability and enhance the control performance of BC-based LCC in different traffic conditions; (ii) stop-and-go traffic is not necessary for DRL-based LCC to generalize to different traffic conditions; (iii) DRL-based LCC trained with only constant-speed lead vehicle trajectories (not sufficient to ensure identifiability) can be generalized to different traffic conditions; and (iv) stop-and-go traffic increases variance in the training dataset, which improves the convergence of parameter estimation while negatively impacting the convergence of DRL to the optimal control policy. Numerical experiments validate the above findings, illustrating that BC-based LCC entails comprehensive training datasets for generalizing to different traffic conditions, while DRL-based LCC can achieve generalization with simple free-flow traffic training environments. This further suggests DRL as a more promising and cost-effective LCC approach to reduce operational costs, mitigate traffic congestion, and enhance safety and mobility, which can accelerate the deployment and acceptance of CAVs.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Say where you sample: Increasing site selection transparency in urban ecology

Urban ecological studies have the potential to extend our understanding of socio-ecological systems beyond that of an individual city or region. Cross-comparative empirical work and synthesis are imperative to develop a general urban ecological theory. This can be achieved only if studies are replicable and generalizable. Transparency in methods reporting facilitates generalizability and replicability by documenting the decisions scientists make during the various steps of research design; this is particularly true for sampling design and selection because of their impact on both internal and external validity and the potential to unintentionally introduce bias. Three interdependent aspects of sample design are study sample selection (e.g., specific organisms, soils, or water), sample specification (measurement of specific variable of interest), and site selection (locations sampled). Of these, documentation of site selection—the where component of sample design—is underrepresented in the urban ecology literature. Using a stratified random sample of 158 papers from 12 major urban ecology journals, we investigated how researchers selected study sites in urban ecosystems and evaluated whether their site selection methods were transparent. We extracted data from these papers using a 50-question, theory-based questionnaire and a multiple-reviewer approach. Our sample represented almost 45 years of urban ecology research across 40 different countries. We found that more than 80% of the papers read were not transparent in their site selection methodology. We do not believe site selection methods are replicable for 70% of the papers read. Key weaknesses include incomplete descriptions of populations and sampling frames, urban gradients, sample selection methods, and property access. Low transparency in reporting the where methodology limits urban ecologists’ ability to assess the internal and external validity of studies’ findings and to replicate published studies; it also limits the generalizability of existing studies. The challenges of low transparency are particularly relevant in urban ecology, a field where standard protocols for site selection and delineation are still being developed. These limitations interfere with the fields’ ability to build theory and inform policy. We conclude by offering a set of recommendations to increase transparency, replicability, and generalizability.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A cross-study analysis of drug response prediction in cancer cell lines

Abstract To enable personalized cancer treatment, machine learning models have been developed to predict drug response as a function of tumor and drug features. However, most algorithm development efforts have relied on cross-validation within a single study to assess model accuracy. While an essential first step, cross-validation within a biological data set typically provides an overly optimistic estimate of the prediction performance on independent test sets. To provide a more rigorous assessment of model generalizability between different studies, we use machine learning to analyze five publicly available cell line-based data sets: National Cancer Institute 60, ancer Therapeutics Response Portal (CTRP), Genomics of Drug Sensitivity in Cancer, Cancer Cell Line Encyclopedia and Genentech Cell Line Screening Initiative (gCSI). Based on observed experimental variability across studies, we explore estimates of prediction upper bounds. We report performance results of a variety of machine learning models, with a multitasking deep neural network achieving the best cross-study generalizability. By multiple measures, models trained on CTRP yield the most accurate predictions on the remaining testing data, and gCSI is the most predictable among the cell line data sets included in this study. With these experiments and further simulations on partial data, two lessons emerge: (1) differences in viability assays can limit model generalizability across studies and (2) drug diversity, more than tumor diversity, is crucial for raising model generalizability in preclinical screening.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

GraMeR: Gra ph Me ta R einforcement learning for multi-objective influence maximization

Influence maximization (IM) is a combinatorial problem of identifying a subset of seed nodes in a network (graph), which when activated, provide a maximal spread of influence in the network for a given diffusion model and a budget for seed set size. IM has numerous applications such as viral marketing, epidemic control, sensor placement and other network-related tasks. However, its practical uses are limited due to the computational complexity of current algorithms. Recently, deep reinforcement learning has been leveraged to solve IM in order to ease the computational burden. However, there are serious limitations in current approaches, including narrow IM formulation that only consider influence via spread and ignore self-activation, low scalability to large graphs, and lack of generalizability across graph families leading to a large running time for every test network. In this work, we address these limitations through a unique approach that involves: (1) Formulating a generic IM problem as a Markov decision process that handles both intrinsic and influence activations; (2)incorporating generalizability via meta-learning across graph families. There are previous works that combine deep reinforcement learning with graph neural network, but this work solves a more realistic IM problem and incorporates generalizability across graphs via meta reinforcement learning. Extensive experiments are carried out in various standard networks to validate performance of the proposed Graph Meta Reinforcement learning (GraMeR) framework. Finally, the results indicate that GraMeR is multiple orders faster and generic than conventional approaches when applied on small to medium scale graphs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Chemistry Informed Machine Learning-Based Heat Capacity Prediction of Solid Mixed Oxides

Knowing heat capacity is crucial for modeling temperature changes with the absorption and release of heat and for calculating the thermal energy storage capacity of oxide mixtures with energy applications. The current prediction methods (ab initio simulations, computational thermodynamics, and the Neumann–Kopp rule) are computationally expensive, not fully generalizable, or inaccurate. Machine learning has the potential of being fast, accurate, and generalizable, but it has been scarcely used to predict mixture properties, particularly for mixed oxides. Here, we demonstrate a method for the generalizable prediction of heat capacity of solid oxide pseudobinary mixtures using heat capacity data obtained from computational thermodynamics and descriptors from ab initio databases. Further, models trained through this workflow achieved an error (mean absolute error of 0.43 J mol –1 K –1 ) lower than the uncertainty in differential scanning calorimetry measurements, and the workflow can be extended to predict other properties derived from the Gibbs free energy and for higher-order oxide mixtures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Model fusion with physics-guided machine learning: Projection-based reduced-order modeling

The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach, especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.

42 ENGINEERING↗
DOE OSTI2026

Ecological Insights from Transferable Plant Biomass Mapping across the Arctic using High-resolution Structure-from-Motion and LiDAR Data

Warmer temperatures, permafrost thaw, and increased wildfire activity are driving rapid ecological change across the Arctic, significantly altering plant productivity and aboveground biomass (AGB). These rapid changes highlight the urgent need to improve monitoring of vegetation dynamics in the Earth’s northern ecosystems, where high spatiotemporal heterogeneity occurs at scales finer than those captured by traditional satellite observations. The growing use of Unoccupied Aerial Systems (UASs) presents an opportunity to overcome this limitation. Yet, the diversity of UAS platforms, sensors, and data collection and processing workflows presents challenges for developing standardized, generalizable approaches. To address this challenge, we compiled 672 AGB plots co-located with 183 UAS-based Structure-from-Motion (SfM) or Light Detection and Ranging (LiDAR) surveys collected across the Arctic. Here, we: (1) evaluated the generalizability of UAS-derived canopy structure derived from high-resolution SfM and LiDAR for estimating AGB, (2) assessed scaling errors and their sources in two recent satellite-based AGB products derived from Landsat and MODIS, and (3) demonstrated the use of high-resolution AGB maps to quantify biomass variation across tundra plant functional types (PFTs) and to monitor post-fire recovery. Our results show that both SfM and LiDAR accurately captured AGB and its variability across tundra PFTs using a Random Forest (RF) model (overall RMSE: 0.336 kg/m2), with mapping performance varying slightly by region and data source. Using UAS-derived AGB maps as a benchmark, we identified systematic biases in satellite-derived AGB products, largely attributable to the magnitude of AGB and structural heterogeneity within coarse-resolution pixels. Applying our model to repeat UAS surveys following a tundra fire on Seward Peninsula, we observed rapid AGB recovery in non-shrub patches, with biomass recovering to pre-fire levels within 2 years. In contrast, shrub patches recovered more slowly, with AGB gains continuing over 2–4 years through both in-patch growth and lateral expansion (via dispersal) into remaining burned areas. Overall, these findings demonstrate the generalizability of UAS-based SfM and LiDAR data for estimating tundra AGB and highlight the potential of our approach to be broadly applied to generate high-quality AGB data for ecological monitoring and model benchmarking across the Arctic.

Yang, Daryl [ORNL] (ORCID:0000000317057823)↗
DOE OSTI2024

Invariant Features for Accurate Predictions of Quantum Chemical UV-vis Spectra of Organic Molecules

Including invariance of global properties of a phys-ical system as an intrinsic feature in graph neural networks (GNNs) enhances the model's robustness and generalizability and reduces the amount of training data required to obtain a desired accuracy for predictions of these properties. Existing open source GNN libraries construct invariant features only for specific GNN architectures. This precludes the generalization of invariant features to arbitrary message passing neural network (MPNN) layers which, in turn, precludes the use of these libraries for new, user-specified predictive tasks. To address this limitation, we implement invariant MPNNs into the flexible and scalable HydraGNN architecture. HydraGNN enables a seamless switch between various MPNNs in a unified layer sequence and allows for a fair comparison between the predictive performance of different MPNNs. We trained this enhanced HydraGNN archi-tecture on the ultraviolet-visible (UV-vis) spectrum of GDB-9 molecules, a feature that describes the molecule's electronic exci-tation modes, computed with time-dependent density functional tight binding (TD-DFTB) and available open source through the GDB-9-Ex dataset. We assess the robustness (i.e., accuracy and generalizability) of the predictions obtained using different invariant MPNNs with respect to different values of the full width at half maximum (FWHM) for the Gaussian smearing of the theoretical peaks. Our numerical results show that incorporating invariance in the HydraGNN architecture significantly enhances both accuracy and generalizability in predicting UV-vis spectra of organic molecules.

Baker, Justin↗
DOE OSTI2024

Establishing performance metrics for quantitative non-targeted analysis: a demonstration using per- and polyfluoroalkyl substances

Abstract Non-targeted analysis (NTA) is an increasingly popular technique for characterizing undefined chemical analytes. Generating quantitative NTA (qNTA) concentration estimates requires the use of training data from calibration “surrogates,” which can yield diminished predictive performance relative to targeted analysis. To evaluate performance differences between targeted and qNTA approaches, we defined new metrics that convey predictive accuracy, uncertainty (using 95% inverse confidence intervals), and reliability (the extent to which confidence intervals contain true values). We calculated and examined these newly defined metrics across five quantitative approaches applied to a mixture of 29 per- and polyfluoroalkyl substances (PFAS). The quantitative approaches spanned a traditional targeted design using chemical-specific calibration curves to a generalizable qNTA design using bootstrap-sampled calibration values from “global” chemical surrogates. As expected, the targeted approaches performed best, with major benefits realized from matched calibration curves and internal standard correction. In comparison to the benchmark targeted approach, the most generalizable qNTA approach (using “global” surrogates) showed a decrease in accuracy by a factor of ~4, an increase in uncertainty by a factor of ~1000, and a decrease in reliability by ~5%, on average. Using “expert-selected” surrogates ( n = 3) instead of “global” surrogates ( n = 25) for qNTA yielded improvements in predictive accuracy (by ~1.5×) and uncertainty (by ~70×) but at the cost of further-reduced reliability (by ~5%). Overall, our results illustrate the utility of qNTA approaches for a subclass of emerging contaminants and present a framework on which to develop new approaches for more complex use cases. Graphical Abstract

Pu, Shirley (ORCID:0000000201223797)↗
DOE OSTI2023

Building thermal dynamics modeling with deep transfer learning using a large residential smart thermostat dataset

Understanding thermal dynamics and obtaining the computational model of residential buildings enable its scaled application in energy retrofits, control optimization and decarbonization. In this paper, we present a deep learning approach to model building thermal dynamics with smart thermostat data collected from residential buildings, with the goal to investigate model generalizability. In the first stage, we developed and compared different Deep Learning architectures including Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) models and CNN-LSTM to predict indoor air temperature in a multi-step time horizon. In the second stage, we implemented a Transfer Learning (TL) process, which aims to improve the prediction performance on a new set of buildings (targets), exploiting the knowledge of related or similar buildings (sources). Different TL strategies and source model identification methods were investigated. The study showed that the CNN-LSTM performed the best among the architectures compared, with an average Mean Absolute Error (MAE) of 0.26 °C for one-hour-ahead (twelve 5-min future steps) predictions. Furthermore, the results showed that freezing the LSTM layer and fine-tuning the other layers of the CNN-LSTM achieved the best performance among four TL strategies, which further improved the performance with respect to a machine learning approach by 10%, and proving the effectiveness and generalizability of the proposed approach. A comparison of three different source model identification methods showed that randomly selecting source models constrained by similar building characteristics can provide good TL performance while retaining simplicity comparing with other quantitative source identification methods.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗