Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “hierarchical models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution

A central problem in biology is to understand how organisms evolve and adapt to their environment by acquiring variations in the observable characteristics or traits of species across the tree of life. With the growing availability of large-scale image repositories in biology and recent advances in generative modeling, there is an opportunity to accelerate the discovery of evolutionary traits automatically from images. Toward this goal, we introduce Phylo-Diffusion, a novel framework for conditioning diffusion models with phylogenetic knowledge represented in the form of HIERarchical Embeddings (HIER-Embeds). We also propose two new experiments for perturbing the embedding space of Phylo-Diffusion: trait masking and trait swapping, inspired by counterpart experiments of gene knockout and gene editing/swapping. Our work represents a novel methodological advance in generative modeling to structure the embedding space of diffusion models using tree-based knowledge. Our work also opens a new chapter of research in evolutionary biology by using generative models to visualize evolutionary changes directly from images. We empirically demonstrate the usefulness of Phylo-Diffusion in capturing meaningful trait variations for fishes and birds, revealing novel insights about the biological mechanisms of their evolution. (Model and code can be found at imageomics.github.io/phylo-diffusion)

Khurana, Mridul↗
DOE OSTI2023

Insights from application of a hierarchical spatio-temporal model to an intensive urban black carbon monitoring dataset

Existing regulatory pollutant monitoring networks rely on a small number of centrally located measurement sites that are purposefully sited away from major emission sources. While informative of general air quality trends regionally, these networks often do not fully capture the local variability of air pollution exposure within a community. Recent technological advancements have reduced the cost of sensors, allowing air quality monitoring campaigns with high spatial resolution. The 100×100 black carbon (BC) monitoring network deployed 100 low-cost BC sensors across the 15 km 2 West Oakland, CA community for 100 days in the summer of 2017, producing a nearly continuous site-specific time series of BC concentrations which we aggregated to one-hour averages. Leveraging this dataset, we employed a hierarchical spatio-temporal model to accurately predict local spatio-temporal concentration patterns throughout West Oakland, at locations without monitors (average cross-validated hourly temporal R 2 =0.60). Using our model, we identified spatially varying temporal pollution patterns associated with small-scale geographic features and proximity to local sources. In a sub-sampling analysis, here we demonstrated that fine scale predictions of nearly comparable accuracy can be obtained with our modeling approach by using ~30% of the 100×100 BC network supplemented by a shorter-term high-density campaign.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Collaborative Distribution System Restoration Planning and Real-Time Dispatch Considering Behind-the-Meter DERs

Modern distribution systems are becoming more resilient against power outages thanks to the flexible generation capacity provided by the increasing integration of distributed energy resources (DERs). Compared to utility-scale DERs, which can be directly controlled by system operators, small-scale, behind-the-meter DERs are much more difficult to coordinate despite being the vast majority. Two major obstacles to the implementation of DERs for service restoration are the coordination of behind-the-meter DERs and the management of uncertainty. In this paper, a novel distribution system restoration strategy is proposed to overcome these two obstacles. The flexibility of be-hind-the-meter DERs is quantified and integrated into the restoration planning models within a hierarchical structure. The model predictive control technique is employed to enable adaptive restoration planning solution adjustments to accommodate fault scenarios and forecast errors. Besides, real-time power dispatch models are proposed to maintain power balance using available DER capability. Comprehensive integration of adaptive restoration planning schemes and real-time dispatch results are achieved through a collaborative restoration framework. Finally, the effectiveness of the proposed restoration strategy is verified through a modified IEEE 123-bus system and a real-world utility system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Engine-in-the-Loop Study of a Hierarchical Predictive Online Controller for Connected and Automated Heavy-Duty Vehicles

This paper presents a cohesive set of engine-in-the-loop (EIL) studies examining the use of hierarchical model-predictive control for fuel consumption minimization in a class-8 heavy-duty truck intended to be equipped with Level-1 connectivity/automation. This work is motivated by the potential of connected/automated vehicle technologies to reduce fuel consumption in both urban/suburban and highway scenarios. The authors begin by presenting a hierarchical model-predictive control scheme that optimizes multiple chassis and powertrain functionalities for fuel consumption. These functionalities include: vehicle routing, arrival/departure at signalized intersections, speed trajectory optimization, platooning, predictive optimal gear shifting, and engine demand torque shaping. The primary optimization goal is to minimize fuel consumption, but the hierarchical controller explicitly accounts for other key objectives/constraints, including operator comfort and safe inter-vehicle spacing. This work is experimentally experimentally validated via a sequence of EIL studies intended for evaluating the computational costs and fuel savings associated with these algorithms. These EIL studies involve the closed-loop validation of the proposed control strategies, both individually and combined. These studies show that this hierarchy of algorithms is capable of running online, with the round-trip communication delay inherent in EIL simulation being one of the key factors affecting the EIL results. Moreover, the EIL studies are encouraging, both in terms of the successful hierarchical integration of the underlying algorithms and also in the resulting fuel savings seen in the EIL tests. In particular, the EIL results suggest that an aggressive overall goal of reducing vehicle fuel consumption by 15-20% or more is potentially achievable, especially in urban/suburban scenarios.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2023

Effects of surface vacancy-adatom pair formation on PFC tungsten surface morphological response

Abstract We report a simulation study of the effect of He-irradiation-induced surface vacancy-adatom pair formation on the surface morphological evolution of plasma-facing component (PFC) tungsten and examine a number of factors that impact such evolution. Our analysis is based on self-consistent dynamical simulations according to an atomistically-informed, continuum-scale surface evolution model that has been developed following a hierarchical multiscale modeling strategy and can access the spatiotemporal scales of relevance to fuzz formation. The model accounts for the flux of surface adatoms generated as a result of the surface vacancy-adatom pair formation effect upon He implantation, which contributes to the anisotropic growth of surface nanostructural features due to the different rates of adatom diffusion along and across step edges of islands on the tungsten surface. We have carried out atomic-scale computations of optimal diffusion pathways along and across island step edges on the W(110) surface and calculated Ehrlich–Schwoebel (ES) barriers in adatom diffusion along and across such step edges. This aspect of surface adatom diffusion contributes to anisotropic surface atomic fluxes, terrace and step diffusive currents, and has been incorporated into our PFC surface evolution model, which predicts the formation of preferentially aligned nanoridge stripe patterns on the PFC surface. We establish that these anisotropic diffusive currents accelerate nanotendril growth on the PFC surface and the onset of surface nanostructure pattern formation. We also explore systematically the dependence of the PFC surface morphological response on the surface temperature and He ion incident flux, characterize in detail the resulting surface topographies and growth kinetics, and compare the predicted surface morphologies with experimental observations. Our simulation predictions for the emerging surface nanostructure patterns under certain plasma exposure conditions are consistent with experimental findings in the literature.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Does the leaf economic spectrum hold within plant functional types? A Bayesian multivariate trait meta-analysis

The leaf economic spectrum is a widely-studied axis of plant trait variability that defines a trade-off between leaf longevity and productivity. While this has been investigated at the global scale, where it is robust, and at local scales, where deviations from it are common, it has received less attention at the intermediate scale of PFTs. We investigated whether global leaf economic relationships are also present within the scale of plant functional types (PFTs) commonly used by Earth System models, and the extent to which this global-PFT hierarchy can be used to constrain trait estimates. We developed a hierarchical multivariate Bayesian model that assumes separate means and covariance structures within and across PFTs and fit this model to seven leaf traits from the TRY database related to leaf longevity, morphology, biochemistry, and photosynthetic metabolism. Although patterns of trait covariation were generally consistent with the leaf economic spectrum, we found three approximate tiers to this consistency. Relationships among morphological and biochemical traits (SLA, N, P) were the most robust within and across PFTs, suggesting that covariation in these traits is driven by universal leaf construction trade-offs and stoichiometry. Relationships among metabolic traits (R_d, V_{c,max}, J_max) were slightly less consistent, reflecting in part their much sparser sampling (especially for high-latitude PFTs), but also pointing to more flexible plasticity in plant metabolistm. Finally, relationships involving leaf lifespan were the least consistent, indicating that leaf economic relationships related to leaf lifespan are dominated by across-PFT differences and that within-PFT variation in leaf lifespan is more complex and idiosyncratic. Across all traits, these covariance were an important source of information, as evidenced by the improved imputation accuracy and reduced predictive uncertainty in multivariate models compared to univariate models. Ultimately, our study reaffirms the value of studying not just individual traits but the multivariate trait space and the utility of hierarchical modeling for studying the scale dependence of trait relationships.

biodiversity, ecological modeling, ecology, leaf b↗
DOE OSTI2022

Use of Physics to Improve Solar Forecast: Part II, Machine Learning and Model Interpretability

Machine learning (ML) models have been applied to forecast solar energy; however, they often lack clarity of interpretability and underlying physics. This work addresses such challenges by developing a hierarchy of ML models that gradually introduce predictors to improve the forecast accuracy based on a physics-based framework. Three ML models (ARIMA, LSTM, and XGBoost) are examined and compared with four physics-informed persistence models reported in Part I and the simple persistence model to assess the improvement of different models. The 7-year measurements at the U.S. Department of Energy's Atmospheric Radiation Measurement's Southern Great Plains Central Facility site are used for forecasts and evaluations. The results reveal that the step-by-step introduction of predictors leads to different improvements for models at different hierarchical levels. Comparison of the ML models with persistence models shows that LSTM and XGBoost outperform all the persistence models, with LSTM having the overall best performance; however, ARIMA underperforms the four physics-informed persistence models. This study demonstrates the importance and utility of incorporating physics into ML models in improving forecast accuracy by introducing a hierarchy of physics-based predictors, distinguishing predictor contributions, and enhancing the ML interpretability. The combined use of Global Horizontal Irradiance (GHI) and Direct Normal Irradiance (DNI) significantly improves the forecast accuracy compared to using individual irradiances alone because the pair contains more information on cloud-radiation interactions.

interpretability↗
DOE OSTI2020

Probabilistic Groundwater Modeling of the RDX Plume at Los Alamos National Laboratory to Support Risk Assessment - 20359

A plume of the contaminant hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX) with concentrations greater than the New Mexico tap water drinking standard (9.66 ppb) is present in the regional aquifer near the southwestern boundary of Los Alamos National Laboratory (LANL). A risk assessment is performed for exposure to regional aquifer groundwater, with long-term predictions of RDX concentrations provided by a calibrated, probabilistic, numerical fate and transport model. The structure of the model is hierarchical, with the RDX Regional Aquifer (RA) groundwater model acting as the primary tool for analysis of downgradient RDX concentrations. The RA model is deeply informed by the conceptual site model (CSM) and is calibrated using site RDX concentration data and hydraulic head measurements, along with other analyses. The model is calibrated using data through December 2019. Where data are scarce other lines of evidence are used to inform inputs, including the multiphase RDX Vadose Zone (VZ) model and Pipe and Disk analytical screening tool. Model inputs are described with informative prior distributions using a robust approach to development that incorporates all available lines of evidence for every parameter used as an input to the RA model. Calibration is performed using a classical nonlinear optimization routine, which is then used to initialize a Bayesian calibration. The Bayesian calibration constrains the uncertainty in the classical calibration, ultimately providing posterior distributions for all model parameters. The challenges of the calibration include high-dimensional parameter space, including spatially heterogeneous hydraulic conductivities, comparatively sparse data, and low RDX concentrations. Posterior parameter distributions developed in the calibration process are then used for stochastic predictive model runs into the future. The result of the forward runs is spatially and temporally explicit estimates of head and concentration with uncertainty at all points in space and time. The probabilistic modeling approach presented here includes innovative computational and statistical methods that leverage high-performance computing (HPC) resources. It makes use of prior modeling work performed at the chromium plume site in the central LANL area, with extensive updates. The risk assessment will ultimately be used to support decision-making at the site, using multiple appropriately weighted sources of information, as well as uncertainty, sensitivity, and value-of-information analyses. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Bridging Multiscale Processes in Earth System Models with Physics-Guided Hierarchical Machine Learning

Focal Area(s): The focal area of this whitepaper is, “Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising of a hierarchy of models (e.g., AI driven model/component/parameterization selection).” The ideas and frameworks described herein are deemed site agnostic. As a use case, this group will initially focus on the coupling between land processes, surface/subsurface hydrological processes, coastal processes, and human activities in the U.S. Gulf states.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Macroscopic modeling of gas permeability in hierarchical micro/nanoporous media: A unified characterization of rarefaction using Klinkenberg theory and equivalent diameter

Estimating gas transport through a hierarchical micro/nanoporous system is challenging due to non-equilibrium gas dynamics. The primary difficulty lies in determining the rarefaction level, because identifying a representative flow dimension in a complex porous system with multiple pore scales is not straightforward. Our study performed a pore-level analysis for gas permeability in dual-scale porous media with varying porosity, throat size, and secondary pore size under different rarefaction conditions. We found that secondary porosity negatively affects permeability due to increased friction forces, with this influence growing as the secondary pore size and porosity increase until the secondary pore becomes comparable to the throat. However, rarefaction reduces the effects of secondary pores due to boundary slip. Traditional Knudsen number (Kn) calculations based on Darcy-defined height failed to accurately describe the rarefaction effects on gas permeability. Instead, we introduced an equivalent diameter to calculate the Kn, which provided an accurate normalization of apparent gas permeability independent of pore geometry. Furthermore, the extended Kozeny–Carman–Klinkenberg model developed in our previous study successfully yielded a macroscopic model for apparent gas permeability in hierarchical micro/nanoporous systems as a function of the traditional Darcy height and porosity.

04 OIL SHALES AND TAR SANDS↗
DOE OSTI2021

A physics-informed and hierarchically regularized data-driven model for predicting fluid flow through porous media

This paper presents a new deep learning data-driven model for predicting structure dependent pore-fluid velocity fields in rock. The model is based on a Convolutional Auto-Encoder (CAE) artificial neural network capable of learning from image data generated by direct numerical simulations of fluid flow through pore-structures, such as by Lattice Boltzmann or molecular dynamics methods. The main novelty of the model in comparison to previous CAE-based data-driven approaches consists of three parts. The first is a methodology for decomposing the full-domain of the porous media into sub-regions, or “sub-domains”, in order to reduce the overall size of the CAE, batch process the sub-domains in parallel, and enable the CAE to learn local and generalizable nonlinear mappings of pore-fluid velocities. The second consists of embedding the finite difference solutions of the incompressible Navier-Stokes and continuity equations into convolutional layers prior to the CAE in order to provide the CAE with knowledge of fluid dynamics physics (PhyFlow). The third main novelty is that the training of the CAE is regularized with a hierarchical loss function that encourages the learning of fluid flow patterns (in a way similar to ranked modes in principal component analysis), ranking from most to least important. This is shown to increase the stability in learning, reduce over-fitting, and promote interpretability of the CAE neural network layers (HierCAE). The comprehensive new data-driven model, which we call the PhyFlow-HierCAE model, is shown to exhibit improved accuracy and generalizability of flow field predictions over conventional CAE models, attributable to the embedded physical knowledge and the hierarchical regularization, as well as realize orders of magnitude speed-ups in computation times as a surrogate for the direct numerical simulations. Examples of training and forward predictions on unseen pore-structures are provided and evaluated for data from Lattice Boltzmann and molecular dynamics simulations of pore-fluid flow. The model is shown to be a fast and accurate emulator (or “surrogate”) for predicting effective permeability of unseen pore-structures based on learning from relatively small direct numerical simulation datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases

A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture resulting in high-dimensional, difficult to interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper we apply cutting edge techniques from TDA with the goal of gaining insight towards interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers and discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight as to how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.

topological data analysis, deep learning↗
DOE OSTI2021

Methods and devices for preventing computationally explosive calculations in a computer for model parameters distributed on a hierarchy of geometric simplices

A computer-implemented method of preventing computationally explosive calculations. The method includes obtaining, by a processor of the computer, measured data of one of a physical process or a physical object; performing hierarchical numerical modeling of a physical process inclusive of an Earth model containing at least one of (a) infrastructure in the ground and (b) a formation feature in the ground, wherein predicted data is generated; comparing the measured data to the predicted data to calculate an estimated error; analyzing the estimated error via an inversion process to update the at least one of the Earth model and infrastructure model so as to reduce the estimated error and to determine a final composite Earth model of at least one of the infrastructure and the feature; and using the final composite Earth model to characterize at least one of the process and the physical object.

Source record↗
DOE OSTI2020

A novel CMB component separation method: hierarchical generalized morphological component analysis

ABSTRACT We present a novel technique for cosmic microwave background (CMB) foreground subtraction based on the framework of blind source separation. Inspired by previous work incorporating local variation to generalized morphological component analysis (GMCA), we introduce hierarchical GMCA (HGMCA), a Bayesian hierarchical graphical model for source separation. We test our method on Nside = 256 simulated sky maps that include dust, synchrotron, free–free, and anomalous microwave emission, and show that HGMCA reduces foreground contamination by $25{{\ \rm per\ cent}}$ over GMCA in both the regions included and excluded by the Planck UT78 mask, decreases the error in the measurement of the CMB temperature power spectrum to the 0.02–0.03 per cent level at ℓ > 200 (and $\lt 0.26{{\ \rm per\ cent}}$ for all ℓ), and reduces correlation to all the foregrounds. We find equivalent or improved performance when compared to state-of-the-art internal linear combination type algorithms on these simulations, suggesting that HGMCA may be a competitive alternative to foreground separation techniques previously applied to observed CMB data. Additionally, we show that our performance does not suffer when we perturb model parameters or alter the CMB realization, which suggests that our algorithm generalizes well beyond our simplified simulations. Our results open a new avenue for constructing CMB maps through Bayesian hierarchical analysis.

79 ASTRONOMY AND ASTROPHYSICS↗