Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “hierarchical data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Haar-Like Wavelets on Hierarchical Trees

Here, discrete wavelet methods, originally formulated in the setting of regularly sampled signals, can be adapted to data defined on a point cloud if some multiresolution structure is imposed on the cloud. A wide variety of hierarchical clustering algorithms can be used for this purpose, and the multiresolution structure obtained can be encoded by a hierarchical tree of subsets of the cloud. Prior work introduced the use of Haar-like bases defined with respect to such trees for approximation and learning tasks on unstructured data. This paper builds on that work in two directions. First, we present an algorithm for constructing Haar-like bases on general discrete hierarchical trees. Second, with an eye towards data compression, we present thresholding techniques for data defined on a point cloud with error controlled in the $L$ $\infty$ norm and in a Hölder-type norm. In a concluding trio of numerical examples, we apply our methods to compress a point cloud dataset, study the tightness of the $L$ $\infty$ error bound, and use thresholding to identify MNIST classifiers with good generalizability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

ParChain: a framework for parallel hierarchical agglomerative clustering using nearest-neighbor chain

This paper studies the hierarchical clustering problem, where the goal is to produce a dendrogram that represents clusters at varying scales of a data set. We propose the ParChain framework for designing parallel hierarchical agglomerative clustering (HAC) algorithms, and using the framework we obtain novel parallel algorithms for the complete linkage, average linkage, and Ward's linkage criteria. Compared to most previous parallel HAC algorithms, which require quadratic memory, our new algorithms require only linear memory, and are scalable to large data sets. ParChain is based on our parallelization of the nearest-neighbor chain algorithm, and enables multiple clusters to be merged on every round. We introduce two key optimizations that are critical for efficiency: a range query optimization that reduces the number of distance computations required when finding nearest neighbors of clusters, and a caching optimization that stores a subset of previously computed distances, which are likely to be reused. Experimentally, we show that our highly-optimized implementations using 48 cores with two-way hyper-threading achieve 5.8--110.1x speedup over state-of-the-art parallel HAC algorithms and achieve 13.75--54.23x self-relative speedup. Compared to state-of-the-art algorithms, our algorithms require up to 237.3x less space. Our algorithms are able to scale to data set sizes with tens of millions of points, which existing algorithms are not able to handle.

Computer Science↗
DOE OSTI2023

Transactive Energy System Deployment Over Insecure Communication Links

Here, in this paper, the privacy and security issues associated with the transactive energy system (TES) deployment over insecure communication links are addressed. In particular, it is ensured that 1) individual agents’ bidding information is kept private throughout hierarchical market-based interactions; and 2) any extraneous data injection attack can be quickly and easily detected. An implementation framework is proposed to enable the cryptography-based enhancement of privacy and security for the deployment of any general hierarchical systems including TESs. Under the proposed framework, a unified cryptography-based approach is developed to achieve both privacy and security simultaneously. Specifically, privacy preservation is realized by an enhanced Paillier encryption scheme, where a block design is proposed to significantly improve computational efficiency. Attack detection is further achieved by an enhanced Paillier digital signature scheme, where a stamp-concatenation mechanism is proposed to enable detection of data replace and reorder attacks. Simulation results verify the effectiveness of the proposed cyber-resilient design for transactive energy systems. Note to Practitioners—This paper is motivated by addressing the issues of cyber resiliency for practically deploying transactive energy system (TES) but it is also applicable to the problem of enhancing the privacy and security for any general hierarchical control systems. TES is an emerging control approach that engages energy suppliers and customers through market operations and uses the price to optimally allocate energy resources. While it has been shown to be promising for power system applications, the underlying market-based interactions raise significant concerns of privacy (data leakage) and security (data tampering). However, existing TES works only focus on the coordination mechanism instead of privacy and security issues. This paper proposes a new cryptography-based TES design for practical deployment. Specifically, to protect privacy, individual supply and demand amounts to be exchanged are all encrypted in a particular way such that the original amounts cannot be inferred from the encrypted amounts, while the desired computation for setting the market clearing price can be carried out over the encrypted amounts, thus generating an encrypted result which, when decrypted, matches that of the same computation over the original amounts. To achieve security, for each exchanged data, its sender generates a particular digital signature which is exchanged together with the data. This enables the receiver to automatically detect the integrity by checking whether a mathematical relationship holds for the pair of data and signature. In our future research, we will investigate more challenging scenarios where some suppliers and customers themselves could be corrupted and purposely submit distorted amounts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Theoretical and methodological challenges in hierarchical Bayesian inference for model-form uncertainty

This report describes challenges associated with the hierarchical Bayesian approach to inform model-form uncertainty (MFU) representations, which are parameterized modifications to a mathematical models’ governing equations to express uncertainty in form of the equations. To inform model-form uncertainties, hierarchical Bayesian inference is often employed. Here, the MFU parameters are distributed parametrically, and the hyperparameters of the parametric distribution are informed through Bayesian inference, with the aim of determining the MFU parameter distribution that best agrees with calibration data. In practice, however, we have found the hierarchical Bayesian approach falls short of this aim. We discuss theoretical and methodological challenges of the approach, and we present several numerical demonstrations of these challenges. To conclude, we suggest promising alternative approaches for future investigation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Enabling event-by-event precision in γ-ray cascades for neutron-induced reactions

Neutron-induced γ-ray spectra provide key inputs for modern active interrogation applications. A precise modeling of the nuclear reaction and subsequent emission of γ rays is challenging and often impossible due to limitations on evaluated data file formats and nuclear transport simulation codes. We present a framework that addresses these challenges by combining experimental data and reaction-model calculation outputs into an extended candidate version of the Generalized Nuclear Data Structure (GNDS) file, the successor format for the legacy Evaluated Nuclear Data File (ENDF-6). This proposed GNDS hierarchical format contains all the necessary ingredients for inline γ-ray cascade reproduction with event-by-event precision, including continuum–continuum and continuum–discrete transitions following neutron-capture and inelastic neutron scattering reactions. Cascade-event generation based on our approach demonstrates improved energy conservation on an event-by-event basis and permits the use of γ-γ coincidences in applications. This work offers, for the first time, a method to generate neutron-capture and inelastic neutron-scattering γ-ray cascades where energy conservation, correlations, and experimental primaries are fully accounted for.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Traveler: Navigating Task Parallel Traces for Performance Analysis

Understanding the behavior of software in execution is a key step in identifying and fixing performance issues. This is especially important in high performance computing contexts where even minor performance tweaks can translate into large savings in terms of computational resource use. To aid performance analysis, developers may collect an execution trace —a chronological log of program activity during execution. As traces represent the full history, developers can discover a wide array of possibly previously unknown performance issues, making them an important artifact for exploratory performance analysis. However, interactive trace visualization is difficult due to issues of data size and complexity of meaning. Traces represent nanosecond-level events across many parallel processes, meaning the collected data is often large and difficult to explore. The rise of asynchronous task parallel programming paradigms complicates the relation between events and their probable cause. Here, to address these challenges, we conduct a continuing design study in collaboration with high performance computing researchers. We develop diverse and hierarchical ways to navigate and represent execution trace data in support of their trace analysis tasks. Through an iterative design process, we developed Traveler , an integrated visualization platform for task parallel traces. Traveler provides multiple linked interfaces to help navigate trace data from multiple contexts. We evaluate the utility of Traveler through feedback from users and a case study, finding that integrating multiple modes of navigation in our design supported performance analysis tasks and led to the discovery of previously unknown behavior in a distributed array library.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Automating Bayesian inference and design to quantify acoustic particle levitation

Self-propulsion of micro- and nanoparticles powered by ultrasound provides an attractive strategy for the remote manipulation of colloidal matter using biocompatible energy inputs. Quantitative understanding of particle motion and its dependence on size, shape, and composition requires accurate characterization of the acoustic field, which depends sensitively on the experimental setup. Here, we show how automated experiments based on Bayesian inference and design can accurately and efficiently characterize the acoustic field within resonant chambers used to propel acoustic nanomotors. Repeated cycles of observation, inference, and design (OID) are guided by a physical model that describes the rate at which levitating particles approach the nodal plane. Using video microscopy, we observe the relaxation of tracer particles to this plane following the application of the acoustic field. We use sequential Monte Carlo methods to infer model parameters such as the amplitude and frequency of the resonant chamber while accounting for particle-level measurement noise and population-level heterogeneity in the field. Guided by simulated outcomes, we select the optimal design for the next experiment as to maximize the information gain in the relevant parameters. We show how this iterative process serves to discriminate between competing hypotheses and efficiently converges to accurate parameter estimates using only few automated experiments. We discuss the need for model criticism to ensure the validity of the guiding model throughout automated cycles of observation, inference, and design. Furthermore, this work demonstrates how Bayesian methods can learn the parameters of nonlinear, hierarchical models used to describe video microscopy data of active colloids.

42 ENGINEERING↗
DOE OSTI2024

Local-scale heterogeneity of soil thermal dynamics and controlling factors in a discontinuous permafrost region

In permafrost regions, the strong spatial and temporal variability in soil temperature cannot be explained by the weather forcing only. Understanding the local heterogeneity of soil thermal dynamics and their controls is essential to understand how permafrost systems respond to climate change and to develop process-based models or remote sensing products for predicting soil temperature. In this study, we analyzed soil temperature dynamics and their controls in a discontinuous permafrost region on the Seward Peninsula, Alaska. We acquired one-year temperature time series at multiple depths (at 5 or 10 cm intervals up to 85 cm depth) at 45 discrete locations across a 2.3 km 2 watershed. We observed a larger spatial variability in winter temperatures than that in summer temperatures at all depths, with the former controlling most of the spatial variability in mean annual temperatures. We also observed a strong correlation between mean annual ground temperature at a depth of 85 cm and mean annual or winter season ground surface temperature across the 45 locations. We demonstrate that soils classified as cold, intermediate, or warm using hierarchical clustering of full-year temperature data closely match their co-located vegetation (graminoid tundra, dwarf shrub tundra, and tall shrub tundra, respectively). We show that the spatial heterogeneity in soil temperature is primarily driven by spatial heterogeneity in snow cover, which induces variable winter insulation and soil thermal diffusivity. These effects further extend to the subsequent summer by causing variable latent heat exchanges. Finally, we discuss the challenges of predicting soil temperatures from snow depth and vegetation height alone by considering the complexity observed in the field data and reproduced in a model sensitivity analysis.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Distributed Hierarchical Contour Trees

Contour trees are a significant tool for data analysis as they capture both local and global variation. However, their utility has been limited by scalability, in particular for distributed computation and storage. We report a distributed data structure for storing the contour tree of a data set distributed on a cluster, based on a fan-in hierarchy, and an algorithm for computing it based on the boundary tree that represents only the superarcs of a contour tree that involve contours that cross boundaries between blocks. This allows us to limit the communication cost for contour tree computation to the complexity of the block boundaries rather than of the entire data set.

Carr, Hamish A↗
DOE OSTI2022

A reference tissue atlas for the human kidney

Kidney Precision Medicine Project (KPMP) is building a spatially specified human kidney tissue atlas in health and disease with single-cell resolution. Here, we describe the construction of an integrated reference map of cells, pathways, and genes using unaffected regions of nephrectomy tissues and undiseased human biopsies from 56 adult subjects. We use single-cell/nucleus transcriptomics, subsegmental laser microdissection transcriptomics and proteomics, near-single-cell proteomics, 3D and CODEX imaging, and spatial metabolomics to hierarchically identify genes, pathways, and cells. Integrated data from these different technologies coherently identify cell types/subtypes within different nephron segments and the interstitium. These profiles describe cell-level functional organization of the kidney following its physiological functions and link cell subtypes to genes, proteins, metabolites, and pathways. They further show that messenger RNA levels along the nephron are congruent with the subsegmental physiological activity. This reference atlas provides a framework for the classification of kidney disease when multiple molecular mechanisms underlie convergent clinical phenotypes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A Sparse Distributed Gigascale Resolution Material Point Method

In this paper, we present a four-layer distributed simulation system and its adaptation to the Material Point Method (MPM). The system is built upon a performance portable C++ programming model targeting major High-Performance-Computing (HPC) platforms. A key ingredient of our system is a hierarchical block-tile-cell sparse grid data structure that is distributable to an arbitrary number of Message Passing Interface (MPI) ranks. We additionally propose strategies for efficient dynamic load balance optimization to maximize the efficiency of MPI tasks. Our simulation pipeline can easily switch among backend programming models, including OpenMP and CUDA, and can be effortlessly dispatched onto supercomputers and the cloud. Finally, we construct benchmark experiments and ablation studies on supercomputers and consumer workstations in a local network to evaluate the scalability and load balancing criteria. We demonstrate massively parallel, highly scalable, and gigascale resolution MPM simulations of up to 1.01 billion particles for less than 323.25 seconds per frame with 8 OpenSSH-connected workstations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Ab Initio-Based Bond Order Potential for Arsenene Polymorphs Developed via Hierarchical Reinforcement Learning

Arsenene, a less-explored two-dimensional material, holds the potential for applications in wearable electronics, memory devices, and quantum systems. This study introduces a bond-order potential model with Tersoff formalism, the ML-Tersoff, which leverages multireward hierarchical reinforcement learning (RL), trained on an ab initio data set. This data set covers a spectrum of properties for arsenene polymorphs, enhancing our understanding of its mechanical and thermal behaviors without the complexities of traditional models requiring multiple parameter sets. Our RL strategy utilizes decision trees coupled with a hierarchical reward strategy to accelerate convergence in high-dimensional continuous search spaces. Unlike the Stillinger-Weber approach, which demands separate formalisms for buckled and puckered forms, the ML-Tersoff model concurrently captures multiple properties of the two polymorphs by effectively representing the local environment, thereby avoiding the need for different atomic types. Here, we apply the ML model to understand the mechanical and thermal properties of the arsenene polymorphs and nanostructures. We observe an inverse relationship between the critical strain and temperature in arsenene. Thermal conductivity calculations in nanosheets show good agreement with ab initio data, reflecting a decrease in thermal conductivity attributable to increased anharmonic effects at higher temperatures. We also apply the model to predict the thermal behavior of arsenene nanotubes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Comprehensive Evaluation of Agrivoltaics Research: Breadth, Depth, and Insights for Future Research

Agrivoltaics integrates agricultural production with solar energy generation to address challenges related to land use, food security, and renewable energy development. This study provides the most comprehensive evaluation to date of global agrivoltaic research, aiming to classify the literature, identify strengths and gaps, and guide future work. We systematically screened over 3000 English-language publications through 2023 for relevant agrivoltaic publications. A total of 670 studies were categorized in the InSPIRE Data Portal across five agrivoltaic activities and multiple hierarchical themes, including physical, biological, technological, social, and crosscutting domains. We found that research was concentrated on crop production, microclimate dynamics, and PV performance, with gaps in areas like human health, wildlife, policy, and standardized methodologies. Although the U.S. emphasizes animal grazing and habitat-based systems in practice, most U.S.-based studies focused disproportionately on crop production. The analysis revealed uneven geographic and topical representation and highlighted a lack of integrated, interdisciplinary approaches. This study concludes that while agrivoltaic research has grown rapidly, more coordinated efforts could support standardized data collection, address overlooked ecological and social impacts, and align research focus with real-world system implementation, ultimately improving the scalability and successful deployment of agrivoltaic systems.

14 SOLAR ENERGY↗
DOE OSTI2023

Unsupervised learning of representative local atomic arrangements in molecular dynamics data

Molecular dynamics (MD) simulations present a data-mining challenge, given that they can generate a considerable amount of data but often rely on limited or biased human interpretation to examine their information content. By not asking the right questions of MD data we may miss critical information hidden within it. Here we combine dimensionality reduction (UMAP) and unsupervised hierarchical clustering (HDBSCAN) to quantitatively characterize prevalent coordination environments of chemical species within MD data. By focusing on local coordination, we significantly reduce the amount of data to be analyzed by extracting all distinct molecular formulas within a given coordination sphere. We then efficiently combine UMAP and HDBSCAN with alignment or shape-matching algorithms to partition these formulas into structural isomer families indicating their relative populations. The method was employed to reveal details of cation coordination in electrolytes based on molecular liquids.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Efficient Hierarchical State Vector Simulation of Quantum Circuits via Acyclic Graph Partitioning

Early but promising results in quantum computing have been enabled by the concurrent development of quantum algorithms, devices, and materials. Classical simulation of quantum programs has enabled the design and analysis of algorithms and implementation strategies targeting current and anticipated quantum device architectures. In this paper, we present a graph-based approach to achieve efficient quantum circuit simulation. Our approach involves partitioning the graph representation of a given quantum circuit into sub-graphs/circuits that exhibit better data locality. Simulation of each sub-circuit is organized hierarchically, with the iterative construction and simulation of smaller state vectors, improving overall performance. Also, this partitioning reduces the number of passes through data, improving the total computation time. We present three partitioning strategies and observe that acyclic graph partitioning typically results in the best time-to-solution. In contrast, other strategies reduce the partitioning time at the expense of potentially increased simulation times. Experimental evaluation demonstrates the effectiveness of our approach.

Fang, Bo↗
DOE OSTI2023

Chemical structure and complex growth modes of magnesium silicate hydrate: Nanoparticle orientation, aggregation, and fusion

The extent of utilization of magnesium silicate hydrate (MSH) in construction is limited partly because of insufficient data ascertaining the kinetics of its precipitation. In this report MSH grown homogeneously or heterogeneously in the range of magnesium to silicon solution molar concentration ratios, [Mg]/[Si] = 0.5-1.5, and temperature = 25-80 °C was analyzed for its chemical structure and morphology. Infrared spectroscopy and geochemical modeling show that increasing the [Mg]/[Si] ratio results in less silicate polymerization in MSH. Atomic force microscopy reveals the presence of nanoparticles that aggregate and fuse to form films. Oriented attachment of nanoparticles and features indicative of enhanced crystallinity were observed at higher temperatures and longer reaction times. These data provide direct evidence for the persistence of hierarchical structures (i.e., nanoparticles forming 3D microparticles and 2D film layers) at the nanoscale to mesoscale. These findings offer insights into the precise chemical synthesis of MSH and its widespread use as a binder for construction purposes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Hierarchical deep reinforcement learning reveals a modular mechanism of cell movement

Time-lapse images of cells and tissues contain rich information about dynamic cell behaviours, which reflect the underlying processes of proliferation, differentiation and morphogenesis. However, we lack computational tools for effective inference. Here we exploit deep reinforcement learning (DRL) to infer cell–cell interactions and collective cell behaviours in tissue morphogenesis from three-dimensional (3D) time-lapse images. We use hierarchical DRL (HDRL), known for multiscale learning and data efficiency, to examine cell migrations based on images with a ubiquitous nuclear label and simple rules formulated from empirical statistics of the images. When applied to Caenorhabditis elegans embryogenesis, HDRL reveals a multiphase, modular organization of cell movement. Imaging with additional cellular markers confirms the modular organization as a novel migration mechanism, which we term sequential rosettes. Furthermore, HDRL forms a transferable model that successfully differentiates sequential rosettes-based migration from others. Our study demonstrates a powerful approach to infer the underlying biology from time-lapse imaging without prior knowledge.

59 BASIC BIOLOGICAL SCIENCES↗