Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis, visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

GenomeDepot: data management system for microbial comparative genomics

Summary GenomeDepot is an open-source web-based platform for annotation, management, and comparative analysis of microbial genomic sequences and associated data including ortholog families, protein domains, operons, regulatory interactions, strain taxonomy, and sample metadata. GenomeDepot supports rapid creation of websites for user-defined genome collections that include bioinformatic tools for interactive genome browsing, Basic Local Alignment Search Tool (BLAST) search, annotation search, comparative genomic neighborhood visualization, and sequence download. Gene function annotations are generated by a customizable annotation pipeline. The pipeline runs annotation tools in Conda environments and can be easily extended with additional user-specified tools. Availability and implementation GenomeDepot is open source and distributed under the GNU General Public License via GitHub (https://github.com/aekazakov/genome-depot). GenomeDepot is implemented in Python and was tested in Ubuntu Linux. Full installation instructions and documentation are available at https://aekazakov.github.io/genome-depot/. GenomeDepot demo server is freely accessible at https://iseq.lbl.gov/demogd/.

Kazakov, Alexey [Lawrence Berkeley National Labora↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
DOE OSTI2021

Challenging problems of quality assurance and quality control (QA/QC) of meteorological time series data

Abstract Representativeness and quality of collected meteorological data impact accuracy and precision of climate, hydrological, and biogeochemical analyses and predictions. We developed a comprehensive Quality Assurance (QA) and Quality Control (QC) statistical framework, consisting of three major phases: Phase I—Preliminary data exploration, i.e., processing of raw datasets, with the challenging problems of time formatting and combining datasets of different lengths and different time intervals; Phase II—QA of the datasets, including detecting and flagging of duplicates, outliers, and extreme data; and Phase III—the development of time series of a desired frequency, imputation of missing values, visualization and a final statistical summary. The paper includes two use cases based on the time series data collected at the Billy Barr meteorological station (East River Watershed, Colorado), and the Barro Colorado Island (BCI, Panama) meteorological station. The developed statistical framework is suitable for both real-time and post-data-collection QA/QC analysis of meteorological datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Energy Consumption, Performance, and Cost Estimates for Medium and Heavy-Duty Vehicles Based on 2022 Assumptions

Assumptions for this work was collected and the analysis was completed in FY22. This contains information for more than 20 types of medium and heavy duty vehicles. Vehicles with various levels of hybridization, electric and fuel cell powertrains are considered in this work. More details are available in the report published by Argonne accessible from https://vms.taps.anl.gov/research-highlights/u-s-doe-vto-hfto-r-d-benefits/. TechScape, a convenient data visualization tool is also provided by Argonne for this data, accessible from [TechScape Web](https://vms.taps.anl.gov/data/techscape-web-2023/).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Soil Water Retention and Hydraulic Conductivity Data and Model at Pump House in East River Watershed, Colorado 2019-2024

This data package includes soil water retention and hydraulic conductivity data and model fitting results from measurements of ex-situ soil samples and in-situ soil sensors near Pump House at Mount Crested Butte in the East River Watershed. Soil water retention curves (SWRC) characterize soil water content as a function of soil water potential. SWRC depends on soil texture and pore structure and can be used to describe the constraints on biogeochemical processes in terms of soil water availability. In this data package, the sample identification follows the format ER-X-Y, where ER refers to East River, X is the location identifier, and Y is the depth identifier at the same X (shallow Y=1). Specifically, ER-PHS, ER-LMC, ER-LMF, and ER-SMN are associated with ecohydrology sites under the East-Taylor Watershed Community Observatory Sites directory, and ER-RBTn (upslope n=1) are sampling transects during the 2019 Rootball Campaign. The sample and location information can be found in metadata.csv. Sampling and Measurements Each sample falls into one of the three sampling methods – (1) intact cores, (2) repacked samples, or (3) soil sensors – and one of the two measurement methods – (a) laboratory or (b) in-situ. Both intact cores and repacked samples were measured using the laboratory methods, which include measurements of soil water potential (HYPROP & WP4C, METER), saturated (KSAT, METER) and unsaturated hydraulic conductivity (HYPROP). The in-situ method uses a pair of co-located soil sensors to measure volumetric water content (TEROS12, METER) and soil water potential (TEROS21, METER), and the hydraulic conductivity was not measured. In comparison, the laboratory methods progress from full saturation to dry conditions, and the in-situ method includes both dry-to-wet and wet-to-dry cycles. The sampling and measurement methods for each sample can be found in metadata.csv, and more information about the measurements is detailed in the Methods section below. Models Retention and hydraulic conductivity data were fitted with four van-Genuchten-type models (specified by “model_name” column in the files): (1) traditional constrained van Genuchten model (“vG_constrained”), (2) traditional unconstrained van Genuchten model (“vG_unconstrained”), (3) PDI-variant of the constrained van Genuchten model (“vG_constrained_PDI”), and (4) PDI-variant of the unconstrained van Genuchten model (“vG_unconstrained_PDI”). The difference between the constrained (1: n) and the unconstrained (2: n, m) van Genuchten models is the number of pore-size distribution parameters in the model equations, giving the unconstrained model more degrees of freedom when fitting the data. Between the traditional and the PDI-variant models, model fitting differs the most at the dry end of the measurements. The traditional models allow infinite suction at the residual water content (water content does not drop below residual water content), and the PDI-variant models enforce a soil water potential value of pF=6.8 (~ -630 MPa) at oven-dryness (water content reaches 0). The inclusion of the van-Genuchten-type models is due to their common application. If other retention models are required, users can access the data in data.csv for further data fitting. More information about the models can be found in the Methods section below. Fitting Tasks The model fitting can be categorized into three levels of tasks (specified by “fitting_task” column in the files). Level 1 (“fit_retention”) only includes retention data fitting (the only level available for the in-situ method). Level 2 (“fit_retention_conductivity”) includes both retention and hydraulic conductivity data fitting, and the saturated hydraulic conductivity (Ks, a parameter of the hydraulic conductivity functions) is fixed by the measurements from KSAT. Level 3 (“fit_retention_conductivity_Ks”) also includes both retention and hydraulic conductivity data fitting, but Ks is a fitted parameter without the constraints from KSAT measurements. Among the same retention models (e.g. vG_constrained models of the same sample), level 1 should produce the best retention data fitting. Level 2 should have the highest misfit of the retention and hydraulic conductivity data, because the retention and hydraulic conductivity functions share common model parameters, and the unsaturated hydraulic conductivity (HYPROP) data fitting is subject to Ks measured independently by KSAT. Level 3 should have mid-level misfits of the retention and hydraulic conductivity data. While level 3 fits the hydraulic conductivity data better than level 2, the fitted Ks value might be unreasonable due to the lack of constraints at the wet end of the measurements. General recommendation when using this data package: (1) Choice of sampling methods: Intact cores and in-situ soil sensors could be prioritized because these sampling methods are less destructive. While the repacked samples were packed to the target bulk density (estimated post-sampling, when sample volume was known), these samples had altered pore structures. Nevertheless, intact cores might suffer from sample gaps that would lead to overestimation of Ks (sample gaps can be inferred from the “soil_sample_volume” column in metadata.csv when the value is < 249). In-situ method also has higher uncertainty in characterizing the wet end of the SWRC because of sensor limitations and the difficulty in reaching full saturation under natural conditions. (2) Choice of fitting tasks: When only retention data is needed, level 1 (“fit_retention”) should be prioritized. When both retention and hydraulic conductivity data are needed, level 2 (“fit_retention_conductivity”) could be prioritized. (3) Choice of models: This could depend on what the downstream models call for. If no specific model is required, model misfit could be used as a ranking criterion. Model misfit values in terms of RMSE can be found in model_parameters.csv. The following files are included in this data package: (1) metadata.csv – This file includes the general information of each sample, including location (description, geocoordinates, elevation), sampling and measurements details (method, depth, time or period, volume, instruments), and soil physical properties (bulk density, saturated hydraulic conductivity, only applicable to physical soil samples). (2) data.csv – This file includes soil water potential, volumetric water content, and unsaturated hydraulic conductivity data of each sample. Column “instrument” specifies the instrument (HYPROP, WP4C, or TEROS) used to perform the measurements. (3) model_fit.csv – This file includes soil water potential, volumetric water content, and unsaturated hydraulic conductivity fitted from the four models and three fitting tasks. Column “model_name” specifies the retention model used, and “fitting_task” specifies the level of data fitting. Missing values indicate that the variable does not apply to that fitting task. (4) model_parameters.csv – This file includes the fitted model parameters, model misfits, and conventional water content thresholds (field capacity and wilting point) from the four models and three fitting tasks. Column “model_name” specifies the retention model used, and “fitting_task” specifies the level of data fitting. Missing values indicate that the parameter does not apply to that model and/or that fitting task. (5) data_Ks.csv – This file includes the saturated hydraulic conductivity measurements from KSAT. (6) /figure/*.png – This folder includes three quick visualizations of the data, retention model fitting results and misfits, and hydraulic conductivity model fitting results, misfits, and parameters. The model fitting results are separated by samples and fitting tasks and colored by models. Zoom-in required. (7) /hyprop/*.bdhx – This folder includes proprietary hyprop files that require the free Labros SoilView-Analysis (METER) to open. Users can explore data fitting using other retention models (i.e. Brooks-Corey, Fredlund-Xing, Kosugi, bimodal models). Be aware that Ks value is pre-entered under “Fitting tab, Conductivity functions parameters” for level 2 fitting. If the value is lost, please refer to metadata.csv under “Ks” column. (8) Six file-level metadata that summarize file, header, column, and variable information of all files. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

EARTH SCIENCE > LAND SURFACE > SOILS↗
DOE OSTI2026

Soil Water Retention and Hydraulic Conductivity Data and Model at Snodgrass Mountain in East River Watershed, Colorado 2020-2025

This data package includes soil water retention and hydraulic conductivity data and model fitting results from measurements of ex-situ soil samples and in-situ soil sensors at Snodgrass Mountain. Soil water retention curves (SWRC) characterize soil water content as a function of soil water potential. SWRC depends on soil texture and pore structure and can be used to describe the constraints on biogeochemical processes in terms of soil water availability. In this data package, the sample identification follows the format SG-X-Y, where SG refers to Snodgrass Mountain, X is the location identifier, and Y is the depth identifier at the same X (shallow Y=1). Specifically, SG-EHS is associated with ecohydrology sites under the East-Taylor Watershed Community Observatory Sites directory, and SG-ERTn (upslope n=1) are points along the Snodgrass electrical resistivity tomography transect not associated with the existing site names in the directory. The sample and location information can be found in metadata.csv. Sampling and Measurements Each sample falls into one of the three sampling methods – (1) intact cores, (2) repacked samples, or (3) soil sensors – and one of the two measurement methods – (a) laboratory or (b) in-situ. Both intact cores and repacked samples were measured using the laboratory methods, which include measurements of soil water potential (HYPROP & WP4C, METER), saturated (KSAT, METER) and unsaturated hydraulic conductivity (HYPROP). The in-situ method uses a pair of co-located soil sensors to measure volumetric water content (TEROS12, METER) and soil water potential (TEROS21, METER), and the hydraulic conductivity was not measured. In comparison, the laboratory methods progress from full saturation to dry conditions, and the in-situ method includes both dry-to-wet and wet-to-dry cycles. The sampling and measurement methods for each sample can be found in metadata.csv, and more information about the measurements is detailed in the Methods section below. Models Retention and hydraulic conductivity data were fitted with four van-Genuchten-type models (specified by “model_name” column in the files): (1) traditional constrained van Genuchten model (“vG_constrained”), (2) traditional unconstrained van Genuchten model (“vG_unconstrained”), (3) PDI-variant of the constrained van Genuchten model (“vG_constrained_PDI”), and (4) PDI-variant of the unconstrained van Genuchten model (“vG_unconstrained_PDI”). The difference between the constrained (1: n) and the unconstrained (2: n, m) van Genuchten models is the number of pore-size distribution parameters in the model equations, giving the unconstrained model more degrees of freedom when fitting the data. Between the traditional and the PDI-variant models, model fitting differs the most at the dry end of the measurements. The traditional models allow infinite suction at the residual water content (water content does not drop below residual water content), and the PDI-variant models enforce a soil water potential value of pF=6.8 (~ -630 MPa) at oven-dryness (water content reaches 0). The inclusion of the van-Genuchten-type models is due to their common application. If other retention models are required, users can access the data in data.csv for further data fitting. More information about the models can be found in the Methods section below. Fitting Tasks The model fitting can be categorized into three levels of tasks (specified by “fitting_task” column in the files). Level 1 (“fit_retention”) only includes retention data fitting (the only level available for the in-situ method). Level 2 (“fit_retention_conductivity”) includes both retention and hydraulic conductivity data fitting, and the saturated hydraulic conductivity (Ks, a parameter of the hydraulic conductivity functions) is fixed by the measurements from KSAT. Level 3 (“fit_retention_conductivity_Ks”) also includes both retention and hydraulic conductivity data fitting, but Ks is a fitted parameter without the constraints from KSAT measurements. Among the same retention models (e.g. vG_constrained models of the same sample), level 1 should produce the best retention data fitting. Level 2 should have the highest misfit of the retention and hydraulic conductivity data, because the retention and hydraulic conductivity functions share common model parameters, and the unsaturated hydraulic conductivity (HYPROP) data fitting is subject to Ks measured independently by KSAT. Level 3 should have mid-level misfits of the retention and hydraulic conductivity data. While level 3 fits the hydraulic conductivity data better than level 2, the fitted Ks value might be unreasonable due to the lack of constraints at the wet end of the measurements. General recommendation when using this data package: (1) Choice of sampling methods: Intact cores and in-situ soil sensors could be prioritized because these sampling methods are less destructive. While the repacked samples were packed to the target bulk density (estimated post-sampling, when sample volume was known), these samples had altered pore structures. Nevertheless, intact cores might suffer from sample gaps that would lead to overestimation of Ks (sample gaps can be inferred from the “soil_sample_volume” column in metadata.csv when the value is < 249). In-situ method also has higher uncertainty in characterizing the wet end of the SWRC because of sensor limitations and the difficulty in reaching full saturation under natural conditions. (2) Choice of fitting tasks: When only retention data is needed, level 1 (“fit_retention”) should be prioritized. When both retention and hydraulic conductivity data are needed, level 2 (“fit_retention_conductivity”) could be prioritized. (3) Choice of models: This could depend on what the downstream models call for. If no specific model is required, model misfit could be used as a ranking criterion. Model misfit values in terms of RMSE can be found in model_parameters.csv. The following files are included in this data package: (1) metadata.csv – This file includes the general information of each sample, including location (description, geocoordinates, elevation), sampling and measurements details (method, depth, time or period, volume, instruments), and soil physical properties (bulk density, saturated hydraulic conductivity, only applicable to physical soil samples). (2) data.csv – This file includes soil water potential, volumetric water content, and unsaturated hydraulic conductivity data of each sample. Column “instrument” specifies the instrument (HYPROP, WP4C, or TEROS) used to perform the measurements. (3) model_fit.csv – This file includes soil water potential, volumetric water content, and unsaturated hydraulic conductivity fitted from the four models and three fitting tasks. Column “model_name” specifies the retention model used, and “fitting_task” specifies the level of data fitting. Missing values indicate that the variable does not apply to that fitting task. (4) model_parameters.csv – This file includes the fitted model parameters, model misfits, and conventional water content thresholds (field capacity and wilting point) from the four models and three fitting tasks. Column “model_name” specifies the retention model used, and “fitting_task” specifies the level of data fitting. Missing values indicate that the parameter does not apply to that model and/or that fitting task. (5) data_Ks.csv – This file includes the saturated hydraulic conductivity measurements from KSAT. (6) /figure/*.png – This folder includes three quick visualizations of the data, retention model fitting results and misfits, and hydraulic conductivity model fitting results, misfits, and parameters. The model fitting results are separated by samples and fitting tasks and colored by models. Zoom-in required. (7) /hyprop/*.bdhx – This folder includes proprietary hyprop files that require the free Labros SoilView-Analysis (METER) to open. Users can explore data fitting using other retention models (i.e. Brooks-Corey, Fredlund-Xing, Kosugi, bimodal models). Be aware that Ks value is pre-entered under “Fitting tab, Conductivity functions parameters” for level 2 fitting. If the value is lost, please refer to metadata.csv under “Ks” column. (8) Six file-level metadata that summarize file, header, column, and variable information of all files. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

EARTH SCIENCE > LAND SURFACE > SOILS↗
DOE OSTI2024

Pre-Transient Characterization of Historic EBR-II Pins for Transient Testing

Current interest in sodium-cooled fast reactor (SFR) designs, such as TerraPower’s Natrium Reactor, has highlighted the need for advanced reactor fuel technology development. Modern U-Zr and U- Pu-Zr pin designs are primary candidates to fuel SFRs and boast high fuel utilization capacity, increased fuel-cladding compatibility, and improved safety through inherent feedback mechanisms. Despite over 60 years of metallic fuel irradiation, uncertainties exist in the performance of the fuel system, particularly under transient overpower (TOP) and loss of flow (LOF) scenarios. Throughout historical testing within the Experimental Breeder Reactor II (EBR-II) and the Fast Flux Test Facility (FFTF), fuel behavior has demonstrated benign response to transient reactor conditions; however, accurate predictions of failure thresholds to inform operational limitations rely heavily on fuel composition, burnup, and irradiation history. In expanding TOP and LOF testing, the Transient Heat sink Overpower Response (THOR) Capsule will be used to test modern fuel technologies in a static sodium environment in the Transient Reactor Test (TREAT) Facility. The THOR capsule is highly instrumented and will provide time-dependent thermal behavior of SFR fuel pins subjected to accident conditions within TREAT. The THOR-Metallic (THOR- M) campaign aims to validate and expand historical TOP and LOF testing on high burnup U-Zr and U-Pu- Zr fuel alloys previously irradiated in EBR-II by running the rods to failure. This contribution focuses primarily on the pre-transient engineering-scale destructive and non- destructive characterization that has been conducted on both the test and sibling pins used for the TOP and LOF tests. All pins underwent visual examination, neutron radiography, element contact profilometry, and precise gamma scan. The sibling pins used for each test were further analyzed using gas assay, sampling, and recharge analysis (GASR), and optical microscopy. The results from each technique confirmed that the fuel pins were intact and devoid of any atypical developments when compared to historical data. Additionally, the analyzed measurements establish a baseline for comparison to post-transient analysis. Key fuel behaviors quanitifed include axial elongation of the fuel column, diametral strain of the pin, patterns in fluff structure geometry, changes in axial isotope distribution, evolution of constituent redistribution, porosity, and fission gas release. The pre-transient measurements and changes attributed to transient behavior from post-transient measurement will be compared to historical data to capture the behavioral dependence on composition, burnup, and irradiation history. Results from this work advance the initiatives of the THOR-M campaign, which aid in informing fuel performance models and establishing safety criteria for SFR operational limits. The novel combination of test environment, in-situ instrumentation, and comprehensive suite of characterization methods provides greater understanding of transient fuel behavior. Overall, information on the time and condition of pin failure for high burnup U-Pu-Zr will greatly expand the limited existing TOP and LOF test data.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Pre-Transient Characterization of Historic EBR-II Pins for Transient Testing

Current interest in sodium-cooled fast reactor (SFR) designs, such as TerraPower’s Natrium Reactor, has highlighted the need for advanced reactor fuel technology development. Modern U-Zr and U- Pu-Zr pin designs are primary candidates to fuel SFRs and boast high fuel utilization capacity, increased fuel-cladding compatibility, and improved safety through inherent feedback mechanisms. Despite over 60 years of metallic fuel irradiation, uncertainties exist in the performance of the fuel system, particularly under transient overpower (TOP) and loss of flow (LOF) scenarios. Throughout historical testing within the Experimental Breeder Reactor II (EBR-II) and the Fast Flux Test Facility (FFTF), fuel behavior has demonstrated benign response to transient reactor conditions; however, accurate predictions of failure thresholds to inform operational limitations rely heavily on fuel composition, burnup, and irradiation history. In expanding TOP and LOF testing, the Transient Heat sink Overpower Response (THOR) Capsule will be used to test modern fuel technologies in a static sodium environment in the Transient Reactor Test (TREAT) Facility. The THOR capsule is highly instrumented and will provide time-dependent thermal behavior of SFR fuel pins subjected to accident conditions within TREAT. The THOR-Metallic (THOR- M) campaign aims to validate and expand historical TOP and LOF testing on high burnup U-Zr and U-Pu- Zr fuel alloys previously irradiated in EBR-II by running the rods to failure. This contribution focuses primarily on the pre-transient engineering-scale destructive and non- destructive characterization that has been conducted on both the test and sibling pins used for the TOP and LOF tests. All pins underwent visual examination, neutron radiography, element contact profilometry, and precise gamma scan. The sibling pins used for each test were further analyzed using gas assay, sampling, and recharge analysis (GASR), and optical microscopy. The results from each technique confirmed that the fuel pins were intact and devoid of any atypical developments when compared to historical data. Additionally, the analyzed measurements establish a baseline for comparison to post-transient analysis. Key fuel behaviors quanitifed include axial elongation of the fuel column, diametral strain of the pin, patterns in fluff structure geometry, changes in axial isotope distribution, evolution of constituent redistribution, porosity, and fission gas release. The pre-transient measurements and changes attributed to transient behavior from post-transient measurement will be compared to historical data to capture the behavioral dependence on composition, burnup, and irradiation history. Results from this work advance the initiatives of the THOR-M campaign, which aid in informing fuel performance models and establishing safety criteria for SFR operational limits. The novel combination of test environment, in-situ instrumentation, and comprehensive suite of characterization methods provides greater understanding of transient fuel behavior. Overall, information on the time and condition of pin failure for high burnup U-Pu-Zr will greatly expand the limited existing TOP and LOF test data.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

A Visual Comparison of Silent Error Propagation

High-performance computing (HPC) systems play a critical role in facilitating scientific discoveries. Their scale and complexity (e.g., the number of computational units and software stack) continue to grow as new systems are expected to process increasingly more data and reduce computing time. However, with more processing elements, the probability that these systems will experience a random bit-flip error that corrupts a program's output also increases, which is often recognized as silent data corruption. Analyzing the resiliency of HPC applications in extreme-scale computing to silent data corruption is crucial but difficult. An HPC application often contains a large number of computation units that need to be tested, and error propagation caused by error corruption is complex and difficult to interpret. Here, to accommodate this challenge, we propose an interactive visualization system that helps HPC researchers understand the resiliency of HPC applications and compare their error propagation. Our system models an application's error propagation to study a program's resiliency by constructing and visualizing its fault tolerance boundary. Coordinating with multiple interactive designs, our system enables domain experts to efficiently explore the complicated spatial and temporal correlation between error propagations. At the end, the system integrated a nonmonotonic error propagation analysis with an adjustable graph propagation visualization to help domain experts examine the details of error propagation and answer such questions as why an error is mitigated or amplified by program execution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Real-space visualization of short-range antiferromagnetic correlations in a magnetically enhanced thermoelectric

Short-range magnetic correlations can significantly increase the thermopower of magnetic semiconductors, representing a noteworthy development in the decades-long effort to develop high-performance thermoelectric materials. Here, we reveal the nature of the thermopower-enhancing magnetic correlations in the antiferromagnetic semiconductor MnTe. Using magnetic pair distribution function analysis of neutron scattering data, we obtain a detailed, real-space view of robust, nanometer-scale, antiferromagnetic correlations that persist into the paramagnetic phase above the Neel temperature $T_N$ = 307 K. In this work, the magnetic correlation length in the paramagnetic state is significantly longer along the crystallographic c axis than within the ab plane, pointing to anisotropic magnetic interactions. Ab initio calculations of the spin-spin correlations using density functional theory in the disordered local moment approach reproduce this result with quantitative accuracy. These findings constitute the first real-space picture of short-range spin correlations in a magnetically enhanced thermoelectric and inform future efforts to optimize thermoelectric performance by magnetic means.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Field To Farm Aggregation For Agricultural Systems

The Fields to Farms methodology illustrates the generation of farm parcels from the Crop Data Layer (CDL), a raster dataset containing 133 categories representing various crop types and land uses. This methodology involves two primary steps: Field Delineation and Farm Aggregation. The code specifically addresses the aggregation of pre-delineated fields within a county to form farms, adhering to predefined criteria for farm size categories. It is assumed that the field delineation process, which involves creating vector polygons from CDL raster, has been completed beforehand, possibly through external tools or methods. Upon initialization, the script processes county-level fields, preparing them for farm aggregation. In the Farm Aggregation phase, the code iteratively combines delineated fields into farms based on specified criteria, continuing until the aggregated farm size meets predefined thresholds derived from data from the 2017 National Agricultural Statistics Service (NASS) census. Throughout this iterative process, the script dynamically adjusts the aggregation to ensure alignment with the desired distribution reported by NASS. The resulting output of the script is a GeoDataFrame containing classified farms, which are subsequently saved as GeoPackage files. These files enable further analysis and visualization, facilitating comprehensive exploration of the farm landscape generated through the methodology.

Paudel, Rajiv [Idaho National Laboratory (INL), Id↗
DOE OSTI2026

Dynamic Facade Dashboard v0.1.0

The dashboard is a useful tool for early-stage building design decision-making and communication, as it can help users quickly compare the energy and non-energy related performance of various automated, integrated facade systems using a library of pre-computed data. Users can explore the impacts of various design choices by selecting different facade glazing and shading systems, facade control strategies, and lighting control strategies across multiple climate zones. The dashboard instantly visualizes key metrics, including energy usage in HVAC and lighting, peak cooling and heating load, and daylight availability, allowing immediate trade-off analysis to optimize building efficiency and comfort.

Yu, Tammie [Lawrence Berkeley National Laboratory ↗
DOE OSTI2024

Data Visualization: Augmented Reality

In recent years, there has been an increasing interest in developing new technologies for automated characterization and visualization of condition monitoring data. Augmented Reality (AR) is a technology that is being developed to improve such data visualization. Augmented reality has been defined as a technology that merges virtual and physical components in real-time, and in three dimensions. Wearable, commercially-available AR devices allow onsite engineers and technicians to perform inspection tasks with significantly more available information such as comparisons of past and present sensor and imager data, onsite data analysis and result displays, and various forms of metadata including technical drawings, previous inspection reports and maintenance histories, operation manuals, codes and standards, and holograms representing data analysis results superimposed onto the in situ monitored system.

42 ENGINEERING↗
DOE OSTI2021

COMPASO : A new halo finder for competitive assignment to spherical overdensities

We describe a new method (compaso) for identifying groups of particles in cosmological N-body simulations. compaso builds upon existing spherical overdensity (SO) algorithms by taking into consideration the tidal radius around a smaller halo before competitively assigning halo membership to the particles. In this way, the compaso finder allows for more effective deblending of haloes in close proximity as well as the formation of new haloes on the outskirts of larger ones. This halo-finding algorithm is used in the abacussummit suite of N-body simulations, designed to meet the cosmological simulation requirements of the Dark Energy Spectroscopic Instrument (DESI) survey. compaso is developed as a highly efficient on-the-fly group finder, which is crucial for enabling good load-balancing between the GPU and CPU and the creation of high-resolution merger trees. In this paper, we describe the halo-finding procedure and its particular implementation in abacus, accompanying it with a qualitative analysis of the finder. We test the robustness of the compaso catalogues before and after applying the cleaning method described in an accompanying paper and demonstrate its effectiveness by comparing it with other validation techniques. We then visualize the haloes and their density profiles, finding that they are well fit by the NFW formalism. Finally, we compare other properties such as radius–mass relationships and two-point correlation functions with that of another widely used halo finder, rockstar.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Hypergraph Analytics of Domain Name System Relationships

We report on the use of novel mathematical methods in hypergraph analytice over a large quantity of DNS data. Hypergraphs generalize graphs, as used in network science, to better model complex multiway relations in cyber data. Specifically, casting DNS data from Georgia Tech's ActiveDNS repository as hypergraphs allows us to fully represent the interactions between {\em collections} of domains and IP addresses. To facilitate large-scale analytics, we fielded an analytical pipeline of two capabilities. HyperNetX (HNX) is a Python package for the exploration and visualization of hypergraphs, acting as a frontend. For the backend, the Chapel HyperGraph Library (CHGL) is a library for high performance hypergraph analytics written in the exascale programming language Chapel. CHGL was used to process gigascale DNS data, performing compute-intensive calculations for data reduction and segmentation. Identified portions are then sent to HNX for both exploratory analysis and knowledge discovery targeting known tactics, techniques, and procedures.

Joslyn, Cliff A.↗
DOE OSTI2024

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Landsat-based Irrigation Dataset (LANID): 30 m resolution maps of irrigation distribution, frequency, and change for the US, 1997–2017

Abstract. Data on irrigation patterns and trends at field-level detail across broad extents are vital for assessing and managing limited water resources. Until recently, there has been a scarcity of comprehensive, consistent, and frequent irrigation maps for the US. Here we present the new Landsat-based Irrigation Dataset (LANID), which is comprised of 30 m resolution annual irrigation maps covering the conterminous US (CONUS) for the period of 1997–2017. The main dataset identifies the annual extent of irrigated croplands, pastureland, and hay for each year in the study period. Derivative maps include layers on maximum irrigated extent, irrigation frequency and trends, and identification of formerly irrigated areas and intermittently irrigated lands. Temporal analysis reveals that 38.5×106 ha of croplands and pasture–hay has been irrigated, among which the yearly active area ranged from ∼22.6 to 24.7×106 ha. The LANID products provide several improvements over other irrigation data including field-level details on irrigation change and frequency, an annual time step, and a collection of ∼10 000 visually interpreted ground reference locations for the eastern US where such data have been lacking. Our maps demonstrated overall accuracy above 90 % across all years and regions, including in the more humid and challenging-to-map eastern US, marking a significant advancement over other products, whose accuracies ranged from 50 % to 80 %. In terms of change detection, our maps yield per-pixel transition accuracy of 81 % and show good agreement with US Department of Agriculture reports at both county and state levels. The described annual maps, derivative layers, and ground reference data provide users with unique opportunities to study local to nationwide trends, driving forces, and consequences of irrigation and encourage the further development and assessment of new approaches for improved mapping of irrigation, especially in challenging areas like the eastern US. The annual LANID maps, derivative products, and ground reference data are available through https://doi.org/10.5281/zenodo.5548555 (Xie and Lark, 2021a).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

pyXPCSviewer : an open-source interactive tool for X-ray photon correlation spectroscopy visualization and analysis

pyXPCSviewer , a Python-based graphical user interface that is deployed at beamline 8-ID-I of the Advanced Photon Source for interactive visualization of XPCS results, is introduced. pyXPCSviewer parses rich X-ray photon correlation spectroscopy (XPCS) results into independent PyQt widgets that are both interactive and easy to maintain. pyXPCSviewer is open-source and is open to customization by the XPCS community for ingestion of diversified data structures and inclusion of novel XPCS techniques, both of which are growing demands particularly with the dawn of near-diffraction-limited synchrotron sources and their dedicated XPCS beamlines.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗