Results for “Data exploration” — Page 42

DOE OSTI2022

Challenges and Lessons Learned in Applying Sensitivity Analysis to Building Stock Energy Models

Uncertainty Analysis (UA) and Sensitivity Analysis (SA) offer essential tools to determine the limits of inference of a model and explore the factors which have the most effect on the model outputs. However, despite a well established body of work applying UA and SA to models of individual buildings, a review of the literature relating to energy models for larger groups of buildings undertaken by Fennell et al. (2019) highlighted very limited application at larger scales. This contribution describes the efforts undertaken by a group of research teams in the context of IEA-EBC Annex 70 working with a diverse set of Building Stock Models (BSMs) to apply global sensitivity analysis methods and compare their results. Since BSMs are a class of model defined by their output and coverage rather than their structure and inputs, they represent a diverse set of modelling approaches. Key challenges for the application of SA are identified and explored, including the influence of model form, input data types and model outputs. This study combines results from 7 different modelling teams, each using different models across a range of urban areas to explore these challenges and begin the process of developing standardised workflows for SA of BSMs.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗

DOE OSTI2020

K-means-driven Gaussian Process data collection for angle-resolved photoemission spectroscopy

Abstract We propose the combination of k-means clustering with Gaussian Process (GP) regression in the analysis and exploration of 4D angle-resolved photoemission spectroscopy (ARPES) data. Using cluster labels as the driving metric on which the GP is trained, this method allows us to reconstruct the experimental phase diagram from as low as 12% of the original dataset size. In addition to the phase diagram, the GP is able to reconstruct spectra in energy-momentum space from this minimal set of data points. These findings suggest that this methodology can be used to improve the efficiency of ARPES data collection strategies for unknown samples. The practical feasibility of implementing this technology at a synchrotron beamline and the overall efficiency implications of this method are discussed with a view on enabling the collection of more samples or rapid identification of regions of interest.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Traveler: Navigating Task Parallel Traces for Performance Analysis

Understanding the behavior of software in execution is a key step in identifying and fixing performance issues. This is especially important in high performance computing contexts where even minor performance tweaks can translate into large savings in terms of computational resource use. To aid performance analysis, developers may collect an execution trace —a chronological log of program activity during execution. As traces represent the full history, developers can discover a wide array of possibly previously unknown performance issues, making them an important artifact for exploratory performance analysis. However, interactive trace visualization is difficult due to issues of data size and complexity of meaning. Traces represent nanosecond-level events across many parallel processes, meaning the collected data is often large and difficult to explore. The rise of asynchronous task parallel programming paradigms complicates the relation between events and their probable cause. Here, to address these challenges, we conduct a continuing design study in collaboration with high performance computing researchers. We develop diverse and hierarchical ways to navigate and represent execution trace data in support of their trace analysis tasks. Through an iterative design process, we developed Traveler , an integrated visualization platform for task parallel traces. Traveler provides multiple linked interfaces to help navigate trace data from multiple contexts. We evaluate the utility of Traveler through feedback from users and a case study, finding that integrating multiple modes of navigation in our design supported performance analysis tasks and led to the discovery of previously unknown behavior in a distributed array library.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Exploring DAOS as a Burst Buffer for a 100 Gbps DAQ Real-Time Streaming System

We present an experimental evaluation of a burst buffer for a real-time DAQ streaming system designed to transmit instrument data to remote data centers. The system is based on EJ-FAT, a load balancing system capable of Nx 100Gbps streams, distributing data from event sources to processing nodes. We explore applying the DAOS system as a burst buffer to serve a number of purposes: improve resiliency, elasticity and add new functions into the processing pipeline. In the evaluation a sender transmits events over a 100Gbps network to a receiver integrated with DAOS to store the reassembled events using DAOS APIs. We evaluate the system for possible bottlenecks and provide end-to-end evaluation with a burst buffer using DAOS storage abstractions. We show that a receiver node can support 38.1 Gbps. This proves the viability of our approach and allows us to extend this work to investigate scale-out properties and new streaming optimizations.

Mei, Xinxin↗

DOE OSTI2022

Decoding the Mechanisms of Phase Transitions from In Situ Microscopy Observations

Abstract Analysis of the temperature‐ and stimulus‐dependent imaging data toward elucidation of the physical transformations is an ubiquitous problem in multiple fields. Here, temperature‐induced phase transition in BaTiO 3 is explored using the machine learning analysis of domain morphologies visualized via variable‐temperature scanning transmission electron microscopy (STEM) imaging data. This approach is based on the multivariate statistical analysis of the time or temperature dependence of the statistical descriptors of the system, derived in turn from the categorical classification of observed domain structures or projection on the continuous parameter space of the feature extraction‐dimensionality reduction transform. The proposed workflow offers a powerful tool for the exploration of the dynamic data based on the statistics of image representation as a function of the external control variable to visualize the transformation pathways during phase transitions and chemical reactions. This can include the mesoscopic STEM data as demonstrated here, but also optical, chemical imaging, etc., data. It can further be extended to the higher dimensional spaces, for example, analysis of the combinatorial libraries of materials compositions.

Valleti, Sai Mani Prudhvi↗

DOE OSTI2020

Threshold $\pi ^-$ photoproduction on the neutron

Recent data from the PIONS@MAX-lab Collaboration, measuring the total cross section of the incoherent pion photoproduction reaction, γd → π – pp, near threshold, have been used to extract the E 0+ multipole and total cross section of the reaction γn → π – p, also near threshold. These are the first measurements of the reaction γd → π – pp in the threshold region. The value of E 0+ is extracted through a fit to the deuteron data in a photoproduction model accounting for final-state interactions. The model takes an S-wave approximation for the elementary reaction γn → π – p with E 0+ = const in the threshold region. The fit over all the 6 deuteron data points gives the value E 0+ = –31.86 ± 0.8 (in units 10 –3 /mπ). We explore the dependence of our results on the choice of data subsets included in the fit. Here, the obtained values of E 0+ , for different subsets, have overlapping errors and agree with previous determinations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Trace Metal Content and Speciation, Water and Soil Chemistry, and Methane Production for a Wetland in Missouri (September 2015) and in Florida (January 2016)

Data is associated with a manuscript in preparation for submission to explore whether low trace metal availability inhibits methane production in freshwater wetland soils. Reported data are from two field sites, one in Missouri and the second in Florida (see location data). At each site, surface water compositions and properties and soil compositions are reported. Trace metal availability is assessed via sequential chemical extractions, Ni K-edge X-ray absorption near edge structure (XANES) spectroscopy, and X-ray microfluorescence imaging. Soil mineralogy is determined via powder X-ray diffraction. Multi-energy X-ray microfluorescence imaging as well as bulk and microscale S K-edge XANES spectra assess sulfur speciation in the soils. This package also reports data from trace metal amendment experiments, specifically methane (CH4) production versus time in soil incubations, the dissolved trace metal concentrations in these incubations, and the Ni K-edge XANES and extended X-ray absorption fine structure (EXAFS) spectra of soils to which increasing concentrations of Ni were added. A final version of this package will be published upon acceptance of the associated manuscript.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Optimising the processing and storage of visibilities using lossy compression

The next-generation radio astronomy instruments are providing a massive increase in sensitivity and coverage, largely through increasing the number of stations in the array and the frequency span sampled. The two primary problems encountered when processing the resultant avalanche of data are the need for abundant storage and the constraints imposed by I/O, as I/O bandwidths drop significantly on cold storage. An example of this is the data deluge expected from the SKA Telescopes of more than 60 PB per day, all to be stored on the buffer filesystem. While compressing the data is an obvious solution, the impacts on the final data products are hard to predict. In this paper, we chose an error-controlled compressor – MGARD – and applied it to simulated SKA-Mid and real pathfinder visibility data, in noise-free and noise-dominated regimes. As the data have an implicit error level in the system temperature, using an error bound in compression provides a natural metric for compression. MGARD ensures the compression incurred errors adhere to the user-prescribed tolerance. To measure the degradation of images reconstructed using the lossy compressed data, we proposed a list of diagnostic measures, exploring the trade-off between these error bounds and the corresponding compression ratios, as well as the impact on science quality derived from the lossy compressed data products through a series of experiments. We studied the global and local impacts on the output images for continuum and spectral line examples. We found relative error bounds of as much as 10%, which provide compression ratios of about 20, have a limited impact on the continuum imaging as the increased noise is less than the image RMS, whereas a 1% error bound (compression ratio of 8) introduces an increase in noise of about an order of magnitude less than the image RMS. For extremely sensitive observations and for very precious data, we would recommend a 0.1% error bound with compression ratios of about 4. These have noise impacts two orders of magnitude less than the image RMS levels. At these levels, the limits are due to instabilities in the deconvolution methods. We compared the results to the alternative compression tool DYSCO, in both the impacts on the images and in the relative flexibility. MGARD provides better compression for similar error bounds and has a host of potentially powerful additional features.

Techniques: interferometric↗

DOE OSTI2025

OpenCRUMS USA: An Open Machine Learning Framework for Characterizing Variability in Aerosol Reanalysis Data

Advances in artificial intelligence (AI) have called for exploring how these techniques can be used for exploring patterns in large climate datasets. To that regard, the U.S. Department of Energy AI for Earth System Predictability (AI4ESP) supported a pilot initiative called the Open Classification of Regimes in the Southeast USA (OpenCRUMS USA) project to explore how AI can be used to characterize modes of spatial variability in large climate datasets. For this study, we focus on comparing two methods for characterizing the modes of spatial variability of surface aerosol concentration over the Houston region: empirical orthogonal functions (EOFs) and layerwise relevance propagation (LRP) applied to a convolutional neural network (CNN) classifier. We show that EOF analysis typically attributes spatial variability modes that span all of southeast Texas, prohibiting the attribution of spatial variability to localized regions. However, using LRP on the CNN classifier resolves the explanatory parameters at a finer spatial resolution than EOFs. This allows for the attribution of the spatial variability of surface aerosols to local regions of organic carbon which was not possible using EOFs. In addition, the LRP analysis also suggests that synoptic-scale transport of dust is most prevalent during anticyclonic and pretrough synoptic conditions as categorized by self-organizing maps.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

C3F: Collaborative Container-based Model Coupling Framework

Solving complex real-world grand challenge problems requires in-depth collaboration of researchers from multiple disciplines. Such collaboration often involves harnessing multiscale and multi-dimensional data and combining models from different fields to simulate systems. However, the progress on this front has been limited mainly due to significant gaps in domain knowledge and tools that are typically employed in silos of the domains. Researchers from different fields face considerable barriers to understanding and reusing each other’s data/models in order to collaborate effectively. For example, in solving the global sustainability problems, researchers from hydrology, climate science, agriculture, and economics need to run their respective models to study different components of the global and local food, energy and water systems while, at the same time, need to interact with other researchers and integrate the results of one model with another. Developing this kind of model coupling workflow calls for (1) a large amount of data being processed and exchanged across domains and organizations, (2) identifying and processing the output of one model to make it ready for integration into another model, (3) controlling the workflow dynamically so that it runs until a certain convergence condition or other criteria is met, and (4) close collaboration among the modelers to explore, tune, and test the configuration and data transformation needed to link the models. We have developed C3F, a flexible collaborative model coupling framework to help researchers accelerate their model integration and linking efforts by leveraging advanced cyberinfrastructure such as high-performance computing and virtual containers. In this paper, we describe our experience and lessons learned in developing this cyberinfrastructure solution to support the linking of Water Balance Model (WBM) and SIMPLE-G agricultural economic model in an NSF funded INFEWS project and a DOE-funded Program on Coupled Human and Earth Systems (PCHES) to study the implications of groundwater scarcity for food-energy-water systems. The C3F model coupling framework can be extended to facilitate other model linkages as well.

containerization↗

DOE OSTI2025

Elemental and isotopic signatures of Asteroid Ryugu support three early Solar System reservoirs

Understanding the number and locations of different reservoirs present in the early Solar System is crucial to understanding the Solar System’s origin and evolution. Previous work has suggested that three unique isotopic reservoirs existed in the early Solar System but subsequent works have challenged that idea. Here we present elemental abundances along with Ca, Ti, Cr, Fe, Ni, and Zn isotopic data from primitive material returned by the Japan Aerospace Exploration Agency’s (JAXA) Hayabusa2 mission to asteroid (162173) Ryugu to make inferences on the Solar System’s early architecture. Data from Ryugu particle A0208 are consistent with a close genetic heritage between Ryugu and CI chondrites. Here, we employ principal component analysis (PCA) on these Ryugu and published meteorite data to demonstrate that Ryugu and CI chondrites are distinct from other known astromaterials, strongly supporting the existence of a third major isotopic reservoir in the early Solar System.

Isotopes↗

DOE OSTI2021

Radiation Detection Data Competition Report

In FY2018 through FY2020, NA-22, the Defense Nuclear Nonproliferation Research and Development Program, funded a Data Science project to develop and implement statistical methodology to effectively host data competitions with the goal of leveraging the opportunity provided by crowdsourcing. By accessing and engaging expertise from a broader research community, there is an opportunity to attract innovative solutions from a variety of different research disciplines to advance the ability to solve important non-proliferation problems. This report summarizes the key results of this project after hosting two data competitions focused on urban radiation detection. The first competition was focused on attracting participants from the U.S. national laboratories, while the second, hosted by TopCoder, was open to the broader international community and awarded prize money to the top 10 competitors. At the start of the project, there was strong interest from NA-22 to explore and develop the capability to host data competitions as a means of leveraging the broader community to solve important nuclear nonproliferation problems. Having a standard data set on which to compare different approaches based on clearly defined criteria was desirable to be able to evaluate the state of solutions for important problems. Initially, it was not clear that it would even be possible logistically and bureaucratically to host a competition with an international field of competitors and to award the prize money needed to attract solutions from top competitors. Happily, a path to host the competitions was ultimately found that allowed this powerful accelerator of improvements to be leveraged.

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2024

Impact of Higher Fidelity Design Iterations on Critical System Criteria

Nuclear criticality experiments are effective at informing the performance of nuclear data libraries across many applications. This work explores the implications of refining critical experiment MCNP models from their low fidelity optimization phase to penultimate neutronic models. Specifically, this work is focused on two series of plutonium fueled experiments funded through internal programs at Los Alamos National Laboratory building off previous efforts under the EUCLID (Experiments Underpinned by Computational Learning for Improvements in Nuclear Data) collaboration. Thales, the first of the two collaborations, is a fast spectrum Ta-reflected plutonium experiment to support operations at PF-4. The second experiment are twin configurations designed to target the intermediate energy cross sections in 239 Pu. Motivation for this experiment stems from the PARallel Approach of Differential and InteGral Measurements (PARADIGM) collaboration which hopes to achieve a significant reduction in 239 Pu cross section uncertainties in the intermediate region.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Geospatial Characterization of Low-Temperature Heating and Cooling Demand in the United States: Preprint

Geothermal resources at temperatures below 150 degrees C have great potential as energy sources for various direct-use applications including heating and cooling in residential and commercial buildings. This study geospatially quantifies U.S. heating and cooling demand in residential, commercial, and manufacturing sectors; heating demand in the agricultural sector; and cooling demand in data centers at the county level through end-use energy consumption, expenditure, and commissioned power analyses. Heating and cooling demand in the residential sector was estimated using energy consumption data obtained from the U.S. Energy Information Administration accounting for different U.S. climate zones. For commercial sector analysis, the end-use major fuel energy intensity at the census division level was disaggregated to the county level with respect to principal building activities. Heating and cooling demand analysis for the manufacturing sector was based on end-use energy consumption for direct-use total process categorized by the North America Industry Classification System. Fuel expenditures in the U.S. Department of Agriculture Farm Production Expenditures were examined for heating demand analysis in the agricultural sector, particularly for the greenhouse, nursery, and floriculture production category. Lastly, commissioned power for data centers in the United States were explored for cooling demand analysis. Results indicated a significant fraction of U.S. primary energy consumption is used for low-temperature heating and cooling applications. Heating and cooling demand in residential and commercial sectors is significantly affected by the number of housing units and climate zone designations, while heating and cooling demand in manufacturing and agricultural sectors and data centers are mainly dependent on the number of facilities and their locations. Maps were generated visualizing where heating and cooling demand is high and, overlain with geothermal resource maps, can indicate locations where geothermal energy can supply this heating and cooling demand.

cooling demand↗

DOE OSTI2023

Geospatial Characterization of Low-Temperature Heating and Cooling Demand in the United States

Geothermal resources at temperatures below 150 degrees C have great potential as energy sources for various direct-use applications including heating and cooling in residential and commercial buildings. This study geospatially quantifies U.S. heating and cooling demand in residential, commercial, and manufacturing sectors; heating demand in the agricultural sector; and cooling demand in data centers at the county level through end-use energy consumption, expenditure, and commissioned power analyses. Heating and cooling demand in the residential sector was estimated using energy consumption data obtained from the U.S. Energy Information Administration accounting for different U.S. climate zones. For commercial sector analysis, the end-use major fuel energy intensity at the census division level was disaggregated to the county level with respect to principal building activities. Heating and cooling demand analysis for the manufacturing sector was based on end-use energy consumption for direct-use total process categorized by the North America Industry Classification System. Fuel expenditures in the U.S. Department of Agriculture Farm Production Expenditures were examined for heating demand analysis in the agricultural sector, particularly for the greenhouse, nursery, and floriculture production category. Lastly, commissioned power for data centers in the United States were explored for cooling demand analysis. Results indicated a significant fraction of U.S. primary energy consumption is used for low-temperature heating and cooling applications. Heating and cooling demand in residential and commercial sectors is significantly affected by the number of housing units and climate zone designations, while heating and cooling demand in manufacturing and agricultural sectors and data centers are mainly dependent on the number of facilities and their locations. Maps were generated visualizing where heating and cooling demand is high and, overlain with geothermal resource maps, can indicate locations where geothermal energy can supply this heating and cooling demand.

cooling demand↗

DOE OSTI2021

Machine Learning for Improved Availability of the SNS Klystron High Voltage Converter Modulators

Beam availability has increased at the SNS, however, the targeted availability is greater than 95 %, while the SNS has failed to meet lower targets in the past. The HVCM used to power the linac klystrons have been one source of lost beam time and was chosen to explore using AI/ML techniques to improve reliability. Among the possibilities being explored are automating the tuning of HVCMs and predicting component failures such as capacitor aging, rectifier assemblies containing hundreds of diodes, and insulating oil degradation. The methodology pursued includes data cleaning, de-noising, post-analysis data labeling, and machine learning model development. We explore using Long Short-Term Memory and autoencoders for anomaly detection and prognostication used to schedule maintenance. We evaluate the use of model regularizers and constraints to improve the performance of the model and investigate methods to estimate the uncertainty of the models to provide a robust prediction with statistical interoperability. This paper describes the operational experience and known failures of the HVCMs and the proposed ML methodology and the preliminary results of training the AI/ML algorithms.

Pappas, G. C.↗

DOE OSTI2025

MSD CoP Webinar: Energy and AI

Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Abstract: Projections of the need for new data centers to support Artificial Intelligence (AI) are large but highly uncertain. Recent projections indicate up to a 15% annual growth rate in data center electricity demand within the next 5-10 years. Given that most electric utilities are required to have a reserve margin of roughly the same magnitude as the projected growth in demand, these new data center loads could soon threaten resource adequacy and reliability unless data centers build their own generation, interruptible loads are negotiated, commensurate new capacity and/or transmission is built, or some combination of these options. Similarly, depending on the cooling technology and geographic location of new data centers, they could threaten water adequacy in water scarce regions. This webinar will provide an overview of the interactions between energy and AI and highlight two MSD projects exploring the grid and water implications of new data centers to support AI. Presenters : Dr. Casey Burleyson (Pacific Northwest National Laboratory); Dr. Stephanie Morris (Pacific Northwest National Laboratory); Kendall Mongird (Pacific Northwest National Laboratory) Moderator: Patrick M. Reed (MSD CoP Facilitation Team) This webinar was held on: June 16th, 2025 from 1-2 PM EST.

Artificial Intelligence↗

DOE OSTI2021

Machine Learning for Improved Availability of the SNS Klystron High Voltage Converter Modulators

Beam availability has increased at the SNS, however, the targeted availability is greater than 95 %, while the SNS has failed to meet lower targets in the past. The HVCM used to power the linac klystrons have been one source of lost beam time and was chosen to explore using AI/ML techniques to improve reliability. Among the possibilities being explored are automating the tuning of HVCMs and predicting component failures such as capacitor aging, rectifier assemblies containing hundreds of diodes, and insulating oil degradation. The methodology pursued includes data cleaning, de-noising, post-analysis data labeling, and machine learning model development. We explore using Long Short-Term Memory and autoencoders for anomaly detection and prognostication used to schedule maintenance. We evaluate the use of model regularizers and constraints to improve the performance of the model and investigate methods to estimate the uncertainty of the models to provide a robust prediction with statistical interoperability. This paper describes the operational experience and known failures of the HVCMs and the proposed ML methodology and the preliminary results of training the AI/ML algorithms.

Pappas, Chris↗