Ease of use and deployment for a fast, scalable data movement infrastructure
Explore the source record for details and available documents.
42 ENGINEERING↗
SEARCH · Engineering Papers
Results for “data analysis, visualization”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
MSD CoP Webinar: "Advances in MSD-LIVE to Support the MSD Community of Practice"
Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Advances in MSD-LIVE to Support the MSD Community of Practice Presenters: Casey Burleyson and Zoe Guillen (Pacific Northwest National Laboratory) Abstract: The MultiSector Dynamics Living, Intuitive, Value-adding, Environment (MSD-LIVE; msdlive.org) is a cloud-based data management system and advanced computing platform that enables MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and workflows within the MSD Community of Practice. Recently, several high-profile datasets have attracted many new users to MSD-LIVE. This webinar has two goals: 1) To refamiliarize the MSD community and new users with the components of the platform (e.g., the data repository, model training notebooks, and data dashboards) and to highlight examples of how these components are advancing MSD science and 2) To demonstrate new features in v3 of the platform, released in late 2025. The main new feature in v3 is the ability to interactively explore data in MSD-LIVE without downloading it. MSD-LIVE users can now click a button in our data repository and launch a blank Jupyter notebook with access to the underlying data on AWS. Users can use the notebook to write analysis, visualization, or subsetting routines that process the data directly on the AWS cloud. We also added a GitHub integration feature that allows users to share analysis or visualization code they develop with the community of MSD-LIVE users. The webinar will wrap up with a look at what's coming next for MSD-LIVE in 2026. Moderator: Patrick M. Reed (MSD CoP Facilitation Team) This webinar was held on: May 12th, 2026 from 1-2 PM EST.
Open Science↗
Data-Driven Computation of Probabilistic Marching Cubes for Efficient Visualization of Level-Set Uncertainty
Uncertainty visualization is an important emerging research area. Being able to visualize data uncertainty can help scientists improve trust in analysis and decision-making. However, visualizing uncertainty can add computational overhead, which can hinder the efficiency of analysis. In this paper, we propose novel data-driven techniques to reduce the computational requirements of the probabilistic marching cubes (PMC) algorithm. PMC is an uncertainty visualization technique that studies how uncertainty in data affects level-set positions. However, the algorithm relies on expensive Monte Carlo (MC) sampling for the multivariate Gaussian uncertainty model because no closed-form solution exists for the integration of multivariate Gaussian. In this work, we propose the eigenvalue decomposition and adaptive probability model techniques that reduce the amount of MC sampling in the original PMC algorithm and hence speed up the computations. Our proposed methods produce results that show negligible differences compared with the original PMC algorithm demonstrated through metrics, including root mean squared error, maximum error, and difference images. We demonstrate the performance and accuracy evaluations of our data-driven methods through experiments on synthetic and real datasets.
Athawale, Tushar↗
Visualizing how inclusion of higher reciprocal space in SWAXS data analysis improves shape restoration of biomolecules: case of lysozyme
Not provided.
Biochemistry & Molecular Biology↗
ALPINE Overview
Data-driven sampling enables probabilistic identification of interesting regions in the data automatically, prioritizing important regions. Applied in situ to Nyx, important halo regions are preserved.
97 MATHEMATICS AND COMPUTING↗
FatPlants: a comprehensive information system for lipid-related genes and metabolic pathways in plants
Abstract FatPlants, an open-access, web-based database, consolidates data, annotations, analysis results, and visualizations of lipid-related genes, proteins, and metabolic pathways in plants. Serving as a minable resource, FatPlants offers a user-friendly interface for facilitating studies into the regulation of plant lipid metabolism and supporting breeding efforts aimed at increasing crop oil content. This web resource, developed using data derived from our own research, curated from public resources, and gleaned from academic literature, comprises information on known fatty-acid-related proteins, genes, and pathways in multiple plants, with an emphasis on Glycine max, Arabidopsis thaliana, and Camelina sativa. Furthermore, the platform includes machine-learning based methods and navigation tools designed to aid in characterizing metabolic pathways and protein interactions. Comprehensive gene and protein information cards, a Basic Local Alignment Search Tool search function, similar structure search capacities from AphaFold, and ChatGPT-based query for protein information are additional features. Database URL: https://www.fatplants.net/
59 BASIC BIOLOGICAL SCIENCES↗
A Dynamic Contingency Analysis Visualization Tool
We are developing a web-based visualization that shows the results of running a contingency analysis on a power grid system. Power grid analysists will be able to use this tool to visualize power event simulations and be better prepared for contingencies that may arise. The tool consists of a map, showing the power grid and its current state, and tables and charts showing the current status of various elements on the grid. The user can iterate over a number of cascading contingencies to visualize how the power grid will change under various scenarios.
Contingency analysis, power grid, smart grid, visu↗
py4DSTEM: A Software Package for Four-Dimensional Scanning Transmission Electron Microscopy Data Analysis
Scanning transmission electron microscopy (STEM) allows for imaging, diffraction, and spectroscopy of materials on length scales ranging from microns to atoms. By using a high-speed, direct electron detector, it is now possible to record a full two-dimensional (2D) image of the diffracted electron beam at each probe position, typically a 2D grid of probe positions. These 4D-STEM datasets are rich in information, including signatures of the local structure, orientation, deformation, electromagnetic fields, and other sample-dependent properties. However, extracting this information requires complex analysis pipelines that include data wrangling, calibration, analysis, and visualization, all while maintaining robustness against imaging distortions and artifacts. In this paper, we present py4DSTEM, an analysis toolkit for measuring material properties from 4D-STEM datasets, written in the Python language and released with an open-source license. We describe the algorithmic steps for dataset calibration and various 4D-STEM property measurements in detail and present results from several experimental datasets. We also implement a simple and universal file format appropriate for electron microscopy data in py4DSTEM, which uses the open-source HDF5 standard. We hope this tool will benefit the research community and help improve the standards for data and computational methods in electron microscopy, and we invite the community to contribute to this ongoing project.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Data processing pipeline for Tianlai experiment
The Tianlai project is a 21cm intensity mapping experiment for detecting dark energy by measuring the baryon acoustic oscillation (BAO) features in the large scale structure power spectrum. This experiment provides an opportunity to test the data processing methods for cosmological 21cm signal extraction, which is still a great challenge in current radio astronomy research. The 21cm signal is much weaker than the foregrounds and easily aected by the imperfections in the instrumental responses. Furthermore, processing the large volumes of interferometer data poses a practical challenge. We have developed a data processing pipeline called tlpipe to process the drift scan survey data from the Tianlai experiment. It performs oine data processing tasks such as radio frequency interference (RFI) agging, array calibration, binning, and map-making, etc. It also includes utility functions needed for the data analysis, such as data selection, transformation, visualization and others. A number of new algorithms are implemented, for example the eigenvector decomposition method for array calibration and the Tikhnov regularization for m-mode analysis. In this paper we describe the design and implementation of the pipeline and illustrate its functions with some analysis of real data. Finally, we outline directions for future development of this publicly code.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Assessment of Local Observation of Atomic Ordering in Alloys via the Radial Distribution Function: A Computational and Experimental Approach
As a powerful analytical technique, atom probe tomography (APT) has the capacity to acquire the spatial distribution of millions of atoms from a complex sample. However, extracting information at the Ångstrom-scale on atomic ordering remains a challenge due to the limits of the APT experiment and data analysis algorithms. The development of new computational tools enable visualization of the data and aid understanding of the physical phenomena such as disorder of complex crystalline structures. Here, we report progress towards this goal using two steps. We describe a computational approach to evaluate atomic ordering in the crystal structure by generating radial distribution functions (RDF). Atomic ordering is rendered as the Fractional Cumulative Radial Distribution Function (FCRDF) which allows for greater visibility of local compositions at short range in the structure. Further, we accommodate in the analysis additional parameters such as uncertainty in the atomic coordinates and the atomic abundance to ascertain short-range ordering in APT data sets. We applied the FCRDF analysis to synthetic and experimental APT data sets for Ni 3 Al. The ability to observe a signal of atomic ordering consistent with the known L1 2 crystal structure is heavily dependent on spatial uncertainty, irrespective of abundance. Detection of atomic ordering is subject to an upper limit of spatial uncertainty of atoms described with Gaussian distributions with a standard deviation of 1.3 Å. The FCRDF analysis was also applied to the APT data set for a six-component alloy, Al 1.3 CoCrCuFeNi. In this case, we are currently able to visualize elemental segregation at the nanoscale, though unambiguous identification of atomic ordering at the Ångstrom (nearest-neighbor) scale remains a goal.
36 MATERIALS SCIENCE↗
Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures
Abstract Large biomolecular structures are being determined experimentally on a daily basis using established techniques such as crystallography and electron microscopy. In addition, emerging integrative or hybrid methods (I/HM) are producing structural models of huge macromolecular machines and assemblies, sometimes containing 100s of millions of non-hydrogen atoms. The performance requirements for visualization and analysis tools delivering these data are increasing rapidly. Significant progress in developing online, web-native three-dimensional (3D) visualization tools was previously accomplished with the introduction of the LiteMol suite and NGL Viewers. Thereafter, Mol* development was jointly initiated by PDBe and RCSB PDB to combine and build on the strengths of LiteMol (developed by PDBe) and NGL (developed by RCSB PDB). The web-native Mol* Viewer enables 3D visualization and streaming of macromolecular coordinate and experimental data, together with capabilities for displaying structure quality, functional, or biological context annotations. High-performance graphics and data management allows users to simultaneously visualise up to hundreds of (superimposed) protein structures, stream molecular dynamics simulation trajectories, render cell-level models, or display huge I/HM structures. It is the primary 3D structure viewer used by PDBe and RCSB PDB. It can be easily integrated into third-party services. Mol* Viewer is open source and freely available at https://molstar.org/.
Sehnal, David↗
An Integrated Testbed for Trojans in Printed Circuit Boards with Fuzzing Capabilities
This paper showcases an all-in-one testing environment that combines Trojan detection and fuzzing capabilities for printed circuit boards using the OpenPLC “NYU Trojan Edition” and a dedicated Trojan detection framework. The demo system is self-contained and equipped with two OpenPLC-based boards (one with a Trojan and one without), and automated tools for inserting the Trojan and collecting side-channel data. We developed a graphical user interface for interactive Trojan selection, data visualization, and anomaly detection analysis.
Trojan detection, anomaly detection, PCB, timing l↗
gcxgclab: Two Dimensional Gas Chromatography Preprocessing and Analysis
The goal of gcxgclab is to provide a comprehensive program for preprocessing and analysis of two dimensional gas chromatography data. It is equipped with functions for baseline correction, smoothing, peak identification, peak alignment, identification of EICs, Mass Spectra, targeted analysis, compound identification with NIST and non-targeted analysis, plus plotting and data visualization.
GAMBLE, STEPHANIE↗
Community Resilience Indicator Analysis: Commonly Used Indicators from Peer-Reviewed Research (Updated for Research Published 2003-2021)
In 2017, FEMA’s National Integration Center (NIC) Technical Assistance (TA) Branch identified a need to establish a data-driven basis for prioritizing locations for TA investment and guiding local emergency management planning. To achieve this goal, FEMA tasked Argonne National Laboratory (Argonne) with identifying commonly used indicators of community resilience across the landscape of published peer-reviewed research. FEMA and Argonne completed the first Community Resilience Indicator Analysis (CRIA) in 2018 and repeated the process in 2022. The CRIA process begins with a literature review and cataloguing of published peer-reviewed assessment methodologies on social vulnerability and community resilience. The literature review findings are then filtered by inclusion criteria established by the CRIA research team to ensure the methodologies are: (1) Quantitative, (2) Data and methodology are publicly available, (3) Calculated at the county level or lower, (4) Examine generalized hazard risk (rather than a singular hazard), and (5) Focused on pre-disaster community conditions. After this, the research team identifies the commonly used indicators across these methodologies and selects the best data source for each indicator. Finally, the research team bins the data for visual display, conducts a correlation analysis and creates a composite index, the FEMA Community Resilience Index (FEMA CRI). In 2018, the CRIA identified eight resilience and vulnerability assessment methodologies and 20 commonly used indicators (indicators used in three or more of the eight methodologies). The FEMA CRI in 2018 was created from these 20 indicators and was produced for at the county level. The 2022 CRIA updated the literature review to expand the list of methodologies examined and followed the same process, resulting in an analysis of 14 methodologies published between 2003 and 2021 and 22 indicators identified as commonly used (indicators used in five or more of the 14 methodologies). In 2022, the research team produced the FEMA CRI at the county and the census tract levels. To make the CRIA data more accessible and more actionable, each individual indicator and the FEMA CRI is binned and included in FEMA’s Resilience Analysis and Planning Tool (RAPT). RAPT enables emergency managers and community partners to quickly visualize relative differences in potential resilience by county, tribe and census tract. By reviewing the data for each of these 22 indicators individually, emergency managers can gain insights for targeted outreach strategies, planning, mitigation investments and response and recovery operations. Communities, regional governments and others can use this data to better understand potential challenges to resilience. As the social science field of examining and validating indicators of resilience evolves, FEMA will update RAPT to provide emergency managers and community partners with additional data and tools to inform planning, mitigation, response and recovery. It is important to understand that the role of the emergency manager is not to change or to “improve” the data, but to plan appropriately for the community characteristics reflected in the data. These datasets are community characteristics that researchers have identified as important considerations for resilience. For example, people with disabilities may have greater challenges to be resilient to disasters. If a community has a high population of people with disabilities, the emergency manager(s) may need to create tailored preparedness outreach programs and strategies to ensure those residents have support if evacuation is necessary. Rather than label these indicators as an absolute measure of resilience, FEMA considers “potential challenges to resilience” a better frame to understand these indicators. Everyone is vulnerable to disasters. While scholars theorize that certain characteristics may make an individual or a household more socially vulnerable, the data does not reflect measures that individuals and/or communities have taken to address potential challenges, such as emergency management planning and outreach or household preparedness measures. To aid emergency managers in understanding how to use these indicators, calling them potential challenges to resilience supports a more positive and strategic application of the data in all phases of emergency management.
99 GENERAL AND MISCELLANEOUS↗
Visualizing metagenomic and metatranscriptomic data: A comprehensive review
The fields of Metagenomics and Metatranscriptomics involve the examination of complete nucleotide sequences, gene identification, and analysis of potential biological functions within diverse organisms or environmental samples. Despite the vast opportunities for discovery in metagenomics, the sheer volume and complexity of sequence data often present challenges in processing analysis and visualization. This article highlights the critical role of advanced visualization tools in enabling effective exploration, querying, and analysis of these complex datasets. Emphasizing the importance of accessibility, the article categorizes various visualizers based on their intended applications and highlights their utility in empowering bioinformaticians and non-bioinformaticians to interpret and derive insights from meta-omics data effectively.
59 BASIC BIOLOGICAL SCIENCES↗
Treatment of Tilted Sonar Data for Salt Cavern Analysis
Structural modeling and visualization of salt caverns requires three-dimensional representations. These representations are typically produced from sonar surveys conducted by companies that then produce a report of depths, distances, and volumes. There are multiple formats that are vendor dependent, and, as technology improves, there have been changes from only horizontal surveys to inclined shots for ceilings and floors to mid-cavern inclined shots. For geomechanical modeling, leaching predictions, and cavern stability visualizations, Sandia has previously written in-house software, called SONAR8, that created a consistent geometry format from the processed sonar reports. However, the increase in the need for mid-cavern inclined surveys led to the discovery of certain limitations in that code. This report describes methods used to process the multiple different formats to handle inclined shots in a consistent and accurate manner in our modeling efforts. A set of file formats and a database schema that was developed for this work is also documented in the appendices.
02 PETROLEUM↗
Opportunities for an Integrated Web-Based Workbench for Data Access and Analysis - 20244
Management of environmental issues can require integration of multiple types of data and information, conducting data analysis and interpretation, and providing data visualization for effective communications. These data elements are important for site management to support regulator interactions and provide defensibility for remedial decisions. Databases and information repositories are core elements of managing data; however, efficient data access and analysis also enable effective site management. The U.S. Department of Energy (DOE) Hanford Site is an example of a complex site with a voluminous quantity of environmental data and a need for efficient site management. Different tiers of data and information tools have been developed and deployed to address site needs. These tools are configured for ready access via the web site interfaces and meet the rigorous quality requirements for environmental site management. Evolving efforts are focused on an integrated platform to meet site environmental management needs. In this platform, users can access site information at multiple levels of detail based on their need and permissions, so that data and associated analyses are presented within the context of the site mission and the user's management or technical needs. This concept is not only applicable at individual sites like Hanford but also applicable at other sites within the DOE complex. An integrated web-based architecture that links data visualization, data analytics, and management tools can provide holistic access to large data sets, minimize complexity, and maximize interactivity and technical communication. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Driving Next-Generation Workflows from the Data Plane
We observe the emergence of a new generation of scientific workflows that process data produced at a sustained rate by scientific instruments and large scale numerical simulations. This data is consumed by multiple analysis, visualization, or Machine Learning components not only to enable inference and justify the scientific program, but also to monitor and steer the evolution of these experiments. In such workflows, moving intermediate data efficiently is key to performance, more than efficiently scheduling computational tasks. However, most traditional workflow management systems focus on optimizing task scheduling and then deal with data management, assuming a “move little, compute for long” model, which makes them unfit to the efficient management of this new generation of workflows. Therefore, we advocate for a new way to manage scientific workflows. We propose to consider an efficiently and independently managed data plane that can store and stream data. Workflows compute components, in the application plane can then interact with the data plane, abstracted from complexities of data management. Then, the role of a workflow management system would become that of a control plane that allows users to connect services together to execute the workflow and manages connections between the application and data planes. In this position paper, we characterize several next-generation workflow motifs and describe how their interaction with the data plane is a challenge to traditional workflow management systems. Then, we express a set of requirements that a workflow management system should meet to efficiently manage next-generation workflows at different scales. Based on these requirements, we expose our vision of driving next-generation workflows from the data plane and list remaining open challenges.
Suter, Fred↗