Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Visualization & Storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Integrated System and Application Continuous Performance Monitoring and Analysis Capability

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia's Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia's ElectroMagnetic Plasma In Realistic Environments ( EMPIRE ) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia's HPC users.

97 MATHEMATICS AND COMPUTING↗

Integrated System and Application Continuous Performance Monitoring and Analysis Capability (Final)

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia’s Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia’s ElectroMagnetic Plasma In Realistic Environments (EMPIRE) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia’s HPC users.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Visual Examination of Aluminum Containers for Extended Wet Storage of Non-Aluminum-Clad Spent Nuclear Fuel (FY 2021)

The goal of the Augmented Monitoring and Condition Assessment Program (AMCAP) program is to provide a condition assessment of the storage containers for non-aluminum-clad spent nuclear fuel (NASNF) in L Basin and demonstrate continued safe storage of all NASNF pending retrieval for ultimate disposition. The storage configuration of NASNF is vulnerable to inside-out corrosion attack. The approach being used in AMCAP is to develop and deploy examination methods for remote, in-situ inspection of the bundle storage in VTS and for OSC storage in the OSC racks. A method for Visual Examination (VE) using video-recording cameras was readied as part of the full-scale mockup development, and was deployed in L Basin for the initial condition assessment work. This report describes the development and the deployment of the VE for inspection of selected bundles in L Basin. The first deployment of the VE system was for visually examining five spent fuel bundles. Bundles were chosen based on their vulnerability in terms of inner fuel contents, history, storage duration, storage configuration, and possibility of inside out corrosion due to galvanic coupling of fuel cladding with the aluminum bundle wall.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

IOPro: a parallel I/O profiling and visualization framework for high-performance storage systems

Efficient execution of large-scale scientific applications requires high-performance computing systems designed to meet the I/O requirements. To achieve high-performance, such data-intensive parallel applications use a multi-layer layer I/O software stack, which consists of high-level I/O libraries such as PnetCDF and HDF5, the MPI library, and parallel file systems. To design efficient parallel scientific applications, understanding the complicated flow of I/O operations and the involved interactions among the libraries is quintessential. Such comprehension helps identify I/O bottlenecks and thus exploits the potential performance in different layers of the storage hierarchy. To profile the performance of individual components in the I/O stack and to understand complex interactions among them, we have implemented a GUI-based integrated profiling and analysis framework, IOPro. IOPro automatically generates an instrumented I/O stack, runs applications on it, and visualizes detailed statistics based on the user-specified metrics of interest. We present experimental results from two different real-life applications and show how our framework can be used in practice. By generating an end-to-end trace of the whole I/O stack and pinpointing I/O interference, IOPro aids in understanding I/O behavior and improving the I/O performance significantly.

97 MATHEMATICS AND COMPUTING↗

Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures

A key trend facing extreme-scale computational science is the widening gap between computational and I/O rates, and the challenge that follows is how to best gain insight from simulation data when it is increasingly impractical to save it to persistent storage for subsequent visual exploration and analysis. One approach to this challenge is centered around the idea of in situ processing, where visualization and analysis processing is performed while data is still resident in memory. This paper examines several key design and performance issues related to the idea of in situ processing at extreme scale on modern platforms: Scalability, overhead, performance measurement and analysis, comparison and contrast with a traditional post hoc approach, and interfacing with simulation codes. We illustrate these principles in practice with studies, conducted on large-scale HPC platforms, that include a miniapplication and multiple science application codes, one of which demonstrates in situ methods in use at greater than 1M-way concurrency.

Ayachit, Utkarsh↗

Visualizing the Surface Infrastructure Used to Move 2 MtCO2/year from the Dakota Gasification Company to the Weyburn CO2 Enhanced Oil Recovery Project: Version of July 1, 2009

Google Earth Pro has been employed to create an interactive flyover of the world’s largest operational carbon dioxide capture and storage project. The visualization focuses on the transport and storage of 2 MtCO2/year which is captured from the Dakota Gasification Facility (Beula, North Dakota) and transported 205 miles and injected into the Weyburn oil field in Southeastern Saskatchewan.

02 PETROLEUM↗

THE STORAGE AND DISSIPATION OF MAGNETIC ENERGY IN THE QUIET SUN CORONA DETERMINED FROM SDO/HMI MAGNETOGRAMS

In recent years, higher cadence, higher resolution observations have revealed the quiet-Sun photosphere to be complex and rapidly evolving. Since magnetic fields anchored in the photosphere extend up into the solar corona, it is expected that the small-scale coronal magnetic field exhibits similar complexity. For the first time, the quiet-Sun coronal magnetic field is continuously evolved through a series of non-potential, quasi-static equilibria, deduced from magnetograms observed by the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory, where the photospheric boundary condition which drives the coronal evolution exactly reproduces the observed magnetograms. The build-up, storage, and dissipation of magnetic energy within the simulations is studied. We find that the free magnetic energy built up and stored within the field is sufficient to explain small-scale, impulsive events such as nanoflares. On comparing with coronal images of the same region, the energy storage and dissipation visually reproduces many of the observed features. The results indicate that the complex small-scale magnetic evolution of a large number of magnetic features is a key element in explaining the nature of the solar corona.

79 ASTRONOMY AND ASTROPHYSICS↗

Grid Data Crossing Platform (GriD-Xing)

This project is to develop a data management and analytics platform called Grid Data Crossing (GriD-Xing) that integrates data processing, storage, analysis, and visualization functionalities for users at industry (e.g., EMS and DMS) and universities. It intends to increase the applicability of large set of multi-source data (e.g., SCADA, PMU, μPMU, AMI, inverter, load/DER forecasting, and cyber data) and eventually improve the resilience, reliability, and security of critical energy infrastructure. GriD-Xing also intends to be a research platform to identify the usefulness of large multi-source data via case studies of interest to industry (e.g., 2011 southwest cold weather and 2016 Blue Cut fire events). In the first year, the team focused on the development and demonstration of large multi-source data (e.g., PMU, μPMU, SCADA, AMI, and cyber data) processing, storage, and analysis. Specifically, the team designed GriD-Xing framework in functional and structural aspects, developed GriD-Xing algorithms for large multi-source grid data processing, storage, and analysis, and finally demonstrated GriD-Xing prototype algorithms performance through a series of tests and case studies using real-world and simulation data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

In-situ visual inspection of the CASTOR V/21 used nuclear fuel dry storage cask: a demonstration of camera system capabilities - 15505

On June 11, 2014 the Gesellschaft fur Nuklear Service (GNS) CASTOR V/21 spent fuel dry storage cask was inspected in-situ at the Idaho Nuclear Technology and Engineering Center (INTEC). The purpose of this inspection is to demonstrate and document the technology necessary to perform inspections of spent fuel dry storage casks. The inspection includes internal temperature measurement, internal radiation measurement, and a visual examination using a bore-scope type camera system inserted through the cask fill/drain penetration. It is concluded from this inspection that in-situ inspections of bolted lid spent fuel storage containers is possible using current camera technologies. Using 3D Phase Measurement Technology, measurements of entities such as cracks or corrosion are possible. The internal surfaces of the CASTOR V/21 cask that were inspected were free of corrosion and exhibited no evidence of degradation. One crack identified in previous inspections was examined and appeared the same as in the previous inspections. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

GRACC: GRid ACcounting Collector

The OSG has long maintained a central accounting system called Gratia. It uses small probes on each computing and storage resource in order to collect resource usage. The probes report to a central collector which stores the usage in a database. The database is then queried to generate reports. As the OSG aged, the size of the database grew very large. It became too large for the database technology to efficiently query to generate detailed reports. The design of a replacement requires data storage that could be queried efficiently to generate multi-year reports. Additionally, it requires flexibilityto add new attributes to the collected data.In this paper we will describe updates to the GRACC architecture in the last 18 months. GRACC uses modern web technologies that were designed for large data storage, query, and visualization. That includes the open source database Elasticsearch, message broker software RabbitMQ, and Grafana and Kibana as data visualization platforms. It uses multiple agents that perform operations on the data to transform it for easier querying and summarization.

Weitzel, Derek↗

A near infra-red video system as a protective diagnostic for electron cyclotron resonance heating operation in the Wendelstein 7-X stellarator

The Wendelstein 7-X stellarator is a large nuclear fusion device based at Max-Planck-Institut für Plasmaphysik in Greifswald in Germany. The main plasma heating system for steady state operation in W7-X is electron cyclotron resonance heating (ECRH). During operation, part of plama facing components will be directly heated by the non-absorbed power of 1 MW rf beams of ECRH. In order to avoid damages of such components made of graphite tiles during the first operational phase, a near infra-red video system has been developed as a protective diagnostic for safe and secure ECRH operation. Both the mechanical design housing the camera and the optical system are very flexible and respect the requirements of steady state operation. The full system including data acquisition and control system has been successfully tested in the vacuum vessel, including on-line visualization and data storage of the four cameras equipping the ECRH equatorial launchers of W7-X.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

CPU Isosurface Ray Tracing of Adaptive Mesh Refinement Data

Adaptive mesh refinement (AMR) is a key technology for large-scale simulations that allows for adaptively changing the simulation mesh resolution, resulting in significant computational and storage savings. However, visualizing such AMR data poses a significant challenge due to the difficulties introduced by the hierarchical representation when reconstructing continuous field values. Here, we detail a comprehensive solution for interactive isosurface rendering of block-structured AMR data. We contribute a novel reconstruction strategy—the octant method—which is continuous, adaptive and simple to implement. Furthermore, we present a generally applicable hybrid implicit isosurface ray-tracing method, which provides better rendering quality and performance than the built-in sampling-based approach in OSPRay. Finally, we integrate our octant method and hybrid isosurface geometry into OSPRay as a module, providing the ability to create high-quality interactive visualizations combining volume and isosurface representations of BS-AMR data. Lastly, we evaluate the rendering performance, memory consumption and quality of our method on two gigascale block-structured AMR datasets.

97 MATHEMATICS AND COMPUTING↗

Scientific Computing Strategic Plan for the Idaho National Laboratory

Scientific computing is a critical foundation of modern science. Without innovations in the field of computational science, the essential missions of the Department of Energy (DOE) would go unrealized. Taking a leadership role in such innovations is Idaho National Laboratory’s (INL’s) challenge and charge, and is central to INL’s ongoing success. Computing is an essential part of INL’s future. DOE science and technology missions rely firmly on computing capabilities in various forms. Modeling and simulation, fueled by innovations in computational science and validated through experiment, are a critical foundation of science and engineering. Big data analytics from an increasing number of widely varied sources is opening new windows of insight and discovery. Computing is a critical tool in education, science, engineering, and experiments. Advanced computing capabilities in the form of people, tools, computers, and facilities, will position INL competitively to deliver results and solutions on important national science and engineering challenges. A computing strategy must include much more than simply computers. The foundational enabling component of computing at many DOE national laboratories is the combination of a showcase like data center facility coupled with a very capable supercomputer. In addition, network connectivity, disk storage systems, and visualization hardware are critical and generally tightly coupled to the computer system and co located in the same facility. The existence of these resources in a single data center facility opens the doors to many opportunities that would not otherwise be possible.

97 MATHEMATICS AND COMPUTING↗

Energy Systems Integration Facility Stewardship Summary: Fiscal Year 2022

A summary of NREL's good stewardship of the nationally unique Energy Systems Integration Facility (ESIF) highlighting performance metrics, infrastructure and capability upgrades, and examples of R&D impact. In fiscal year 2022, ESIF researchers made breakthroughs in everything from long-duration energy storage and cybersecurity visualizations to a world record in heavy-duty hydrogen vehicle fueling, and built out research assets to advance microgrid operation and controls, renewable hydrogen production, electric vehicles, energy-efficient buildings, and more.

ADVANCED PROPULSION SYSTEMS,ENERGY CONSERVATION, C↗

GUIDE: A Scalable Information Directory Service to Collect, Federate, and Analyze Logs for Operational Insights into a Leadership HPC Facility

In this paper, we describe the GUIDE framework to collect, federate, and analyze log data from the Oak Ridge Leadership Computing Facility's (OLCF), and how we use it derive insights into center operations. We collectsystem logs and extract monitoring data at every level of the various OLCF subsystems, and have developed a suite of pre-processing tools to make the raw data consumable. The cleansed logs are then ingested and federated into a central, scalable data warehouse, Splunk, that offers storage, indexing, querying, and visualization capabilities. We have further developed and deployed a set of analytics tools to analyze these multiple disparate log streams in concert, and derive operational insights. We describe our experience from developing and deploying the GUIDE infrastructure, and deriving valuable insights on the various subsystems, based on two years of operations in the production OLCF environment.

Vazhkudai, Sudharshan↗

DOECGF 2010 Site Report

The Data group provides data analysis and visualization support to its customers. This consists primarily of the development and support of VisIt, a data analysis and visualization tool. Support ranges from answering questions about the tool, providing classes on how to use the tool, and performing data analysis and visualization for customers. The Information Management and Graphics Group supports and develops tools that enhance our ability to access, display, and understand large, complex data sets. Activities include applying visualization software for large scale data exploration; running video production labs on two networks; supporting graphics libraries and tools for end users; maintaining PowerWalls and assorted other displays; and developing software for searching and managing scientific data. Researchers in the Center for Applied Scientific Computing (CASC) work on various projects including the development of visualization techniques for large scale data exploration that are funded by the ASC program, among others. The researchers also have LDRD projects and collaborations with other lab researchers, academia, and industry. The IMG group is located in the Terascale Simulation Facility, home to Dawn, Atlas, BGL, and others, which includes both classified and unclassified visualization theaters, a visualization computer floor and deployment workshop, and video production labs. We continued to provide the traditional graphics group consulting and video production support. We maintained five PowerWalls and many other displays. We deployed a 576-node Opteron/IB cluster with 72 TB of memory providing a visualization production server on our classified network. We continue to support a 128-node Opteron/IB cluster providing a visualization production server for our unclassified systems and an older 256-node Opteron/IB cluster for the classified systems, as well as several smaller clusters to drive the PowerWalls. The visualization production systems includes NFS servers to provide dedicated storage for data analysis and visualization. The ASC projects have delivered new versions of visualization and scientific data management tools to end users and continue to refine them. VisIt had 4 releases during the past year, ending with VisIt 2.0. We released version 2.4 of Hopper, a Java application for managing and transferring files. This release included a graphical disk usage view which works on all types of connections and an aggregated copy feature for quickly transferring massive datasets quickly and efficiently to HPSS. We continue to use and develop Blockbuster and Telepath. Both the VisIt and IMG teams were engaged in a variety of movie production efforts during the past year in addition to the development tasks.

97 MATHEMATICS AND COMPUTING↗

Understanding Performance-Quality Trade-offs in Scientific Visualization Workflows with Lossy Compression

The cost of I/O is a significant challenge on current supercomputers, and the trend is likely to continue into the foreseeable future. This challenge is amplified in scientific visualization because of the requirement to consume large amounts of data before processing can begin. Lossy compression has become an important technique in reducing the cost of performing I/O. In this paper we consider the implications of using compressed data for visualization within a scientific workflow. We use visualization operations on simulation data that is reduced using three different state-of-the-art compression techniques. We study the storage efficiency and preservation of visualization features on the resulting compressed data, and draw comparisons between the three techniques used. Our contributions can help inform both scientists and researchers in the use and design of compression techniques for preservation of important visualization details.

Chen, Jieyang↗