Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

Alignment-free genome comparison enables accurate geographic sourcing of white oak DNA

Background: The application of genomic data and bioinformatics for the identification of restricted or illegally-sourced natural products is urgently needed. The taxonomic identity and geographic provenance of raw and processed materials have implications in sustainable-use commercial practices, and relevance to the enforcement of laws that regulate or restrict illegally harvested materials, such as timber. Improvements in genomics make it possible to capture and sequence partial-to-complete genomes from challenging tissues, such as wood and wood products. Results: In this paper, we report the success of an alignment-free genome comparison method, d 2; that differentiates different geographic sources of white oak (Quercus) species with a high level of accuracy with very small amount of genomic data. The method is robust to sequencing errors, different sequencing laboratories and sequencing platforms. Conclusions: This method offers an approach based on genome-scale data, rather than panels of pre-selected markers for specific taxa. The method provides a generalizable platform for the identification and sourcing of materials using a unified next generation sequencing and analysis framework.

59 BASIC BIOLOGICAL SCIENCES↗

RAVIS: Resource Forecast and Ramp Visualization for Situational Awareness - An Introduction to the Open-Source Tool and Use Cases

The Resource Forecast and Ramp Visualization for Situational Awareness (RAVIS) is an open-source tool for visualizing variable renewable resource forecasts and ramp alerts for significant up/down ramps in renewable resource and the consequent net-load. The modular dashboard of RAVIS contains configurable panes for viewing- probabilistic time series forecasts, ramp event alerts on the look-ahead timeline, spatially resolved resource sites and forecasts, and system simulation and market clearing data such as transmission lines utilization, nodal prices and available generation flexibility. RAVIS uses a technology suite that is assembled to provide optimum visualization facility while maintaining a wide pool of potential deployment and client environments. The tool is designed to take advantage of web application technologies, open source visualization libraries and tooling. Utilizing this technology will enable deployment in any environment, using any operating system, and is scalable to much higher spatial and temporal scales of visualization. As a prototype of the tool and demonstrating a use case of variable renewable integration, RAVIS currently integrates site-specific solar power forecasts in the California Independent System Operator (CAISO) and Mid-continent ISO (MISO) footprint from the IBM WattSun forecasting platform, and also superimposes market simulation data for CAISO footprint from as in-house NREL market clearing tool. The tool has the ability to alert the viewer for excessive up or down ramps for both individual solar sites as well as regionally aggregated net-load ramps, and alerts can also be qualified with respect to available flexible generation. This report will provide an introduction to the RAVIS tool, and summarize the above mentioned capabilities, typical use cases and possible extensions of the tool. The RAVIS development team believes there are likely to be high economic and reliability benefits of integrating probabilistic forecasts of variable renewables into control center visualizations and improved ramp events situational awareness for system operators and forecasting teams in the ISOs and electric utilities in comparison with their business as usual practices.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Dataset for "A primer on forest structure measurement with lidar for ecologists"

This repository includes data and code accompanying the case study included in the manuscript "A primer on forest structure measurement with lidar for ecologists" (submitted to Ecosphere). We compiled lidar datasets from multiple platforms in a common area to: 1. Demonstrate how differences in sensor characteristics influence density and resolution of lidar data. 2. Provide open-source, co-located datasets for users to further inspect differences in lidar data. 3. Provide example code to perform basic lidar analysis. This case study is meant to allow readers to get hands-on experience with real-world data from different platforms. This case study is not meant to be a rigorous comparison of derived ecological metrics among all sensors; such comparisons can be found throughout other publications referenced throughout the main manuscript. Code includes basic functions in R commonly used to visualize and manipulate lidar data accessible with a normal laptop computer; more sophisticated algorithms for advanced users are also referenced throughout the main manuscript. Terrestrial laser scanning (TLS), mobile laser scanning (MLS), UAS laser scanning (ULS), airborne laser scanning (ALS), and spaceborne laser scanning (SLS) data were collected within the Smithsonian Environmental Research Center (SERC) forest dynamics plot in Maryland, USA. TLS, MLS, and ALS data were collected within 1 month of the 2021 growing season; ULS data were collected in November 2020 (“leaf-off” data) and July 2022 (“leaf-on” data).

54 ENVIRONMENTAL SCIENCES↗

Novel Platform for Digitization of Radiation Measurements, Decommissioning and Dismantling and Nuclear Waste Management - 18229

An innovative Platform for a seamless digitization of the information produced during the characterization of radioactive waste, contaminated objects, and in general radioprotection measurements has ben developed by CAEN SyS. The Platform is based on a spectroscopic handheld instrument integrating RFID technology, barcode reader, camera, voice recorder, and geo-localization, a selection of radiation tolerant RFID tags tested above 300 Gy integrated dose, ruggedized portable RFID readers and a secure database framework capable to collect data also from external sources such as gamma scanners and imaging systems. The Platform allows for real-time measurement of dose rate, radiation spectroscopy, tagging, tracking and inventorying of items. All data produced by the Platform are securely stored into the non-volatile memory of the RFID tags and in the database for both onsite and remote access to the information by operators. This paper presents the current development of the Platform and shows the advantages of such approach to achieve an easy yet reliable waste traceability, waste bagging optimization, and a structured methodology for radiation measurements, meeting an excellent quality control as increasingly required by Safety Authorities. Applications here envisaged and explored are dismantling and decontamination operations, radioprotection measurements and patrols, nuclear material management, safeguards and verification inspections. More in general, the Platform can address the needs of all plants, research laboratories and facilities that are looking to improve their Safety Standards, decrease operational costs, have a precise knowledge of the contaminated environment and objects as well as optimization for personnel dose minimization. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

The need to scale next-generation industrial engineering problems to the largest computational platforms presents unique challenges. This paper focuses on data management related problems faced by the Uintah simulation framework at a production scale of 260K processes. Uintah provides a highly scalable asynchronous many-task runtime system, which in this work is used for the modeling of a 1000 megawatt electric (MWe) ultra-supercritical (USC) coal boiler. At 260K processes, we faced both parallel I/O and visualization related challenges, e.g., the default file-per-process I/O approach of Uintah did not scale on Mira. In this paper we present a simple to implement, restructuring based parallel I/O technique. We impose a restructuring step that alters the distribution of data among processes. The goal is to distribute the dataset such that each process holds a larger chunk of data, which is then written to a file independently. This approach finds a middle ground between two of the most common parallel I/O schemes–file per process I/O and shared file I/O–in terms of both the total number of generated files, and the extent of communication involved during the data aggregation phase. To address scalability issues when visualizing the simulation data, we developed a lightweight renderer using OSPRay, which allows scientists to visualize the data interactively at high quality and make production movies. Finally, this work presents a highly efficient and scalable radiation model based on the sweeping method, which significantly outperforms previous approaches in Uintah, like discrete ordinates. The integrated approach allowed the USC boiler problem to run on 260K CPU cores on Mira.

97 MATHEMATICS AND COMPUTING↗

Population-Based Incidence Rates of Diarrheal Disease Associated with Norovirus, Sapovirus, and Astrovirus in Kenya

Diarrheal diseases remain a major cause of mortality in Africa and worldwide. While the burden of rotavirus is well described, population-based rates of disease caused by norovirus, sapovirus, and astrovirus are lacking, particularly in developing countries. Data on diarrhea cases were collected through a population-based surveillance platform including healthcare encounters and household visits in Kenya. We analyzed data from June 2007 to October 2008 in Lwak, a rural site in western Kenya, and from October 2006 to February 2009 in Kibera, an urban slum. Stool specimens from diarrhea cases of all ages who visited study clinics were tested for norovirus, sapovirus, and astrovirus by RT-PCR. Of 334 stool specimens from Lwak and 524 from Kibera, 85 (25%) and 159 (30%) were positive for norovirus, 13 (4%) and 31 (6%) for sapovirus, and 28 (8%) and 18 (3%) for astrovirus, respectively. Among norovirus-positive specimens, genogroup II predominated in both sites, detected in 74 (87%) in Lwak and 140 (88%) in Kibera. The adjusted community incidence per 100,000 person-years was the highest for norovirus (Lwak: 9,635; Kibera: 4,116), followed by astrovirus (Lwak: 3,051; Kibera: 440) and sapovirus (Lwak: 1,445; Kibera: 879). For all viruses, the adjusted incidence was higher among children aged <5 years (norovirus: 22,225 in Lwak and 17,511 in Kibera; sapovirus: 5,556 in Lwak and 4,378 in Kibera; astrovirus: 11,113 in Lwak and 2,814 in Kibera) compared to cases aged ≥ 5 years. Although limited by a lack of controls, this is the first study to estimate the outpatient and community incidence rates of norovirus, sapovirus, and astrovirus across the age spectrum in Kenya, suggesting a substantial disease burden imposed by these viruses. By applying adjusted rates, we estimate approximately 2.8–3.3 million, 0.45–0.54 million, and 0.77–0.95 million people become ill with norovirus, sapovirus, and astrovirus, respectively, every year in Kenya.

60 APPLIED LIFE SCIENCES↗

Exploring MDSplus data-acquisition software and custom devices

MDSplus is a software tool designed for data acquisition, storage, and analysis of complex scientific experiments. Over the years, MDSplus has primarily been used for data management for fusion experiments. This paper demonstrates that MDSplus can be used for a much wider variety of systems and experiments. We present a step-by-step tutorial describing how to create a simple experiment, manage the data, and analyze it using MDSplus and Python. To this end, a custom example device was developed to be used as the data source. This device was built on an opensource electronic hardware platform, and it consists of a microcontroller and two sensors. We read data from these sensors, store it in MDSplus, and use JupyterLab to visualize and process it. This project and code demo are available on the GitHub site at this URL: https://github.com/santorofer/MDSplusAndCustomeDevices

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Novus: An Unmanned Aerial System and Virtual Reality Flight Deck for Data Collection and Visualization

Many emerging technologies offer tremendous new capabilities that will, without question, redefine how we interact with, utilize, understand, and optimize the energy systems around us. Drones offer a new ability to remotely capture data in ways that have previously been impractical. However, their flight control systems remain rudimentary and limit the ability of researchers to fully take advantage of these platforms as remote sensing devices. In this project, we constructed a hexarotor drone and outfitted it with advanced (yet inexpensive) sensors as well as a telemetry data management and control package. We then created a virtual reality flight deck that allows a drone operator to define waypoints "on-the-fly" or take manual control over flight operations. The flight deck provides a live on-board video stream display, a 3-dimensional topographic map with the drone’s position in space overlaid onto it, as well as a data visualization framework that keeps critical information immediately available to the operator. These advances provide a new platform for scientific data collection such as thermography of building envelopes or solar arrays. As the drone is in flight and data are being collected, these data are relayed to the virtual reality environment where they can be inspected for integrity and the flight plan altered if necessary to collect higher quality data. Future work will focus on implementing 3-dimensional mapping using both visible and thermal imagery collected from the drone, and streaming those into the virtual reality flight deck in real time.

Source record↗

Simultaneous electrical recording of cardiac electrophysiology and contraction on chip

Prevailing commercialized cardiac platforms for in vitro drug development utilize planar microelectrode arrays to map action potentials, or impedance sensing to record contraction in real time, but cannot record both functions on the same chip with high spatial resolution. We report a novel cardiac platform that can record cardiac tissue adhesion, electrophysiology, and contractility on the same chip. The platform integrates two independent yet interpenetrating sensor arrays: a microelectrode array for field potential readouts and an interdigitated electrode array for impedance readouts. Together, these arrays provide real-time, non-invasive data acquisition of both cardiac electrophysiology and contractility under physiological conditions and under drug stimuli. Furthermore, we cultured human induced pluripotent stem cell-derived cardiomyocytes as a model system, and used to validate the platform with an excitation–contraction decoupling chemical. Preliminary data using the platform to investigate the effect of the drug norepinephrine are combined with computational efforts. Finally, this platform provides a quantitative and predictive assay system that can potentially be used for comprehensive assessment of cardiac toxicity earlier in the drug discovery process.

59 BASIC BIOLOGICAL SCIENCES↗

Deep Learning on Operational Facility Data Related to Large-Scale Distributed Area Scientific Workflows

Abstract—Distributed computing platforms provide a robust mechanism to perform large-scale computations by splitting the task and data among multiple locations, possibly located thousands of miles apart geographically. This distribution of resources can lead to benefits such as redundancy of data, engagement with scientific teams that involve a diverse group of domain experts distributed spatially around the globe. It allows scientists to share their expertise and data with everyone and achieve insights by deploying collective intelligence of entire team. However, this distributed computing comes with its associated problems such as rampant duplication of file transfers increasing congestion, long job completion times, unexpected site crashing, suboptimal data transfer rates, unpredictable reliability in a time range, overshooting congestion, suboptimal usage of storage elements. In addition, each sub-system becomes a potential failure node that can trigger system wide disruptions.

Machine Learning, Provenance, Time Series, Semanti↗

Wildlife Densities and Habitat Use Across Temporal and Spatial Scales on the Mid-Atlantic Outer Continental Shelf. Final Report to the Department of Energy EERE Wind & Water Power Technologies Office

The Mid-Atlantic Baseline Studies Project helped address environmental barriers to offshore wind energy development in the mid-Atlantic region by providing regulators, developers, and other stakeholders with comprehensive baseline ecological data and analyses. Project funders and collaborators from a range of academic institutions, non-governmental organizations, federal agencies, foundations, and private companies came together to study bird, sea turtle, and marine mammal distributions, densities, and movements on the mid-Atlantic Outer Continental Shelf between 2012 and 2014. Specific project activities and goals included the following: (1) Conduct standardized surveys to quantify bird, sea turtle, and marine mammal densities seasonally and annually throughout the study region and identify important habitat use or aggregation areas. (2) Develop statistical models to help understand the drivers of wildlife distribution and abundance patterns. (3) Use individual tracking data for several focal bird species to provide information on population connectivity and individual movements that is complementary to survey data. (4) Identify species that are likely to be exposed to offshore wind energy development activities in the mid-Atlantic study area. (5) Develop U.S.-based technological resources and assessment methods for future monitoring efforts, including a comparison of high resolution digital video aerial surveys to boat-based surveys. (6) Help meet data needs associated with National Environmental Policy Act (NEPA), Marine Mammal Protection Act, and Endangered Species Act requirements, by contributing several years of data and analysis towards future Environmental Impact Statements. This report consists of six parts: Project overview (executive summary and Chapters 1-2); Examining wildlife distributions and relative abundance from a digital video aerial survey platform (Chapters 3-6); Examining wildlife distributions and abundance using boat-based surveys (Chapters 7-12); Integrating data across survey platforms (Chapters 13-19); Individual movements and habitat use for focal bird species (Chapters 20-25); and Nocturnal avian migration monitoring (Chapters 26-27). Boat-based and digital video aerial surveys each had specific advantages and disadvantages, but were largely complementary (Chapters 1, 5, 13-14). Digital aerial surveys were particularly useful for covering offshore areas at broad scales, where general distributions of taxonomic groups were a priority; boat surveys could provide more detailed data on species identities and behaviors, but were more limited in geographic scope due to their slower survey pace. The mid-Atlantic study area was important for wintering and breeding taxa, and its location also made it a key migratory corridor. There was considerable variation in species composition and spatial patterns by season, largely driven by dynamic environmental conditions (Chapters 12, 15, and 20-22). Habitat gradients in nearshore waters, however, were reliable influences on productivity and patterns of species distributions and abundance. Areas within about 30-40 km of the coast offshore of the mouths of Chesapeake and Delaware Bays, as well as to the south of Delaware Bay along the coast, were consistent hotspots of abundance and species diversity, regardless of survey methodology or analytical approach (Chapters 2, 12, 17). Inter-annual variation was substantial, and the importance of certain environmental variables in predicting animal distributions indicates that these species may well respond to future environmental shifts brought about by anthropogenic effects and climatic change. This study is an important first step, however, towards understanding how bird, marine mammal, and sea turtle populations in the mid-Atlantic may be exposed to offshore wind energy development and other anthropogenic activities. The results of this study provide insight to help address environmental permitting requirements for current and future offshore development projects, and serve as a starting point for more site-specific studies, risk analyses, and evaluation of potential measures to avoid and minimize those risks.

17 WIND ENERGY↗

PMU Data Quality and Sensor Health Monitoring

Phasor Measurement Units (PMUs) play a critical role in the evolution of the electric power industry by providing high-precision, real-time monitoring of essential power system metrics. However, effectively detecting abnormalities and critical events from PMU data is a complex task, complicated by intricate temporal patterns, a scarcity of labeled data for training algo- rithms, and constraints on online computational power. In this study, we apply TranAD, an innovative algorithm that combines transformer architectures with the refinement of adversarial learning, to both synthetic and real-world PMU datasets for developing a data quality and sensor online health monitoring platform for utilities. Our findings reveal that TranAD not only provides efficient detection and localization but also enhances the detail with which abnormalities are detected, marking a a significant step forward in the field of clean data acquisition processes for power system monitoring

deep neural network, machine learning (ML)↗

Investigation of cyclic disturbances in granular medium

The author studies stability of a structure (platform) on a pile foundation using physical modeling. Disturbing force is applied by a pendulum to generate seismic impulse vibrations. The measurement data on displacements of two points on the platform surface are used to find a platform incline under variable number of impacts, impact energy, distance to the impact point and the platform weight.

42 ENGINEERING↗

Bifacial PV Module Energy Modeling Validation Study

The cost delta between monofacial and bifacial photovoltaic (PV) modules was decreasing in 2018 when this study was initially proposed, making bifacial modules an attractive offering. However, there were limited field studies available that could be considered utility-scale representative, and PV module manufacturers were stating large ranges for the expected energy yield improvements of bifacial technology. This generated uncertainties regarding bifacial performance gains and the actual LCOE (levelized cost of energy). There was also relatively low confidence in the industry’s energy modeling tools’ ability to accurately predict bifacial energy yield, meaning that early adopters of bifacial modules were not able to fully account for the increased energy yield in their energy and financial models. The intent of this project was to provide the solar industry with greater certainty on the energy yield gains of bifacial modules and provide higher confidence in the ability for different energy modeling software to model bifacial PV site performance. Achieving this would allow for bifacial technology to become bankable (i.e. accepted by Independent Engineers, site financiers and other PV site stakeholders), offering a step change in PV site performance that had not been seen since the widespread adoption of the single-axis tracker. Over the course of this study other economic factors including a significant tariff exemption for bifacial modules resulted in accelerated adoption of bifacial technology throughout the US utility scale solar segment. The number of early adopters increased rapidly, and with many investors accepting this module choice the question of bifacial bankability was seemingly answered faster than expected. However, PVEL’s study has still achieved significant accomplishments in demonstrating the accuracy of bifacial modeling across three different software platforms. The results of this study have shown that the mean biased error (MBE) between the field data and the predicted values from all three software platforms were aligned with a maximum MBE of 1.3% and a minimum MBE of -1.8%.

14 SOLAR ENERGY↗

Analysis & Tools to Spur Increased Deployment of “Waste Heat” Rejection/Recycling Hybrid Ground-source Heat Pump Systems in Hot, Arid or Semiarid Climates Like Texas

This project team analyzed supplemental heat rejection/recovery (SHR) devices or systems that could be used in hybrid ground source heat pump (HGHP) systems located in arid or semi-arid regions in southwestern U.S. Identification of effective SHR solutions would enhance the deployment of ground source heat pumps (GHP) in these regions. In a parallel effort, the team developed integrated GHP models that coupled the building load, heat pump, and ground loop subsystems and which could be applied to residential and commercial office buildings. Then GHP and HGHP performances could be compared in terms of operational performance and life-cycle costs. Several potential SHR devices were analyzed by applying two strategies: 1) to remove heat directly from the water in the ground loop before it enters the ground and 2) to remove heat in the refrigerant loop of the vapor compression cycle (VCC) of the heat pump so less heat is transferred to the water loop at the condenser of the VCC. Cooling towers, adsorption coolers, and thermoelectric liquid coolers were included in strategy 1, and expanded desuperheaters, thermosyphons, and an optimized VCC were included in strategy 2. Of all SHR devices analyzed, only the cooling tower provided a cost-effective performance enhancement. For the integrated GHP model, the project team selected the building load model HAMBASE and its powerful computational Simulink/MatLab platform, empirical performance map models of the heat pumps based upon manufacturers’ performance data, and a ground loop model developed by Oklahoma State University and rewritten for this project in Simulink/MatLab. The design process used GLHEPRO, also from Oklahoma State University, to size the borehole fields. The building load and ground loop models were compared with simulations from eQuest, ASHRAE 140-2008 standards, EnergyPlus, and GLHEPRO and were found to predict those subsystems’ performance well. The integrated GHP model was applied to a 195m 2 (2100ft 2 ) residential building and a 4,982m 2 (53,628ft 2 ) three-story commercial office building, and it ran 10-15 year simulations. The integrated GHP model and its Simulink platform provided residential data, ranging from seconds to years, and commercial office building data, ranging from minutes to years. A cooling tower model was coupled to the base case integrated GHP model for the residential building and the resulting HGHP system provided a cost-effective solution for the Austin, TX location. Simulations for both the residential and commercial building models were run with varying degrees of SHR (device/system not identified) and the results were found to significantly decrease installation costs, increase heat pump efficiency (lower entering water temperature), and prolong the lifetime of the borehole field. Lifetime cycle costs were estimated from the simulation results. Sensitivity studies on system operating performance and lifetime costs were performed on design parameters, such as construction materials, borehole length, borehole configuration and spacing, grout conductivity, and effects of SHR. While some of the results are intuitive, these studies provided quantitative estimates of improved performance and cost. One of the most important results of this sensitivity study is that overall system performance is very sensitive to these design parameters and that modeling and simulation are essential tools to design cost-effective systems.

15 GEOTHERMAL ENERGY↗

A Collaborative Extensible User Environment for Simulation and Knowledge Management

In scientific simulation, scientists use measured data to create numerical models, execute simulations and analyze results from advanced simulators executing on high performance computing platforms. This process usually requires a team of scientists collaborating on data collection, model creation and analysis, and on authorship of publications and data. This paper shows that scientific teams can benefit from a user environment called Akuna that permits subsurface scientists in disparate locations to collaborate on numerical modeling and analysis projects. The Akuna user environment is built on the Velo framework that provides both a rich client environment for conducting and analyzing simulations and a Web environment for data sharing and annotation. Akuna is an extensible toolset that integrates with Velo, and is designed to support any type of simulator. This is achieved through data-driven user interface generation, use of a customizable knowledge management platform, and an extensible framework for simulation execution, monitoring and analysis. This paper describes how the customized Velo content management system and the Akuna toolset are used to integrate and enhance an effective collaborative research and application environment. The extensible architecture of Akuna is also described and demonstrates its usage for creation and execution of a 3D subsurface simulation.

collaborative science tools↗

BETTER Together

The Standard Energy Efficiency Data (SEED) and Building Efficiency Targeting Tool for Energy Retrofits (BETTER) platforms are both developed by the Department of Energy and work better together. SEED is a database to manage building characteristics and performance data from a variety of sources. BETTER provides simple energy efficiency measure analyses based on high level data about the building or portfolio of buildings. A demonstration of each platform and their integration will be provided. The inputs for BETTER are building type, floor area, location, utility data, and whether PV shall be included in the analysis. The BETTER analysis can be manually set up through the web application or data can be uploaded with a BuildingSync XML file either directly or through the API. SEED can be the source of this data and the data can be sent to BETTER through the SEED application after the BETTER API token has been entered. The benefit of utilizing SEED is that it has connections to many other sources of data such as ENERGY STAR Portfolio Manager, Audit Template, and Salesforce. Therefore, it is likely that a user of SEED will already have the required inputs for BETTER in SEED already and can create BETTER analyses across their whole portfolio in a couple mouse clicks. This is a major time savings and enables decision makers an easy path to identify buildings that should undergo more detailed audits or retrofit pathways.

ASHRAE↗

Characterizing Impacts of Storage Faults on HPC Applications: A methodology and insights

In recent years, the increasing complexity in scientific simulations and emerging demands for training heavy artificial intelligence models require massive and fast data accesses, which urges high-performance computing (HPC) platforms to equip with more advanced storage infrastructures such as solid-state disks (SSDs). While SSDs offer high-performance I/O, it remains unclear about the reliability challenges faced by the HPC applications under the SSD-related failures, in particular, failures resulting in data corruptions. The goal of this paper is to understand the impact of SSD-related data corruptions on the behaviors of complex HPC applications. To this end, we propose FFIS, a FUSE-based fault injection framework that systematically introduces storage faults into the application layer to model the errors originated from SSDs. FFIS is able to plant different I/O related faults into the data returned from underlying file systems, which also enables the investigation on the error resilience characteristics of the scientific file format for the first time. We demonstrate the use of FFIS with three representative real HPC applications, show how each application reacts to the data corruptions, and provide insights on the error resilience of the widely-adopted HDF5 file format for the HPC applications.

Fang, Bo↗