Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Fault Injection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Towards Precision-Aware Fault Tolerance Approaches for Mixed-Precision Applications

Graphics Processing Units (GPUs), the dominantly adopted accelerators in HPC systems, are susceptible to transient hardware fault. New generation of GPUs feature mixed-precision architectures such as NVIDIA Tensor Cores to accelerate matrix multiplications. While being widely adapted, how would they behave under transient hardware faults remain unclear. In this study, we conduct a large-scale fault injection experiments on GEMM kernels implemented with different floating-point data types on the V100 and A100 Tensor Cores, and show distinct error resilience characteristics for the GEMMS with different formats. In the future, we plan to explore this space by building precision-aware floating-point fault tolerance techniques for applications such as DNNs that exercise low-precision computations.

Fang, Bo↗

Evaluation of “Autotune” calibration against manual calibration of building energy models

Our paper demonstrates the application of Autotune, a methodology aimed at automatically producing calibrated building energy models using measured data, in two case studies. In the first case, a building model is de-tuned by deliberately injecting faults into more than 60 parameters. This model was then calibrated using Autotune and its accuracy with respect to the original model was evaluated in terms of the industry-standard normalized mean bias error and coefficient of variation of root mean squared error metrics set forth in ASHRAE Guideline 14. In addition to whole-building energy consumption, outputs including lighting, plug load profiles, HVAC energy consumption, zone temperatures, and other variables were analyzed. In the second case, Autotune calibration is compared directly to experts’ manual calibration of an emulated-occupancy, full-size residential building with comparable calibration results in much less time. Lastly, our paper concludes with a discussion of the key strengths and weaknesses of auto-calibration approaches.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

On-Orbit Results for the Xilinx Virtex-4 FPGA

This paper provides information regarding the use of the Xilinx Virtex-4 field-programmable gate array in a spacecraft deployed to low earth orbit. The results are compared to pre-deployment accelerated and fault injection testing.

42 ENGINEERING↗

HAPPA: A Modular Platform for HPC Application Resilience Analysis with LLMs Embedded

High-performance computing (HPC) systems are increasingly vulnerable to soft errors, which pose significant challenges in maintaining computational accuracy and reliability. Predicting the resilience of HPC applications to these errors is crucial for robust code protection and detailed resilience analysis. In this study, we present HAppA, a modular platform designed for HPC Application Resilience Analysis. Embedding Large Language Models (LLMs), HAppA addresses understanding the context information of long code sequences typical in HPC applications. HAppA implements a novel code representation module that chunks the code into fixed-size segments and aggregates the embeddings of these segments. Three aggregation methods have been explored: MeanPooling, MaxPooling, and LSTM-based techniques. We built a DAtaset for REsilience analysis using Fault Injection (FI), named DARE. Using our DARE dataset, HAppA is trained for regression prediction tasks. Our evaluation results demonstrate the predictive accuracy of HAppA compared to other models, particularly noting that the LSTM-based aggregation method -- HAppA-LSTM -- achieves a mean squared error (MSE) of 0.078 for SDC prediction, surpassing the existing state-of-the-art PARIS model, which recorded an MSE of 0.1172. Additionally, HAppA with the KeyBERT model extracts a list of keywords representing the source code. A comprehensive importance analysis of these keywords further elucidates the code patterns contributing to the error rate. These findings highlight the effectiveness of HAppA in analyzing the resilience of HPC applications and establish a new benchmark for predictive accuracy in resilience.

Jiang, Hailong [Kent State University]↗

Diagnosis of Performance Faults in LargeScale MPI Applications via Probabilistic Progress-Dependence Inference

Debugging large-scale parallel applications is challenging. Most existing techniques provide little information about failure root causes. Further, most debuggers significantly slow down program execution, and run sluggishly with massively parallel applications. This paper presents a novel technique that scalably infers the tasks in a parallel program on which a failure occurred, as well as the code in which it originated. Our technique combines scalable runtime analysis with static analysis to determine the least-progressed task(s) and to identify the code lines at which the failure arose. We present a novel algorithm that infers probabilistically progress dependence among MPI tasks using a globally constructed Markov model that represents tasks' control-flow behavior. In comparison to previous work, our algorithm infers more precisely the least-progressed task. Further, we combine this technique with static backward slicing analysis, further isolating the code responsible for the current state. A blind study demonstrates that our technique isolates the root cause of a concurrency bug in a molecular dynamics simulation, which only manifests itself at 7,996 tasks or more. We extensively evaluate fault coverage of our technique via fault injections in 10 HPC benchmarks and show that our analysis takes less than a few seconds on thousands of parallel tasks.

97 MATHEMATICS AND COMPUTING↗

Reconfigurable Framework for Resilient Semantic Segmentation for Space Applications

Deep learning (DL) presents new opportunities for enabling spacecraft autonomy, onboard analysis, and intelligent applications for space missions. However, DL applications are computationally intensive and often infeasible to deploy on radiation-hardened (rad-hard) processors, which traditionally harness a fraction of the computational capability of their commercial-off-the-shelf counterparts. Commercial FPGAs and system-on-chips present numerous architectural advantages and provide the computation capabilities to enable onboard DL applications; however, these devices are highly susceptible to radiation-induced single-event effects (SEEs) that can degrade the dependability of DL applications. In this article, we propose Reconfigurable ConvNet (RECON), a reconfigurable acceleration framework for dependable, high-performance semantic segmentation for space applications. In RECON, we propose both selective and adaptive approaches to enable efficient SEE mitigation. In our selective approach, control-flow parts are selectively protected by triple-modular redundancy to minimize SEE-induced hangs, and in our adaptive approach, partial reconfiguration is used to adapt the mitigation of dataflow parts in response to a dynamic radiation environment. Combined, both approaches enable RECON to maximize system performability subject to mission availability constraints. We perform fault injection and neutron irradiation to observe the susceptibility of RECON and use dependability modeling to evaluate RECON in various orbital case studies to demonstrate a 1.5–3.0× performability improvement in both performance and energy efficiency compared to static approaches.

97 MATHEMATICS AND COMPUTING↗

A Fault-Oblivious Extreme-Scale Execution Environment (FOX)

IBM Research’s contribution to the Fault Oblivious Extreme-scale Execution Environment (FOX) revolved around three core research deliverables: • collaboration with Boston University around the Kittyhawk cloud infrastructure which both enabled a development and deployment platform for the project team and provided a fault-injection testbed to evaluate prototypes • operating systems research focused on exploring role-based operating system technologies through collaboration with Sandia National Labs on the NIX research operating system and collaboration with the broader IBM Research community around a hybrid operating system model which became known as FusedOS • IBM Research also participated in an advisory capacity with the Boston University SESA project, the core of which was derived from the K42 operating system research project funded in part by DARPA’s HPCS program. Both of these contributions were built on a foundation of previous operating systems research funding by the Department of Energy’s FastOS Program. Through the course of the X-stack funding we were able to develop prototypes, deploy them on production clusters at scale, and make them available to other researchers. As newer hardware, in the form of BlueGene/Q, came online, we were able to port the prototypes to the new hardware and release the source code for the resulting prototypes as open source to the community. In addition to the open source coded for the Kittyhawk and NIX prototypes, we were able to bring the BlueGene/Q Linux patches up to a more recent kernel and contribute them for inclusion by the broader Linux community. The lasting impact of the IBM Research work on FOX can be seen in its effect on the shift of IBM’s approach to HPC operating systems from Linux and Compute Node Kernels to role-based approaches as prototyped by the NIX and FusedOS work. This impact can be seen beyond IBM in follow-on ideas being incorporated into the proposals for the Exasacale Operating Systems/Runtime program.

97 MATHEMATICS AND COMPUTING↗

Enabling Exascale Hardware and Software Design through Scalable System Virtualization (Final Report)

In this grant, we enhanced the Palacios virtual machine monitor to increase its scalability and suitability for addressing exascale system software design issues. This included a wide range of research on core Palacios features, large-scale system emulation, fault injection, perfomrance monitoring, and VMM extensibility. This research resulted in large number of high-impact publications in well-known venues, the support of a number of students, and the graduation of two Ph.D. students and one M.S. student. In addition, our enhanced version of the Palacios virtual machine monitor has been adopted as a core element of the Hobbes operating system under active DOE-funded research and development.

97 MATHEMATICS AND COMPUTING↗

Evaluating Application Resilience with XRay

The rising count and shrinking feature size of transistors within modern computers is making them increasingly vulnerable to various types of soft faults. This problem is especially acute in high-performance computing (HPC) systems used for scientific computing, because these systems include many thousands of compute cores and nodes, all of which may be utilized in a single large-scale run. The increasing vulnerability of HPC applications to errors induced by soft faults is motivating extensive work on techniques to make these applications more resilient to such faults, ranging from generic techniques such as replication or checkpoint/restart to algorithm-specific error detection and tolerance techniques. Effective use of such techniques requires a detailed understanding of how a given application is affected by soft faults to ensure that (i) efforts to improve application resilience are spent in the code regions most vulnerable to faults and (ii) the appropriate resilience technique is applied to each code region. This paper presents XRay, a tool to view the application vulnerability to soft errors, and illustrates how XRay can be used in the context of a representative application. In addition to providing actionable insights into application behavior XRay automatically selects the number of fault injection experiments required to provide an informative view of application behavior, ensuring that the information is statistically well-grounded without performing unnecessary experiments.

97 MATHEMATICS AND COMPUTING↗

Virtualized hardware environments for supporting digital I and C verification - 324

The technology of virtualized hardware testing platforms within the nuclear context in order to provide a framework for evaluating virtualized hardware/software approaches is discussed. To understand the applicability of virtualized hardware approaches, we have developed a virtual hardware model of an actual smart sensor using the Imperas product OVPsim development environment, as is seen in . An evaluation of the modeling effort required to create the digital I and C smart sensor is provided, followed by an analysis and findings on the various tools in the OVPsim environment for supporting testing, evidence collecting, and analysis. The analyzed tools include traditional input/output testing, model-based testing, mutation testing, and fault injection techniques. Finally, the goal of this research effort is to analyze virtual hardware platforms and present findings to the nuclear community on the efficacy and viability of this technology as a complementary means to actual hardware-in-the-loop testing Factory Acceptance Testing for Nuclear Power Plant (NPP) digital I and C systems. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Reusable Verification Components for High-Energy Physics readout ASICs

Verification is a critical aspect of designing front-end (FE) readout ASICs for High-Energy Physics (HEP) experiments. These ASICs share several similar functional features, resulting in similar verification objectives, which can be addressed using comparable verification strategies. This contribution presents a set of re-usable verification components for addressing common verification tasks, such as clock generation, reset handling, configuration, as well as hit and fault injections. The components were developed as part of the CHIPS initiative and they have been successfully used in the verification of multiple HEP ASICs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Stabilization of fault slip by fluid injection in the laboratory and in situ

Faults can slip seismically or aseismically depending on their hydromechanical properties, which can be measured in the laboratory. Here, we demonstrate that fault slip induced by fluid injection in a natural fault at the decametric scale is quantitatively consistent with fault slip and frictional properties measured in the laboratory. The increase in fluid pressure first induces accelerating aseismic creep and fault opening. As the fluid pressure increases further, friction becomes mainly rate strengthening, favoring aseismic slip. Our study reveals how coupling between fault slip and fluid flow promotes stable fault creep during fluid injection. Seismicity is most probably triggered indirectly by the fluid injection due to loading of nonpressurized fault patches by aseismic creep.

58 GEOSCIENCES↗

Seismicity Rate Surge on Faults after Shut–in: Poroelastic Response to Fluid Injection

Injection of large amounts of fluid into the subsurface alters the states of pore pressure and stress in the formation, potentially inducing earthquakes. Increase in the seismicity rate after shut–in is often observed at fluid–injection operation sites, but mechanistic study of the rate surge has not been investigated thoroughly. Considering full poroelastic coupling of pore pressure and stress, the earthquake occurrence after shut–in can be driven by two mechanisms: (1) post shut–in diffusion of pore pressure into distant faults and (2) poroelastic stressing caused by fluid injection. Interactions of these mechanisms can depend on fault geometry, hydraulic and mechanical properties of the formation, and injection operation. In this work, a 2D aerial view of the target reservoir intersected by strike–slip basement faults is used to evaluate the impact of injection–induced pressure buildup on seismicity rate surge. A series of sensitivity tests are performed by considering the variation in (1) permeability of the fault zone, (2) locations and the number of faults with respect to the injector, and (3) well operations with time–dependent injection rates. Lower permeability faults have higher seismicity rates than more permeable faults after shut–in due to delayed diffusion and poroelastic stressing. Hydraulic barriers, depending on their relative location to injection, can either stabilize or weaken a conductive fault via poroelastic stresses. In conclusion, gradual reduction of the injection rate minimizes the coulomb stress change and the least seismicity rates are predicted due to slower relaxation of coupling–induced compression as well as pore–pressure dissipation.

58 GEOSCIENCES↗

Factors controlling injection-induced rupture of intersecting faults during geological sequestration of CO 2

This study addresses coupled multiphase fluid flow and geomechanics effects on potential fault activation associated with subsurface CO 2 injection around intersecting faults. An enhanced fault-representation model is used to capture geomechanical responses of two intersecting faults with finite length during CO 2 injection. The faults are embedded in a strike-slip stress regime of a caprock-reservoir-basement system with the faults represented by zero-thickness interfaces with adjacent finite-thickness damage zones. A sensitivity analysis is conducted to study the effect of fault permeability, slip-weakening behavior, well location relative to the orientation of faults, and well placement (the number and location of injection wells). Five metrics (pressure, CO 2 plume, shear state on the fault, as well as shear displacement and stress path at selected fault monitoring points) are selected to assess CO 2 migration and reactivation of intersecting faults. The results show that induced ruptures are favored by low permeability faults due to high pressure buildup and by slip-weakening behavior resulting from fault strength reduction. The location of one injection well relative to fault orientation determines the magnitude of changes in effective normal stress and shear stress, affecting the location of induced ruptures. Well placement (two injection wells used in the paper) dominates pressure diffusion around the intersection and tips of faults. This redistributes changes in effective normal stress caused by each injection well, influencing the spatial distribution of ruptures along faults. A larger injection volume induces far-field ruptures that are controlled by stress transfer within the injection layer. The findings presented here can provide valuable insights into engineering operations for a long-term, safe, and reliable geologic CO 2 storage.

Fault permeability↗

Seismic velocity changes associated with aseismic deformations of a fault stimulated by fluid injection

Fluid pressure plays an important role in the stability of tectonic faults. However, the in situ mechanical response of faults to fluid pressure variations is still poorly known. To address this question, we performed here a fluid injection experiment in a fault zone in shales while monitoring fault movements at the injection source and seismic velocity variations from a near-distance (<10 m) monitoring network. We measured and located the P and S wave velocity perturbations in and around the fault using repetitive active sources. We observed that seismic velocity perturbations dramatically increase above 1.5 MPa of injection pressure. This is consistent with an increase of fluid flow associated with an aseismic dilatant shearing of the fault as shown by numerical modeling. We find that seismic velocity changes are sensitive to both fault opening by fluid invasion and effective stress variations and can be an efficient measurement for monitoring fluid-driven aseismic deformations of faults.

58 GEOSCIENCES↗

Pressure Monitoring to Detect Fault Rupture Due to CO 2 Injection

The capacity for fault systems to be reactivated by fluid injection is well-known. In the context of CO 2 sequestration, however, the consequence of reactivated faults with respect to leakage and monitoring is poorly understood. Using multi-phase fluid flow simulations, this study addresses key questions concerning the likelihood of ruptures, the timing of consequent upward leakage of CO 2 , and the effectiveness of pressure monitoring in the reservoir and overlying zones for rupture detection. A range of injection scenarios was simulated using random sampling of uncertain parameters. These include the assumed distance between the injector and the vulnerable fault zone, the critical overpressure required for the fault to rupture, reservoir permeability, and the CO 2 injection rate. We assumed a conservative scenario, in which if at any time during the five-year simulations the critical fault overpressure is exceeded, the fault permeability is assumed to instantaneously increase. For the purposes of conservatism we assume that CO 2 injection continues ‘blindly’ after fault rupture. We show that, despite this assumption, in most cases the CO 2 plume does not reach the base of the ruptured fault after 5 years. As a result, one possible implication of this result is that leak mitigation strategies such as pressure management have a reasonable chance of preventing a CO 2 leak.

58 GEOSCIENCES↗

Basement Fault Reactivation by Fluid Injection Into Sedimentary Reservoirs: Poroelastic Effects

Abstract To investigate mechanisms of seismic fault reactivation in crystalline basement in response to fluid injection in overlying sedimentary reservoirs, we conducted three‐dimensional finite element simulations to assess the effects of direct pore pressure communication and indirect poroelastic stress transfer on the change in Coulomb failure stress of favorably oriented faults of varying permeability structure in normal, strike‐slip, and reverse faulting stress regimes. We demonstrate that the direct pore pressure effect transmitted along a hydraulically conductive fault exceeds the indirect poroelastic effect, but alone is insufficient for fault reactivation in the basement. The poroelastic effect on the Coulomb failure stress results from induced normal tractions and, to a lesser extent, from induced shear tractions that relate to the flexing of the fault as the reservoir expands poroelastically with fluid injection. Assuming a higher Biot coefficient for reservoir over basement rock as previously reported, the combined direct pore pressure and indirect poroelastic effects result in reactivation of hydraulically conductive faults in the basement in normal and strike‐slip faulting stress regimes and in the reservoir in reverse faulting regimes. Sealing normal faults that are not preferentially conductive also preferentially reactivate in the reservoir. These findings apply to injection in either hanging or footwall of normal and reverse faults. Reducing the contrast in Biot coefficient between reservoir and basement favors fault reactivation in the reservoir for injection in the footwall in normal faulting stress regimes. These simulations demonstrate that geomechanical models without coupled poroelasticity underestimate the potential of fault reactivation in crystalline basement.

Fan, Zhiqiang↗

Utah FORGE: Fault Reactivation Through Fluid Injection Induced Seismicity Laboratory Experiments

Included are results from shear reactivation experiments on laboratory faults pre-loaded close to failure and reactivated by the injection of fluid into the fault. The sample comprises a single-inclined-fracture (SIF) transecting a cylindrical sample of Westerly granite. All experiments are conducted at ambient temperature and follow a similar protocol: (i) application of confining stresses (3MPa) on the fault fully saturated with DI water, (ii) shear-mobilization through the increase of axial loading at a constant displacement rate until a post-peak steady-state condition is reached, (iii) reduction of axial loading and related shear stress to a prescribed fraction of the peak steady-state frictional strength (typically 60% to 90%, representing intermediate to high magnitudes) and (iv), fault reactivation triggered by a stepwise increase of pore pressure on the fault in 0.1 MPa increments held constant for 1-5 minutes. Mechanical data from three ISCO pumps connected to a Temco pressure vessel measure axial, confining, and fault-related parameters, including fluid pressure (kPa), fluid flow rate (mL/min), and axial displacement (mm). See included code for initial data analysis and visualization for select experiments. Resource names represent experiment numbers found in the "Read Me" file, which describes each experimental setup and parameters.

15 GEOTHERMAL ENERGY↗