Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “interconnected network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

FORCE Regression Testing

Via programs including the Light Water Reactor Sustainability and Integrated Energy Systems, the U.S. Department of Energy has invested in the Framework for Optimization of ResourCes and Economics (FORCE) software framework (Idaho National Laboratory 2024a) for the technical and economic analysis of nuclear-integrated energy systems (IES). Nuclear IES expand the use of nuclear from traditional baseload electricity generation to a flexible and adaptive source of combined heat and power. Nuclear heat can be used in the production of a variety of energy currencies such as hydrogen and ammonia as well as other heat applications including water desalination and district heating. FORCE is designed with the intent to provide interconnected analysis tools that enable the accurate technical and economic assessment of specific nuclear IES configurations for individual energy markets. FORCE consists of three main analysis pathways: HYBRID (Idaho National Laboratory 2024b), which contains high-resolution physical models for IES; Holistic Energy Resource Optimization Network (HERON) (Idaho National Laboratory 2024c), which analyzes IES long-term economic viability; and Optimization of Real-time Capacity Allocation (ORCA) (Idaho National Laboratory 2024d), designed for real-time control of IES via digital twins and optimal decision making, including autonomous and remote operation research. Development of the FORCE ecosystem is guided by three pillars: capability, which assures that the computational requirements of IES analysis are met by the software tools; reliability, which provides for consistent code performance and expected behaviors; and accessibility, which lowers the barrier to entry for using the software and accelerates analysis by users beyond the FORCE primary developers. Reliability of the FORCE ecosystem is established according to the American Nuclear Society?s Nuclear Quality Assurance (NQA-1) program [American Society of Mechanical Engineers 1982], with specific levels of software quality assurance (SQA) within NQA-1 applied to each software tool in FORCE. As the tools within FORCE have matured, some integration algorithms to accurately connect the software tools for holistic analysis have been developed and deployed within the FORCE software repository. In accordance with NQA-1 standards, regression tests are required to guarantee the software performs consistently even when new capabilities are added to the software. In this report, we document the deployment of both unit tests, which test the consistent behavior of small pieces of the FORCE code base, as well as integration tests, which test the consistent performance of full use cases for the FORCE integration algorithms. We further document the encapsulation of these tests within a test harness, which collectively checks for each successful test completion on demand. Finally, we document the automation of the test harness using GitHub Actions [GitHub 2024], which require all tests succeed before any new capability or other changes can be added to the FORCE integration software

97 MATHEMATICS AND COMPUTING↗

Models and methods for assessing the value of HVDC and MVDC technologies in modern power grids

This report reflects the results of U.S. Department of Energy’s (DOE) Grid Modernization project 0074 “Models and methods for assessing the value of HVDC [high-voltage direct current] and MTDC [multi-terminal direct current] technologies in modern power grids.” The work was done by the Pacific Northwest National Laboratory (PNNL) and Oak Ridge National Laboratory (ORNL) in cooperation with Mid-Continent Independent System Operator (MISO) and Siemens. The main motivation of this study was to show the benefit of using direct current (DC) systems larger than those in existence today as they overlap with the alternating current (AC) systems. Proper use of their flexibility in terms of active/reactive power control and fast response can provide much-needed services to the grid at the same time as moving large blocks of energy to take advantage of cost diversity. Ultimately, the project’s success will enable decision-makers and investors to make well-informed decisions regarding this use of DC systems. This project showed the technical feasibility of HVDC macrogrid for frequency control and congestion relief in addition to bulk power transfers. Industry-established models for commonly used technologies were employed, along with high-fidelity models for recently developed HVDC converter technologies; like the modular multilevel converters (MMCs), a voltage source converters (VSC). Models for General Electric Positive Sequence Load Flow (GE PSLF) and Siemens Power System Simulator (PSS/E), widely used analysis programs, were for the first time adapted to include at the same time both Western Electricity Coordinating Council (WECC) and Eastern Interconnection (EI), the two largest North American interconnections. The high-fidelity models and their control were developed in detail for MMC system and extended to HVDC systems in point-to-point and in three-node multi-terminal configurations. Using a continental-level mixed AC-DC grid model, and using a HVDC macrogrid power flow and transient stability model, the results showed that the HVDC macrogrid relieved congestion and mitigated loop flows in AC networks, and provided up to 24% improvement in frequency responses. These are realistic studies, based on the 2025 heavy summer and EI multi-regional modeling working group (MMWG) 2026 summer peak cases. This work developed high-fidelity models and simulation algorithms to understand the dynamics of MMC. The developed models and simulation algorithms are up to 25 times faster than the existing algorithms. Models and control algorithms for high-fidelity models were designed and tested for point-to-point and multi-terminal configurations. The multi-terminal configuration was tested connecting simplified models of EI, WI, and Electric Reliability Council of Texas (ERCOT). The developed models showed up to 45% improvement in frequency response with the connection of all the three asynchronous interconnections in the United States using fast and advanced DC technologies like the multi-terminal MMC-DC system. Future work will look into developing high-fidelity models of other advanced DC technologies, combining high-fidelity models with the continental-level model, incorporating additional services. More scenarios involving large-scale HVDC and MTDC will be evaluated.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Secure, Scalable Control and Communications for Distributed PV

An increasing number of jurisdictions are adopting Distributed Energy Resource (DER) interconnection standards which require photovoltaic (PV) inverters, energy storage systems, and other DER to include interoperable grid-support functionality. These functions provide grid operators the nobs to support local and bulk power system operations with DER equipment, but the associated grid operator-to-DER communications networks must be deployed with appropriate cybersecurity features. In some situations, additional security features may prevent control system scalability or increase communication latencies and dropouts. These unintentional consequences of the security features would therefore hinder the ability of the grid operator to implement specific control algorithms. This project evaluated the tradeoffs between power system performance and cybersecurity metrics for several grid services.

14 SOLAR ENERGY↗

Secure Scalable Control and Communications for Distributed PV (Final Technical Report)

An increasing number of jurisdictions are adopting Distributed Energy Resource (DER) interconnection standards which require photovoltaic (PV) inverters, energy storage systems, and other DER to include interoperable grid-support functionality. These functions provide grid operators the nobs to support local and bulk power system operations with DER equipment, but the associated grid operator-to-DER communications networks must be deployed with appropriate cybersecurity features. In some situations, additional security features may prevent control system scalability or increase communication latencies and dropouts. These unintentional consequences of the security features would therefore hinder the ability of the grid operator to implement specific control algorithms. This project evaluated the tradeoffs between power system performance and cybersecurity metrics for several grid services. This was conducted in two parts.

24 POWER TRANSMISSION AND DISTRIBUTION↗

P38 heterogeneous multi-tiled system with support for message queues (MoSAIC) v0.1

The proposed system is written in the hardware description language (HDL) verilog targeting an FPGA board. It is intended as a testbed to explore architecture tradeoffs in multi-tiled heterogeneous architectures. Although we target FPGAs, the system can be implemented as a monolithic SoC or a package comprised of many chiplets that are interconnected in the same package using a NoC. The proposed NoC is lightweight and follows an axi-lite interface. The endpoints of the NoC are a heterogeneous mix of "tiles" as endpoints that are general purpose processors, fixed function accelerators, and programmable accelerators. We assume that the network interfaces for the NoC endpoints are all addressable in a global name-space in that they represent an address range (for memory addresses) or a range of unique identifiers that are associated with each individual tile. This makes the functionality abstract from the standpoint of the NoC design details. Message queues offer a direct inter-processor interface between peer general purpose cores and diverse accelerators that comprise an SoC. Although they share the same NoC infrastructure for inter-tile communication within an SoC or SiP, the hardware message queues bypass the memory hierarchy and thus do not pollute the memory state or invoke the cache coherence mechanism.

Gonzalez, LouisaPatricia↗

Explorations of new selenites of the group IIIA and IVA metals

Systematic explorations of new phases in the Ga{sup III}/In{sup III}/Ge{sup IV}-Se{sup IV}-O systems by hydrothermal syntheses or solid-state reactions at high-temperature led to six new ternary compounds, namely, M{sub 2}Se{sub 2}O{sub 7} (M=Ga 1, In 2), M(OH)(SeO{sub 3}) (M=Ga 3, In 4), {alpha}-Ge(SeO{sub 3}){sub 2}5 and {beta}-Ge(SeO{sub 3}){sub 2}6. Ga{sub 2}Se{sub 2}O{sub 7}1 displays a 3D open framework composed of 2D gallium oxide layers being further bridged and capped by SeO{sub 3} groups. In{sub 2}Se{sub 2}O{sub 7}2 features a 3D indium oxide framework formed by corner- and edge- sharing InO{sub 6} octahedra with SeO{sub 3} groups attached on the cavities and the 8-member ring tunnels of the structure. The isostructural of M(OH)(SeO{sub 3}) (M=Ga 3, In 4) exhibit a 2D metal selenite layer composed of 1D edge-sharing MO{sub 6} octahedral chains that are interconnected by SeO{sub 3} groups. {alpha}-Ge(SeO{sub 3}){sub 2} (P2{sub 1}/n) 5 displays a 3D open framework with 1D 8-member ring tunnels along the a-axis while {beta}-Ge(SeO{sub 3}){sub 2} (Pa-3) 6 exhibits a condensed 3D network. - Graphical abstract: Highlights: Black-Right-Pointing-Pointer Up to now, selenites of the group IIIA and IVA metals are still rare. Black-Right-Pointing-Pointer Hydrothermal or solid state reactions yielded six new compounds in this system. Black-Right-Pointing-Pointer They are M{sub 2}Se{sub 2}O{sub 7} (M=Ga, In), M(OH)(SeO{sub 3}) (M=Ga, In), {alpha}-Ge(SeO{sub 3}){sub 2} and {beta}-Ge(SeO{sub 3}){sub 2}. Black-Right-Pointing-Pointer They exhibit four different 3D and one 2D structural types. Black-Right-Pointing-Pointer {alpha}-Ge(SeO{sub 3}){sub 2} and {beta}-Ge(SeO{sub 3}){sub 2} represent the first examples of germanium selenites.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Phase Transitions and a Model Order Selection Criterion for Spectral Graph Clustering

One of the longstanding open problems in spectral graph clustering (SGC) is the so-called model order selection problem: automated selection of the correct number of clusters. This is equivalent to the problem of finding the number of connected components or communities in an undirected graph. Here, we propose automated model order selection (AMOS), a solution to the SGC model selection problem under a random interconnection model (RIM) using a novel selection criterion that is based on an asymptotic phase transition analysis. AMOS can more generally be applied to discovering hidden block diagonal structure in symmetric non-negative matrices. Numerical experiments on simulated graphs validate the phase transition analysis, and real-world network data is used to validate the performance of the proposed model selection procedure.

97 MATHEMATICS AND COMPUTING↗

Ground-referenced single-ended memory interconnect

A system is provided for transmitting signals. The system includes a ground-referenced single-ended signaling (GRS) driver circuit that is configured to pre-charge a first capacitor to store a first charge between a first output node and a first reference node based on a first input data signal during a first pre-charge phase and drive an output signal relative to a ground network based on the first charge during a first drive phase. A control circuit is configured to generate a first set of control signals based on the first input data signal and a first clock signal, where the first set of control signals causes the first GRS driver circuit to operate in either the first pre-charge phase or in the first drive phase.

Dally, William J.↗

Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect

High performance multi-GPU computing becomes an inevitable trend due to the ever-increasing demand on computation capability in emerging domains such as deep learning, big data and planet-scale simulations. However, the lack of deep understanding on how modern GPUs can be connected and the real impact of state-of-the-art interconnect technology on multi-GPU application performance become a hurdle. In this paper, we fill the gap by conducting a thorough evaluation on five latest types of modern GPU interconnects: PCIe, NVLink-V1, NVLink-V2, NVLink-SLI and NVSwitch, from six high-end servers and HPC platforms: NVIDIA P100-DGX-1, V100-DGX-1, DGX-2, OLCF’s SummitDev and Summit supercomputers, as well as an SLI-linked system with two NVIDIA Turing RTX-2080 GPUs. Based on the empirical evaluation, we have observed four new types of GPU communication network NUMA effects: three are triggered by NVLink’s topology, connectivity and routing, while one is caused by PCIe chipset design issue. These observations indicate that, for a multi-GPU application running in a multi-GPU node, choosing the right GPU combination can impose considerable impact on GPU communication efficiency, as well as an application’s overall performance. Our evaluation can be leveraged in building practical multi-GPU performance models, which are vital for GPU task allocation, scheduling and migration in a shared environment (e.g., AI cloud and HPC centers), as well as communication-oriented performance tuning.

Performance Evaluation, GPU, Interconnect, NUMA, P↗

DS-GL: Advancing Graph Learning via Harnessing the Power of Nature within Dynamic Systems

With the rapid digitization of the world, an increasing number of real-world applications are turning to nonEuclidean data, modeled as graphs. Due to their intrinsic high complexity and irregularity, learning from graph data demands tremendous computational power. Recently, CMOS-compatible Ising machines, i.e., dynamic systems composed of CMOS components, have emerged as a new approach that harnesses the inherent power of natural annealing within dynamic systems to efficiently resolve binary optimization problems and have been adopted for traditional graph computation, such as max-cut. However, when performing complex Graph Learning (GL) tasks, Ising machines face significant hurdles: (i) they are inherently binary and thus ill-suited for real-valued problems; (ii) their expensive all-to-all coupling network that guarantees effective natural annealing poses daunting scalability concerns. To address these challenges, this paper proposes a nature-powered graph learning framework dubbed DS-GL, which is the first effort to transform the process of solving graph learning problems into the natural annealing process within a parameterized dynamic system embodied as a CMOS chip. To tackle the two major hurdles, DS-GL first augments the Ising machine architecture to modify the self-reaction term of its Hamiltonian function from linear to quadratic, effectively serving as an energy regulator. This adjustment maintains the system’s original physical interpretation while enabling it to process continuous, real-valued data. Second, to address the scaling issue, DS-GL further upgrades the real-valued dense Ising machine by decomposing it into a mesh-based multi-PE dynamic system that supports efficient distributed spatial-temporal co-annealing across different PEs through sparse interconnects. By exploiting the inherent sparsity and component structures in real-world graphs, DS-GL is able to map complex graph learning tasks onto the scalable dynamic system while maintaining high accuracy. Evaluations with three diverse GL applications across six real-world datasets, including traffic flow and COVID-19 prediction, show that DS-GL can deliver from 102× to 106× speedups and 500× energy reduction over Graph Neural Networks on GPUs, with 5% - 20% accuracy enhancement.

Song, Ruibing↗

Cybersecurity Certification Recommendations for Interconnected Grid Edge Devices and Inverter Based Resources

Escalating deployment of PV and grid-edge devices on the distribution grid has increased the sustainability and efficiency of the electric grid. However, the increasing number of distributed energy resources (DERs) deployed creates a heightened cyber-physical interdependency on the distribution grid and thus creates more vectors for cyber-attacks to exploit through information and communication technology (ICT) systems and networks. For example, control signal packets can be modified, intercepted, or corrupted due to vulnerabilities in communication protocols used by microgrid controllers and grid edge devices for power control. Therefore, to mitigate and prevent cyber-attacks on grid edge devices and the inverter-based resources connected to the distribution grid, the U.S. Department of Solar Energy Technologies Office (SETO) awarded funding to the National Renewable Energy Laboratory and Sandia National Laboratory (SNL) to research, develop, and harmonize cybersecurity standards for Photovoltaic (PV) systems and for other kinds of DERs. To help develop a standard for DER cybersecurity, NREL established certification recommendations and test cases, in consensus with the solar industry and UL, for ensuring intrinsic design security for DERs. These recommendations were developed to bolster the cybersecure functionalities such as TLS, MAC, CRL, session resumption/renegotiation, and password, system, and service security management within the DER devices. The proposed test cases verify authentication, authorization, confidentiality, and data integrity for data and communications of DERs that use Transmission Control Protocol/Internet Protocol (TCP/IP). They were also developed to protect DER communications from eavesdropping, replay, man-in-the-middle, denial of service (DoS), spoofing through security certificates, least-privilege violation, and brute-force credentials. This report, which has been validated and reviewed by UL, expands upon those test cases to provide DER cybersecurity certification recommendations which increase DER resiliency and help to mitigate cyber-attacks. UL's collaboration with NREL and approval of this document will accelerate the adoption of a UL standard for DER cybersecurity.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Topolgy Agnostic Hot-Spot Avoidance with InfiniBand

InfiniBand has become a very popular interconnect, due to its advanced features and open standard. Large scale InfiniBand clusters are becoming very popular, as reflected by the TOP 500 supercomputer rankings. However, even with popular topologies like constant bi-section bandwidth Fat Tree, hot-spots may occur with InfiniBand, due to inappropriate configuration of network paths, presence of other jobs in the network and un-availability of adaptive routing. In this paper, we present a hot-spot avoidance layer (HSAL) for InfiniBand, which provides hot-spot avoidance using path bandwidth estimation and multi-pathing using LMC mechanism, without taking the network topology into account. We propose an adaptive striping policy with batch based striping and sorting approach, for efficient utilization of disjoint network paths. Integration of HSAL with MPI, the de facto programming model of clusters, shows promising results with collective communication primitives and MPI applications.

97 MATHEMATICS AND COMPUTING↗

Fail-Safe Logic Design Strategies Within Modern FPGA Architectures

Fail-safe computing refers to computing systems that revert to a non-operational safe state when a fault occurs. In this paper, we investigate a circuit level technique as mitigation for single event upsets (SEUs) and fault injection attacks on field programmable gate arrays (FPGAs), and analyze the effectiveness of the technique as a fail-safe monitor for an encryption algorithm. The propagation of fault effects through FPGA primitives including lookup tables (LUTs) and programmable interconnect points (PIPs) is assessed within an FPGA architecture created using an open source tool, and validated using fault injection experiments on an FPGA. The analysis reveals additional vulnerabilities exist within reconfigurable architectures over those in equivalent fail-safe application specific integrated circuit (ASIC), thus requiring a more elaborate network of redundant circuits and checking logic. The configuration memory bits (CMBs), which configure routing and designate logic functions within the LUTs of the FPGA, add complexity to fail-safe design strategies by introducing additional fault conditions and fault propagation paths. A resource-efficient fail-safe circuit design technique called DEsign for Fail-safe in reCONfigurable systems (DEFCON) is proposed. The benefits and limitations associated with DEFCON are described in the context of fault injection experiments carried out as simulations and in FPGA hardware.

Bhakta, Priya A. [Univ. of New Mexico, Albuquerque↗

The DUNE-DAQ Application Framework

The deep underground neutrino experiment (DUNE) is a next-generation neutrino experiment that will probe the properties of these elusive particles with unparalleled precision. It will also act as an observatory for neutrino bursts caused by nearby supernovae, in the event that one occurs, while the experiment is in operation. Given these goals, the DUNE trigger and DAQ system must be able to maintain extremely high uptime and provide a path for full readout of the detectors for very long times (up to 100 s). To achieve these ends, we have designed the DUNE DAQ system around a flexible “application framework,” which provides a modular interface for specific tasks while handling the interconnections between them. The application framework collects modules into applications, which can then be interacted with as units by the control, configuration, and monitoring systems. One of the key features of the framework is its communication abstraction layer, which allows for modules to interact with both internal queues and external network connections with a single transport-agnostic interface. We will report on the architecture and features of the framework.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The DUNE DAQ Application Framework

The Deep Underground Neutrino Experiment (DUNE) is a next-generation neutrino experiment that will probe the properties of these elusive particles with unparalleled precision. It will also act as an observatory for neutrino bursts caused by nearby supernovae, in the event that one occurs while the experiment is in operation. Given these goals, the DUNE trigger and DAQ system must be able to maintain extremely high uptime and provide a path for full readout of the detectors for very long times (up to 100s). To achieve these ends, we have designed the DUNE DAQ system around a flexible “application framework”, which provides a modular interface for specific tasks while handling the interconnections between them. The application framework collects modules into applications which can then be interacted with as units by the control, configuration and monitoring systems. One of the key features of the framework is its communications abstraction layer, which allows for modules to interact with both internal queues and external network connections with a single transport-agnostic interface. We will report on the architecture and features of the framework.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Knots are not for naught: Design, properties, and topology of hierarchical intertwined microarchitected materials

Lightweight and tough engineered materials are often designed with three-dimensional hierarchy and interconnected structural members whose junctions are detrimental to their performance because they serve as stress concentrations for damage accumulation and lower mechanical resilience. We introduce a previously unexplored class of architected materials, whose components are interwoven and contain no junctions, and incorporate micro-knots as building blocks within these hierarchical networks. Tensile experiments, which show close quantitative agreements with an analytical model for overhand knots, reveal that knot topology allows a new regime of deformation capable of shape retention, leading to a ~92% increase in absorbed energy and an up to ~107% increase in failure strain compared to woven structures, along with an up to ~11% increase in specific energy density compared to topologically similar monolithic lattices. Our exploration unlocks knotting and frictional contact to create highly extensible low-density materials with tunable shape reconfiguration and energy absorption capabilities.

36 MATERIALS SCIENCE↗

Resilience Design Patterns - A Structured Approach to Resilience at Extreme Scale (version 1.2)

We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers’ understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In this version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different effciency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-effcient manner despite frequent faults, errors, and failures of various types.

97 MATHEMATICS AND COMPUTING↗

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (V.2.0)

Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance & power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers' understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types. Version 2.0 expands the resilience design pattern classification and catalog to include self-stabilization patterns and reliability, availability and performance models for each structural pattern.

97 MATHEMATICS AND COMPUTING↗