Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software defined networking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Some Aeronautical Communications Experiments

Classically there has existed an asymmetry between the computing and communicating sides of aerospace systems. Over the past few decades, this asymmetry has shifted to favoring communication link technologies, meaning that advancements in available central processing units (CPUs), storage devices, and internal data buses have stagnated. Indeed, the increased emphasis placed on refining subsystem performance such as with antenna bandwidth in phased arrays, amplifier power efficiency, software defined radio (SDR) flexibility and encoding for data compression and error correction has given rise to successful debuts of multi-gigabit-per-second data return from long space-link distances. These accomplishments are easily quantifiable through link budgets and illustrate what is possible, but also reveal the deficiencies in overall communications capabilities. In particular, the ever-accelerating presence of aerospace vehicles gives rise to newer and larger classes of challenges to address the needs of 21st century systems. Furthermore remote sensing and imaging capabilities have far outpaced our ability to transmit their products to the ground, so we are increasingly dependent on pre-processing and downselection to contend with the communications bottleneck. No longer may we depend upon the constrained logistics in delivering end-to-end data delivery through manual reconfigurations, static event scheduling and execution on a per-vehicle basis, for these methods do not scale and therefore must give way to dynamic, networked approaches with an overall systems view in mind. Emerging mission requirements exhibit a trend toward multiple smaller-scale vehicles working together to perform dissimilar observations. Such operations necessitate sensor fusion across a constellation, and where data processing may be distributed throughout a fairly disconnected network whose topology changes over time in non-deterministic manners. Individual communications link performance is still very relevant to deploying an effective communications system, but now must be embedded within a greater architecture of capability to optimally utilize the bandwidth available from each link to generate an ultimate end-to-end quality of service. The deleterious effects of timing uncertainty across the arrangement presents a challenge to measurement synchronization and delivery, so a successful deployed system needs to be tolerant to the delays inherent in time-of-light between elements and digital processing latencies existing at each node. In this presentation we share the flight test results from a high performance Gbps laser communications terminal evaluated with a suite of store and forward capabilities called High-rate Delay Tolerant Networking (HDTN). The communications payload is operated over Lake Erie across a range of configurations including several convergence layers, and is evaluated to determine recovery time after link disruptions, information loss, efficiency and speed. The effectiveness of utilizing a flying laboratory to increase the Technology Readiness Level (TRL) of an integrated system in relevant environments is discussed, as well as the value of conducting aeronautics experiments to retire risk for technology infusion into space missions. Upcoming flight campaigns will be presented, including opportunities to demonstrate secure command and control, data intensive hyperspectral imaging, quantum link characterization, 4k High Definition (HD) video streaming and internetworked space-ground-aero relay operations. These experiments will pave the way for future missions which will depend upon interoperability across disparate government and privately owned networks, involve contention with uncertain and dynamic timing, and require agility to autonomously configure optimal parameters across networks of ever-increasing size and complexity to ensure data delivery. https://www1.grc.nasa.gov/space/scan/acs/tech-studies/dtn/

Daniel Raible↗

Integrating Ideas for International Data Collaborations Through The Committee on Earth Observation Satellites (CEOS) International Directory Network (IDN)

The capabilities of the International Directory Network's (IDN) version MD9.5, along with a new version of the metadata authoring tool, "docBUILDER", will be presented during the Technology and Services Subgroup session of the Working Group on Information Systems and Services (WGISS). Feedback provided through the international community has proven instrumental in positively influencing the direction of the IDN s development. The international community was instrumental in encouraging support for using the IS0 international character set that is now available through the directory. Supporting metadata descriptions in additional languages encourages extended use of the IDN. Temporal and spatial attributes often prove pivotal in the search for data. Prior to the new software release, the IDN s geospatial and temporal searches suffered from browser incompatibilities and often resulted in unreliable performance for users attempting to initiate a spatial search using a map based on aging Java applet technology. The IDN now offers an integrated Google map and date search that replaces that technology. In addition, one of the most defining characteristics in the search for data relates to the temporal and spatial resolution of the data. The ability to refine the search for data sets meeting defined resolution requirements is now possible. Data set authors are encouraged to indicate the precise resolution values for their data sets and subsequently bin these into one of the pre-selected resolution ranges. New metadata authoring tools have been well received. In response to requests for a standalone metadata authoring tool, a new shareable software package called "docBUILDER solo" will soon be released to the public. This tool permits researchers to document their data during experiments and observational periods in the field. interoperability has been enhanced through the use of the Open Archives Initiative s (OAI) Protocol for Metadata Harvesting (PMH). Harvesting of XML content through OAI-MPH has been successfully tested with several organizations. The protocol appears to be a prime candidate for sharing metadata throughout the international community. Data services for visualizing and analyzing data have become valuable assets in facilitating the use of data. Data providers are offering many of their data-related services through the directory. The IDN plans to develop a service-based architecture to further promote the use of web services. During the IDN Task Team session, ideas for further enhancements will be discussed.

Olsen, Lola M.↗

Autonomous Satellite Command and Control Through the World Wide Web

The Automated Space System Experimental Testbed (ASSET) system is a simple yet comprehensive real-world operations network being developed. Phase 3 of the ASSET Project was January-December 1997 and is the subject of this report. This phase permitted SSDL and its project partners to expand the ASSET system in a variety of ways. These added capabilities included the advancement of ground station capabilities, the adaptation of spacecraft on-board software, and the expansion of capabilities of the ASSET management algorithms. Specific goals of Phase 3 were: (1) Extend Web-based goal-level commanding for both the payload PI and the spacecraft engineer. (2) Support prioritized handling of multiple (PIs) Principle Investigators as well as associated payload experimenters. (3) Expand the number and types of experiments supported by the ASSET system and its associated spacecraft. (4) Implement more advanced resource management, modeling and fault management capabilities that integrate the space and ground segments of the space system hardware. (5) Implement a beacon monitoring test. (6) Implement an experimental blackboard controller for space system management. (7) Further define typical ground station developments required for Internet-based remote control and for full system automation of the PI-to-spacecraft link. Each of those goals are examined. Significant sections of this report were also published as a conference paper. Several publications produced in support of this grant are included as attachments. Titles include: 1) Experimental Initiatives in Space System Operations; 2) The ASSET Client Interface: Balancing High Level Specification with Low Level Control; 3) Specifying Spacecraft Operations At The Product/Service Level; 4) The Design of a Highly Configurable, Reusable Operating System for Testbed Satellites; 5) Automated Health Operations For The Sapphire Spacecraft; 6) Engineering Data Summaries for Space Missions; and 7) Experiments In Automated Health Assessment And Notification For The Sapphire Microsatellite.

Cantwell, Brian↗

NASA Near Earth Network (NEN) DVB-S2 Demonstration Testing for Enhancing Higher Data Rates for CubeSat/Small Satellite Missions at X-/Ka-band

National Aeronautics and Space Administration (NASA) CubeSat/SmallSat missions are moving to higher data rates. Digital Video Broadcast, Satellite Second Generation (DVB-S2) is a communications standard that uses power and bandwidth efficient modulation and coding techniques to deliver performance approaching radio frequency (RF) channel theoretical limits. The Near Earth Network (NEN) will test DVB-S2’s ability to provide higher data rates for CubeSat/SmallSat missions at X-band and Ka-band at Wallops Flight Facility (WFF). The goal is to upgrade the NEN with DVB-S2 to increase science data return for missions and enable support for more CubeSat/SmallSat missions. This paper describes NEN DVB-S2 X-band and Ka-band demonstration objectives, scope, and performance measures as well as NEN channel test configuration. The NEN has planned 2020 tests to demonstrate all modulation and coding schemes in the Consultative Committee for Space Data Systems (CCSDS) DVB-S2 standard over X-band and Ka-band. A link analysis study for the trade-offs among achievable data rates, modulations, codes, spacecraft antenna sizes and power amplifiers (PA) is provided. This paper identifies Commercial off-the-shelf (COTS) CubeSat/SmallSat X-band and Ka-band communication systems and discusses low cost DVB-S2 X-band software defined radio (SDR) transmitter development concepts and implementation with a practical system for CubeSats/SmallSats.

Yen Wong↗

Work Coordination Engine

The Work Coordination Engine (WCE) is a Java application integrated into the Service Management Database (SMDB), which coordinates the dispatching and monitoring of a work order system. WCE de-queues work orders from SMDB and orchestrates the dispatching of work to a registered set of software worker applications distributed over a set of local, or remote, heterogeneous computing systems. WCE monitors the execution of work orders once dispatched, and accepts the results of the work order by storing to the SMDB persistent store. The software leverages the use of a relational database, Java Messaging System (JMS), and Web Services using Simple Object Access Protocol (SOAP) technologies to implement an efficient work-order dispatching mechanism capable of coordinating the work of multiple computer servers on various platforms working concurrently on different, or similar, types of data or algorithmic processing. Existing (legacy) applications can be wrapped with a proxy object so that no changes to the application are needed to make them available for integration into the work order system as "workers." WCE automatically reschedules work orders that fail to be executed by one server to a different server if available. From initiation to completion, the system manages the execution state of work orders and workers via a well-defined set of events, states, and actions. It allows for configurable work-order execution timeouts by work-order type. This innovation eliminates a current processing bottleneck by providing a highly scalable, distributed work-order system used to quickly generate products needed by the Deep Space Network (DSN) to support space flight operations. WCE is driven by asynchronous messages delivered via JMS indicating the availability of new work or workers. It runs completely unattended in support of the lights-out operations concept in the DSN.

Zendejas, Silvino↗

MarCO: Interplanetary Mission Development on a CubeSat Scale

Shortly after JPL’s Interior Exploration using Seismic Investigations, Geodesy and Heat Transport (InSight) mission launches, separates, and commences its cruise phase, two CubeSats will deploy from the launch vehicle’s upper stage and begin independent flight to Mars (Fig. 1). During InSight’s entry, descent, and landing (EDL) sequence, these twin Mars Cube One (MarCO) spacecraft will fly 3,500 km above the Martian surface, recording and relaying InSight UHF radio data to the Deep Space Network (DSN) on Earth1. MarCO is a twin CubeSat mission developed by the NASA Jet Propulsion Laboratory (JPL) to accompany the InSight (Interior Exploration using Seismic Investigations, Geodesy and Heat Transport) Mars mission lander. MarCO's primary mission objective is to launch with InSight and independently fly to Mars to serve as a communications relay during InSight's entry, descent, and landing (EDL) phase. MarCO represents a new type of deep space mission: CubeSats at Mars. Building on the development of JPL's first interplanetary CubeSat project, the Interplanetary Nano-Spacecraft Pathfinder in Relevant Environment (INSPIRE), MarCO further refined the approach to hardware, software, and ground architecture development to solve the challenges of quickly building low-budget spacecraft to fly to Mars. The greatest constraint, beyond others typical of CubeSat missions, was time. The duration between MarCO's conception to completion of spacecraft assembly was less than two years - an unprecedented schedule for any planetary mission to date. Through necessity, MarCO has built on previous experience, procedures, systems, and development methodologies, defining a new niche for supporting larger primary missions. The MarCO spacecraft are poised to write a new chapter in deep space exploration. Originally slated to launch and reach Mars in 2016, the InSight mission schedule subsequently slipped to 2018. During the original landing of InSight, Earth would not be in view, and no orbiter around Mars would have been in position to both receive UHF EDL data and simultaneously relay it back to Earth. It was from this obstacle that MarCO was conceived. Regardless of any changes to InSight’s 2018 EDL configuration geometry, MarCO is still expected to fly and serve in the same capacity as originally designed: the first CubeSat mission to Mars. CubeSats have historically been firmly in the domain of universities and small companies. As first conceived, they served as a platform upon which to teach all aspects of the space mission lifecycle. JPL took on this mission type with Interplanetary Nano-Spacecraft Pathfinder in Relevant Environment2 (INSPIRE), moving the concept into a new domain: deep space. Building from the INSPIRE platform and lessons learned, MarCO addressed new challenges in the domain of planetary missions: independent interplanetary flight and navigation, integration with a large-scale mission, long-distance and long-delay communication, short development time, and a small development team. Of these, the greatest constraint was schedule: only 18 months passed from conception of mission concept until delivery of fully assembled and tested flight hardware. Careful selection of mission team, along with extensive use of off-the-shelf equipment, and streamlining automated processes, was essential. This achievement represents the next step in the evolution of CubeSats beyond low-Earth orbit.

Werne, Thomas↗

Characterization of Extremes and Compound Impacts: Applications of Machine Learning and Interpretable Neural Networks

Focal Area: This white paper responds to Focal area III by exploring data fusion, learning and explainable AI methods in characterizing hydrological extremes and interconnections. It also addresses Focal area II by using probabilistic AI and ensemble ML for predicting extremes and compound extremes. Science Challenge: A key question associated with the integrated water (or hydrological) cycle grand challenge in the Earth and Environmental Systems Sciences Division (EESSD) strategic plan, is how the frequency and intensity of hydrological events will change. Prediction of the tail behavior (extremes) of the hydrological cycle is especially challenging, because of their stochasticity and low probability. These extreme events and their compound impacts have significant societal and economic consequences. It is anticipated for the next-generation Earth System models (ESMs), that model predictability of the water cycle will improve with increased resolution (e.g., regionally refined E3SM), advanced software and computational architectures, and improved model physics based on the data from ARM measurements and high-fidelity models. However, the challenges for predictability of low-probability high-impact extreme events will unlikely be alleviated with conventional modeling and data-driven approaches, as ESMs are calibrated largely for capturing the high-frequency mean climate states. Recent AI and ML applications have shown great potential in quantifying well-defined climate extremes (e.g., supervised learning of tropical cyclones/atmospheric rivers by ClimateNet1) but few efforts are dedicated to compound events, extreme drivers and uncertainty estimation. We envision the opportunity to develop and apply ML and interpretable AI methods extended on the existing efforts, specifically, for: (1) identification of compound extremes, (2) diagnosing drivers of extremes, (3) bias correction in extreme predictions and (4) probabilistic modeling of extremes.

54 ENVIRONMENTAL SCIENCES↗

Reconfigurable Transceiver and Software-Defined Radio Architecture and Technology Evaluated for NASA Space Communications

The NASA Glenn Research Center is investigating the development and suitability of a software-based open-architecture for space-based reconfigurable transceivers (RTs) and software-defined radios (SDRs). The main objectives of this project are to enable advanced operations and reduce mission costs. SDRs are becoming more common because of the capabilities of reconfigurable digital signal processing technologies such as field programmable gate arrays and digital signal processors, which place radio functions in firmware and software that were traditionally performed with analog hardware components. Features of interest of this communications architecture include nonproprietary open standards and application programming interfaces to enable software reuse and portability, independent hardware and software development, and hardware and software functional separation. The goals for RT and SDR technologies for NASA space missions include prelaunch and on-orbit frequency and waveform reconfigurability and programmability, high data rate capability, and overall communications and processing flexibility. These operational advances over current state-of-art transceivers will be provided to reduce the power, mass, and cost of RTs and SDRs for space communications. The open architecture for NASA communications will support existing (legacy) communications needs and capabilities while providing a path to more capable, advanced waveform development and mission concepts (e.g., ad hoc constellations with self-healing networks and high-rate science data return). A study was completed to assess the state of the art in RT architectures, implementations, and technologies. In-house researchers conducted literature searches and analysis, interviewed Government and industry contacts, and solicited information and white papers from industry on space-qualifiable RTs and SDRs and their associated technologies for space-based NASA applications. The white papers were evaluated, compiled, and used to assess RT and SDR system architectures and core technology elements to determine an appropriate investment strategy to advance these technologies to meet future mission needs. The use of these radios in the space environment represents a challenge because of the space radiation suitability of the components, which drastically reduces the processing capability. The radios available for space are considered to be RTs (as opposed to SDRs), which are digitally programmable radios with selectable changes from an architecture combining analog and digital components. The limited flexibility of this design contrasts against the desire to have a power-efficient solution and open architecture.

Reinhart, Richard C.↗

A mission planning concept and mission planning system for future manned space missions

The international character of future manned space missions will compel the involvement of several international space agencies in mission planning tasks. Additionally, the community of users requires a higher degree of freedom for experiment planning. Both of these problems can be solved by a decentralized mission planning concept using the so-called 'envelope method,' by which resources are allocated to users by distributing resource profiles ('envelopes') which define resource availabilities at specified times. The users are essentially free to plan their activities independently of each other, provided that they stay within their envelopes. The new developments were aimed at refining the existing vague envelope concept into a practical method for decentralized planning. Selected critical functions were exercised by planning an example, founded on experience acquired by the MSCC during the Spacelab missions D-1 and D-2. The main activity regarding future mission planning tasks was to improve the existing MSCC mission planning system, using new techniques. An electronic interface was developed to collect all formalized user inputs more effectively, along with an 'envelope generator' for generation and manipulation of the resource envelopes. The existing scheduler and its data base were successfully replaced by an artificial intelligence scheduler. This scheduler is not only capable of handling resource envelopes, but also uses a new technology based on neuronal networks. Therefore, it is very well suited to solve the future scheduling problems more efficiently. This prototype mission planning system was used to gain new practical experience with decentralized mission planning, using the envelope method. In future steps, software tools will be optimized, and all data management planning activities will be embedded into the scheduler.

Wickler, Martin↗

SDR Input Power Estimation Algorithms

The General Dynamics (GD) S-Band software defined radio (SDR) in the Space Communications and Navigation (SCAN) Testbed on the International Space Station (ISS) provides experimenters an opportunity to develop and demonstrate experimental waveforms in space. The SDR has an analog and a digital automatic gain control (AGC) and the response of the AGCs to changes in SDR input power and temperature was characterized prior to the launch and installation of the SCAN Testbed on the ISS. The AGCs were used to estimate the SDR input power and SNR of the received signal and the characterization results showed a nonlinear response to SDR input power and temperature. In order to estimate the SDR input from the AGCs, three algorithms were developed and implemented on the ground software of the SCAN Testbed. The algorithms include a linear straight line estimator, which used the digital AGC and the temperature to estimate the SDR input power over a narrower section of the SDR input power range. There is a linear adaptive filter algorithm that uses both AGCs and the temperature to estimate the SDR input power over a wide input power range. Finally, an algorithm that uses neural networks was designed to estimate the input power over a wide range. This paper describes the algorithms in detail and their associated performance in estimating the SDR input power.

Nappier, Jennifer M.↗

Intelligent Integrated Health Management for a System of Systems

An intelligent integrated health management system (IIHMS) incorporates major improvements over prior such systems. The particular IIHMS is implemented for any system defined as a hierarchical distributed network of intelligent elements (HDNIE), comprising primarily: (1) an architecture (Figure 1), (2) intelligent elements, (3) a conceptual framework and taxonomy (Figure 2), and (4) and ontology that defines standards and protocols. Some definitions of terms are prerequisite to a further brief description of this innovation: A system-of-systems (SoS) is an engineering system that comprises multiple subsystems (e.g., a system of multiple possibly interacting flow subsystems that include pumps, valves, tanks, ducts, sensors, and the like); 'Intelligent' is used here in the sense of artificial intelligence. An intelligent element may be physical or virtual, it is network enabled, and it is able to manage data, information, and knowledge (DIaK) focused on determining its condition in the context of the entire SoS; As used here, 'health' signifies the functionality and/or structural integrity of an engineering system, subsystem, or process (leading to determination of the health of components); 'Process' can signify either a physical process in the usual sense of the word or an element into which functionally related sensors are grouped; 'Element' can signify a component (e.g., an actuator, a valve), a process, a controller, an actuator, a subsystem, or a system; The term Integrated System Health Management (ISHM) is used to describe a capability that focuses on determining the condition (health) of every element in a complex system (detect anomalies, diagnose causes, prognosis of future anomalies), and provide data, information, and knowledge (DIaK) not just data to control systems for safe and effective operation. A major novel aspect of the present development is the concept of intelligent integration. The purpose of intelligent integration, as defined and implemented in the present IIHMS, is to enable automated analysis of physical phenomena in imitation of human reasoning, including the use of qualitative methods. Intelligent integration is said to occur in a system in which all elements are intelligent and can acquire, maintain, and share knowledge and information. In the HDNIE of the present IIHMS, an SoS is represented as being operationally organized in a hierarchical-distributed format. The elements of the SoS are considered to be intelligent in that they determine their own conditions within an integrated scheme that involves consideration of data, information, knowledge bases, and methods that reside in all elements of the system. The conceptual framework of the HDNIE and the methodologies of implementing it enable the flow of information and knowledge among the elements so as to make possible the determination of the condition of each element. The necessary information and knowledge is made available to each affected element at the desired time, satisfying a need to prevent information overload while providing context-sensitive information at the proper level of detail. Provision of high-quality data is a central goal in designing this or any IIHMS. In pursuit of this goal, functionally related sensors are logically assigned to groups denoted processes. An aggregate of processes is considered to form a system. Alternatively or in addition to what has been said thus far, the HDNIE of this IIHMS can be regarded as consisting of a framework containing object models that encapsulate all elements of the system, their individual and relational knowledge bases, generic methods and procedures based on models of the applicable physics, and communication processes (Figure 2). The framework enables implementation of a paradigm inspired by how expert operators monitor the health of systems with the help of (1) DIaK from various sources, (2) software tools that assist in rapid visualization of the condition of the system, (3) analical software tools that assist in reasoning about the condition, (4) sharing of information via network communication hardware and software, and (5) software tools that aid in making decisions to remedy unacceptable conditions or improve performance.

Smith, Harvey↗

Towards On-Chip Learning for Low Latency Reasoning with End-to-End Synthesis

The Software Defined Architectures (SODA) Synthesizer is an open-source compiler-based tool able to automatically generate domain-specialized systems targeting Application-Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs) starting from high-level programming. SODA is composed of a frontend, SODA-OPT, which leverages the multilevel intermediate representation (MLIR) framework to interface with productive programming tools (e.g., machine learning frame-works), identify kernels suitable for acceleration, and perform high-level optimizations, and of a state-of-the-art high-level synthesis backend, Bambu from the PandA framework, to generate custom accelerators. One specific application of the SODA Synthesizer is the generation of accelerators to enable ultra-low latency inference and control on autonomous systems for scientific discovery (e.g., electron microscopes, sensors in particle accelerators, etc.). This paper provides an overview of the flow in the context of the generation of accelerators for edge processing to be integrated in transmission electron microscopy (TEM) devices, focusing on use cases from precision material synthesis. We show the tool in action with an example of design space exploration for inference on reconfigurable devices with a conventional deep neural network model (LeNet). Finally, we discuss the research directions and opportunities enabled by SODA in the area of autonomous control for scientific experimental workflows.

Castellana, Vito G.↗

Reconstruction of Six-Dimensional Phase Space

A phase space is a mathematical representation of all possible physical states of a system. Particle beams at Fermilab exist within a six-dimensional (6D) phase space defined by three positional components, (x, y, z) and three momentum components, (px, py, pz). To reconstruct this space implies taking measurement data from detectors and mapping out particle behavior using computational methods. The beam detectors, however, are only able to detect spatial distribution among the events of the beam, therefore being limited to positional data. Also, due to the vast number of events in a particle beam, it is extremely difficult to analyze and differentiate every single one’s behavior. However, with Machine Learning (ML), which can distinguish between patterns and map out particle behavior more efficiently. We first used the particle beam software, G4beamline, to simulate a 10,000-event muon beam, adjusting parameters such as initial momentum magnitude (p¬0) and virtual detector position. Using ten virtual detectors, we analyzed p0 values such that minimum 9,990 events were analyzed by every detector. We then input the data from these beam simulations to a C++ program, that randomly selects 100 events, and creates a 2D histogram based on spatial distribution, detector position, and event intensity. This process is repeated 100 times to create 100 histograms per p0 value. These images were then input to a modified ResNet18 Convolutional Neural Network (CNN) for training, and to predict p0 from some unseen set of histograms. The model was accurate when trained on momentum increments of 5 MeV/c and provided with denser training samples around highly variable test values. These results displayed machine learning being able to accurately predict p0 from being trained on different particle behaviors.

Shirlee, Jermain [Fermilab]↗

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (V.2.0)

Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance & power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers' understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types. Version 2.0 expands the resilience design pattern classification and catalog to include self-stabilization patterns and reliability, availability and performance models for each structural pattern.

97 MATHEMATICS AND COMPUTING↗

DOE BSSD Performance Management Metrics Report Q1

Microbes play key roles in our biosphere, from driving global nutrient cycling to impacting plant, animal and human health and disease. Complex data from microbial genomes, proteins, and metabolites provide a window into these tiny engines that drive life on our planet. Yet these data are dispersed among researchers’ laboratories and various repositories, making it difficult to access. This calls for new ways of managing data, improving data interoperability, advancing community standards, and creating an infrastructure where data are shared efficiently. We have built the National Microbiome Data Collaborative (NMDC) to advance how scientists create, use, and reuse data to redefine the way we understand and harness the power of microbes. The vision of the National Microbiome Data Collaborative (NMDC) is to drive a microbiome data sharing network connecting data, people, and ideas to advance microbiome innovation and discovery. The NMDC was launched in 2019 and brought together DOE National Laboratories to collaborate across resources, capabilities, and expertise. The NMDC team was strategically assembled to include software developers, microbial researchers, metadata experts, and multi-omics specialists. The diversity of the NMDC team reflects the inherently interdisciplinary nature of microbiome science, and we leverage the strengths of the DOE National Laboratory system. Towards BER’s goal of advancing an iterative systems biology approach to the understanding of microbial genomes, the NMDC serves as a foundation for infrastructure, data standards, and community building. Together with the flagship DOE User Facilities, the Joint Genome Institute (JGI) and the Environmental Molecular Sciences Laboratory (EMSL), we are developing core capabilities in metadata standards for environmental descriptors and sample handling and processing; standardized bioinformatic workflows; an interface for data search and access; and robust community engagement activities. The NMDC production platform supports long-term data infrastructure and community building for BER’s bioenergy and environmental research goals. Our approach leverages lessons learned and an ambitious framework for collaborative, interdisciplinary data infrastructure to support microbiome research. The NMDC supports data, information, and knowledge access through three defined software tools – the Submission Portal, NMDC EDGE, and the Data Portal – driven by community needs. Herein, we describe the value proposition for the microbiome research community, our overarching strategy, and challenges and opportunities for developing the NMDC as both an infrastructure and community engagement program.

59 BASIC BIOLOGICAL SCIENCES↗

FY21 Progress Report: SRNL Analysis of ICCWR LCM and WAMS data for Corrosion and Cracking

The development of algorithms for machine learning and data analysis for the 3013 Surveillance Program is a collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). For corrosion detection, Laser Confocal Microscope (LCM) or Wide Area 3D Measurement System (WAMS) data is extracted from large binary files, with software written to convert the data to physical attributes (e.g., height, color and grayscale values; all as functions of a location in a plane projection). A user-friendly Matlab Graphical User Interface (GUI) that reads data from either LCM or WAMS files was developed to integrate input data with software developed for processing and evaluation. The GUI can selectively download binary data, interrogate data attributes, label data, flag significant features, execute Machine Learning (ML) algorithms, output parameters for trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. Features can be called out by user-specified thresholds, manual labeling or machine learning algorithms when they have been completed. The ability to rapidly label data is important because of the volume of data required for training machine learning algorithms. The GUI has the flexibility to allow addition of improved ML algorithms, methods for data visualization, and statistical computations. Statistical analyses via the GUI include areas of pits within a defined range of pit depths, correlations between Red-Green-Blue (RGB) or grayscale intensity and relative surface height, covariances between values associated with features, and feature histograms. The development of supervised machine learning algorithms, however, has been hindered by a lack of training data. The machine learning algorithms for crack identification are being refined but require improvements to the true positive rate for crack detection. This shortcoming is an artifact of the limited training data currently available, perhaps more so than the structure of the neural networks. At present, the best results are had from a consensus over an ensemble of randomly generated Deep Neural Network (DNN) or Convolutional Neural Network (CNN) algorithms. Although the consensus accuracy method has yielded optimum true positive and true negative rates in excess of 80%, additional validation testing is necessary. In addition to the suite of LCM data that was initially used, and which represents the majority of the work presented in this report, WAMS image data was also reviewed at a preliminary level. The review included a comparison between image resolution and dynamic range for each method. WAMS (ZON file) image data was found to have a pixel pitch of 3.69μm compared to 1 μm for the LCM (vk4 file) data, which implies a lower resolution for the WAMS images. Conversely, the ratio of dynamic range of the WAMS data to the LCM data was approximately 41:20 for height data, suggesting that information from WAMS should more accurately determine the depth of pits. At present, the significance of the greater dynamic range of the WAMS data relative to the LCM data has not yet been evaluated.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

NWGraph: A Library of Generic Graph Algorithms and Data Structures in C++20

The C++ Standard Library is a valuable collection of generic algorithms and data structures that improves the usability and reliability of C++ software. Graph algorithms and data structures are notably absent from the standard library, and previous attempts to fill this gap have not gained widespread adoption. In this paper we show that the richness of graph algorithms and data structures can in fact be captured by straightforward composition of existing C++ mechanisms. Generic programming is algorithm-oriented. Accordingly, we apply a systematic approach to analyzing a broad set of graph algorithms, “lift” unnecessary constraints from them, and organize the resulting set of minimal common type requirements, i.e., concepts, for defining their interfaces. By using the newly available ranges and concepts in C++20, the type requirements for generic graph algorithms can be succinctly expressed. The generic algorithms and data structures resulting from our analysis are realized in NWGraph, in a modern, composable, and extensible C++ library.

graphs and networks, programming language, C++20↗

Adoption of image-driven machine learning for microstructure characterization and materials design: A Perspective

Microstructure characterization enables the development of structure-processing-property relationships critical to several research areas within the broad field of materials science, from alloy design to the assessment of corrosion resistance, and failure analysis. Conventional approaches to material characterization have relied on either qualitative inference by the human ex-pert or software applications that can extract high-level features from images, such as boundary segmentation, average grain diameter, etc. Such approaches rely heavily on subject matter expert user intervention and knowledge of what phases or more generally, what microstructural features, are of interest. The recent surge in the adoption of machine learning techniques to address problems in materials engineering has brought with it an increased interest and application of Image Driven Machine Learning (IDML) approaches. In this work, we review the applications of IDML to the field of materials characterization. A canonical hierarchy of stages is defined, which when put sequentially together completes an IDML study: problem definition, dataset building, model selection and training, model evaluation, and integration with existing instrumentation or simulation workflow. The studies reviewed in this work are analyzed from the perspective of each of these stages. Such a review permits agranular assessment of the field, for example the impact of IDML on materials characterization at the nanoscale, the size of a typical dataset required to train a semantic segmentation model on electron microscopy images, ubiquitousness of transfer learning in the domain, etc. Finally, we discuss the importance of interpretability and explainability in the field of IDML for materials characterization, and provide an overview of two emerging techniques in the field: semantic segmentation and generative adversarial networks.

Baskaran, Arun↗