Results for “Event detection” — Page 40

DOE OSTI2020

An outflow powers the optical rise of the nearby, fast-evolving tidal disruption event AT2019qiz

ABSTRACT At 66 Mpc, AT2019qiz is the closest optical tidal disruption event (TDE) to date, with a luminosity intermediate between the bulk of the population and the faint-and-fast event iPTF16fnl. Its proximity allowed a very early detection and triggering of multiwavelength and spectroscopic follow-up well before maximum light. The velocity dispersion of the host galaxy and fits to the TDE light curve indicate a black hole mass ≈106 M⊙, disrupting a star of ≈1 M⊙. By analysing our comprehensive UV, optical, and X-ray data, we show that the early optical emission is dominated by an outflow, with a luminosity evolution L ∝ t2, consistent with a photosphere expanding at constant velocity (≳2000 km s−1), and a line-forming region producing initially blueshifted H and He ii profiles with v = 3000–10 000 km s−1. The fastest optical ejecta approach the velocity inferred from radio detections (modelled in a forthcoming companion paper from K. D. Alexander et al.), thus the same outflow may be responsible for both the fast optical rise and the radio emission – the first time this connection has been observed in a TDE. The light-curve rise begins 29 ± 2 d before maximum light, peaking when the photosphere reaches the radius where optical photons can escape. The photosphere then undergoes a sudden transition, first cooling at constant radius then contracting at constant temperature. At the same time, the blueshifts disappear from the spectrum and Bowen fluorescence lines (N iii) become prominent, implying a source of far-UV photons, while the X-ray light curve peaks at ≈1041 erg s−1. Assuming that these X-rays are from prompt accretion, the size and mass of the outflow are consistent with the reprocessing layer needed to explain the large optical to X-ray ratio in this and other optical TDEs, possibly favouring accretion-powered over collision-powered outflow models.

Nicholl, M.↗

DOE OSTI2020

Metastable nuclear isomers as dark matter accelerators

Inelastic dark matter and strongly interacting dark matter are poorly constrained by direct detection experiments since they both require the scattering event to deliver energy from the nucleus into the dark matter in order to have observable effects. We propose to test these scenarios by searching for the collisional deexcitation of metastable nuclear isomers by the dark matter particles. The longevity of these isomers is related to a strong suppression of γ- and β-transitions, typically inhibited by a large difference in the angular momentum for the nuclear transition. The collisional deexcitation by dark matter is possible since heavy dark matter particles can have a momentum exchange with the nucleus comparable to the inverse nuclear size, hence lifting tremendous angular momentum suppression of the nuclear transition. This deexcitation can be observed either by searching for the direct effects of the decaying isomer, or through the rescattering or decay of excited dark matter states in a nearby conventional dark matter detector setup. Existing nuclear isomer sources such as naturally occurring 180m Ta, 137m Ba produced in decaying Cesium in nuclear waste, 177m Lu from medical waste, and 178m Hf from the Department of Energy storage can be combined with current dark matter detector technology to search for this class of dark matter.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

Understanding Impacts of Frequency Calculations on Underfrequency Load Shedding

Power system fundamental frequency-dependent protection decisions and control decisions are increasingly common in the distribution system space. Understanding the algorithms used in frequency measurements is critical to understanding the decisions made by the many intelligent electronic devices in the power system. These decisions are critically important for reliability analysis for interconnected power systems as they bear directly on resource planning to mitigate under-frequency conditions that would result in under-frequency load shedding (UFLS). This work aims to improve the understanding of frequency measurement algorithms, evaluate the intelligent electronic devices that use frequency measurements for decision making, and understand the algorithms' direct impact on frequency related decision making. First, this work presents background information on the use of frequency measurements in protection logic, specifically UFLS. Second, the paper presents a relay hardware evaluation test bed used to detect and protect systems from under-frequency events. Finally, the paper presents the dynamic events used to evaluate commercially available, off-the-shelf relay equipment and the results from the relay evaluation.

COMTRADE↗

DOE OSTI2025

A Weakly Supervised Machine Learning Procedure for Magnet Quench Diagnostics

Voltage taps remain the standard and reliable diagnostic tool for detecting quenches in superconducting magnets. However, they identify a quench only at the time of voltage rise and do not provide information on earlier physical precursors. In this work, we investigate whether acoustic emission data can reveal precursor activity that occurs before conventional voltage detection using machine learning techniques. We introduce an event selection method and a weakly supervised machine learning procedure to learn data-driven criteria for identifying potential acoustic precursors to quenches. Two Convolutional Neural Network (CNN) architectures are trained: one on acoustic sensor events from our selection procedure and one on the Fast Fourier Transforms (FFTs) of these events. Both networks are trained iteratively using confidence-weighted loss functions to associate certain subsets of training data with a precursor label. We evaluate the performance of these models by examining the time distribution of events classified as potential precursors relative to the quench onset. Results indicate that the proposed approach can possibly distinguish acoustic emission events occurring closer to the quench from earlier acoustic activity during ramping, suggesting the potential for flagging quench precursors in acoustic data.

Khan, Maira [Fermilab] (ORCID:0009000891602387)↗

DOE OSTI2021

Detection of Anomalies in Environmental Gamma Radiation Background with Hopfield Artificial Neural Network - Consortium on Nuclear Security Technologies (CONNECT) Q3 Report

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to investigate performance of a Hopfield Neural Network (HNN) in in detection and identification of weak nuisances and anomalies events in the presence of a highly fluctuating background. Performance of HNN algorithm is benchmarked using search data from an environmental screening campaign. One data set contained a 137 Cs source, and another dataset contained a 131 I source.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

Development of Hopfield Artificial Neural Network for Anomaly Detection in Environmental Gamma Radiation Background: Consortium on Nuclear Security Technologies (CONNECT) (Q2 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore supervised machine learning (ML) algorithms for development of a Hopfield Neural Network (HNN) in conjunction with an image processing algorithm for detection and identification of weak nuisances and anomalies events in the presence of a highly fluctuating background.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

Detection of Anomalies in Gamma Background Radiation Data with K-Means and Self-Organizing Map Clustering Algorithms (Consortium on Nuclear Security Technologies (CONNECT) Q1 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. The challenge is that spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2022

In-Field Alpha Spectrometer Development FY2022 Mid-Year Report

In response to needs identified by the International Atomic Energy Agency (IAEA), Idaho National Laboratory (INL) has developed an In-Field Alpha Spectrometer (IFAS) to allow IAEA safeguards inspectors to collect samples of uranium hexafluoride (UF 6 ) at processing facilities, to perform field assessments to verify uranium enrichment. For sample collection, the IFAS method uses Single-Use Destructive Assay (SUDA) samplers, developed at Pacific Northwest National Laboratory (PNNL), which contain thin zeolite coatings that trap UF6 gas and convert it to the safer, more stable form uranyl fluoride (as a dihydrate, UO 2 F 2 ·2H 2 O). For alpha spectrometry, the IFAS instrument employs a large area silicon semiconductor transducer to detect and record alpha particle energy-deposition events. Over the course of this project INL, PNNL, Oak Ridge National Laboratory worked to optimize the SUDA sample design for alpha spectrometry, to optimize processes for manufacturing, loading, and shipping SUDA samples, and to allow the measurement of loaded SUDA samples. In fiscal year (FY) 2022 INL has bene investigating the reproducibility and measurement precision of IFAS measurements and the associated determination of uranium enrichment.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

DOE OSTI2025

Quantum Gravity and Laser Interferometry: Towards Observable Predictions

Understanding quantum gravity remains one of the deepest challenges in modern physics, as direct experimental access to Planck-scale effects is beyond current technological reach. However, recent theoretical advances indicate that quantum fluctuations of spacetime may produce measurable effects in precision experiments, particularly near causal horizons. This opens new avenues for testing quantum gravity phenomena through high-precision measurement techniques. This dissertation develops multiple theoretical models to characterize these effects and examines their potential observational signatures in future gravitational wave interferometers. We begin by investigating the role of quantum fluctuations in near-horizon geometries through the lens of the AdS/CFT correspondence, which provides a powerful framework for understanding the interplay between quantum field theory and general relativity via holographic principles. By modeling stochastic energy-momentum sources in Rindler-AdS spacetime, we demonstrate that vacuum fluctuations transform the Einstein equations into a Langevin-type stochastic differential equation, leading to potentially observable fluctuations in photon traversal times. Extending this approach to Minkowski spacetime, we establish a correspondence between gravitational shockwaves and fluid dynamics, showing that near-horizon perturbations satisfy an equation analogous to that governing incompressible fluids, thereby reinforcing the membrane paradigm and hydrodynamic analogies in the context of the fluid/gravity duality. Furthermore, we construct the covariant phase space of a spherically symmetric causal diamond in Minkowski spacetime, identifying two fundamental charges that govern its evolution. These results provide a foundation for quantizing causal horizons and understanding their microscopic degrees of freedom. Building upon these theoretical developments, we further examine a related stochastic phenomenon: the gravitational wave memory background arising from the cumulative memory steps produced by supermassive black hole mergers. After reviewing the standard stochastic gravitational wave background, gravitational memory effects, and BMS symmetries, we model the stochastic memory background using a Brownian motion framework. We show that while the cumulative memory background initially appears above the sensitivity curve of space-based interferometers like LISA, the realistic subtraction of individually resolvable merger events substantially suppresses the residual signal, making its detection more challenging. This highlights the critical importance of source subtraction when evaluating the detectability of gravitational memory effects. By bridging fundamental theory with experimental prospects, this dissertation contributes to the ongoing effort to uncover the quantum nature of spacetime through precision measurement techniques. Whether through detecting quantum spacetime fluctuations, gravitational memory backgrounds, or probing the symmetries of causal horizons, the pursuit of observable quantum gravity phenomena continues to expand the frontiers of both theory and experiment.

Zhang, Yiwen [Caltech] (ORCID:0000000323559416)↗

DOE OSTI2021

Optimization Of In-field Alpha Spectrometry For Uranium Enrichment Determination In Uranium Hexafluoride

In response to needs identified by the International Atomic Energy Agency (IAEA) research is underway to develop In-Field Alpha Spectrometry (IFAS) as a method to allow IAEA safeguards inspectors to collect samples of uranium hexafluoride (UF6) at processing facilities to assess and verify uranium enrichment. For sample collection, the IFAS method uses Single-Use Destructive Assay (SUDA) samplers, which contain thin zeolite coatings that trap UF6 gas and convert it to the safer, more stable form uranyl fluoride (as a dihydrate, UO2F2·2H2O). For alpha spectrometry, the IFAS instrument employs a large area silicon semiconductor transducer to detect and record alpha particle energy-deposition events. Over the past year optimization work has significantly increased the diameter of useful SUDA samples (from 12.7 mm to 48 mm), improved the manufacturability and reproducibility of SUDA samples, increased the area of the IFAS alpha spectrometer sensor from 1.2 cm to 3.1 cm, and improved source positioning within the IFAS. This paper will report on this optimization work, its impacts on IFAS performance, and future plans for IFAS miniaturization, improvements, and testing.

Chichester, David↗

DOE OSTI2024

Leveraging Hydropower Multi-Sensor Data for Inference and Age-Informed Modeling

Increased demand of operational flexibility such as faster ramp up/down in generation, and more frequent start/stops are putting hydropower plants and their associated components in unprecedented stress. Consequently, these plants are at the high risk of extended and more frequent outage to accommodate unscheduled, and unexpected maintenance. Therefore, hydropower plants are in critical need of data driven and age-informed analysis for their regular and unscheduled operation. Yet not all hydropower plants are exhaustively equipped with sensors and/or measurement streams for their respective components – demanding solutions on how to detect, identify, and locate the cause of any event from the unobservable. Idaho National Laboratory (INL) analyzed the anonymized measurements and event records from the Hydropower Research Institute (HRI) to address this issue, as part of the Water Power Technologies Office (WPTO) funded one year multi-lab project. First, we investigated how time series of multiple sensor measurements can be leveraged to identify an event “root cause” as well as to develop an inference (i.e., estimate the unobservable) problem. INL also investigated how individual hydropower components’ reaction or response times vary across the pre-event, during event, and post-event conditions – enabling the hydropower dynamic models to be age-informed. Finally, the impact of clustering multi-sensor time series on short-term vibration prediction is analyzed. INL will present key findings from these analyses and recommend next steps for stakeholder adoption.

13 HYDRO ENERGY↗

DOE OSTI2021

Optimization of In-Field Alpha Spectrometry for Uranium Enrichment Determination in Uranium Hexafluoride

In response to needs identified by the International Atomic Energy Agency (IAEA) research is underway to develop In-Field Alpha Spectrometry (IFAS) as a method to allow IAEA safeguards inspectors to collect samples of uranium hexafluoride (UF6) at processing facilities to assess and verify uranium enrichment. For sample collection, the IFAS method uses Single-Use Destructive Assay (SUDA) samplers, which contain thin zeolite coatings that trap UF6 gas and convert it to the safer, more stable form uranyl fluoride (UO2F2). For alpha spectrometry, the IFAS instrument employs a large area silicon semiconductor transducer to detect and record alpha particle energy-deposition events. Over the past year optimization work has significantly increased the diameter of useful SUDA samples (from 12.7 mm to 48 mm), improved the manufacturability and reproducibility of SUDA samples, increased the area of the IFAS alpha spectrometer sensor from 1.2 cm to 3.1 cm, and improved source positioning within the IFAS. This paper will report on this optimization work, its impacts on IFAS performance, and future plans for IFAS miniaturization, improvements, and testing.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

DOE OSTI2023

Topological Guided Detection of Extreme Wind Phenomena: Implications for Wind Energy

Extreme wind phenomena play a crucial role in the efficient operation of wind farms for renewable energy generation. However, existing detection methods are computationally expensive, limited to specific coordinate. In real-world scenarios, understanding the occurrence of these phenomena over a large area is essential. Therefore, there is a significant demand for a fast and accurate approach to forecast such events. In this paper, we propose a novel method for detecting wind phenomena using topological analysis, leveraging the gradient of wind speed or critical points in a topological framework. By extracting topological features from the wind speed profile within a defined region, we employ topological distance to identify extreme wind phenomena. Our results demonstrate the effectiveness of utilizing topological features derived from regional wind speed profiles. We validate our approach using high-resolution simulations with the Weather Research and Forecasting model (WRF) over a month in the US East Coast.

MATHEMATICS AND COMPUTING,WIND ENERGY↗

DOE OSTI2023

Topological Guided Detection of Extreme Wind Phenomena: Implications for Wind Energy: Preprint

Extreme wind phenomena play a crucial role in the efficient operation of wind farms for renewable energy generation. However, existing detection methods are computationally expensive, limited to specific coordinate. In real-world scenarios, understanding the occurrence of these phenomena over a large area is essential. Therefore, there is a significant demand for a fast and accurate approach to forecast such events. In this paper, we propose a novel method for detecting wind phenomena using topological analysis, leveraging the gradient of wind speed or critical points in a topological framework. By extracting topological features from the wind speed profile within a defined region, we employ topological distance to identify extreme wind phenomena. Our results demonstrate the effectiveness of utilizing topological features derived from regional wind speed profiles. We validate our approach using high-resolution simulations with the Weather Research and Forecasting model (WRF) over a month in the US East Coast.

MATHEMATICS AND COMPUTING,WIND ENERGY↗

DOE OSTI2021

A New Method for High Resolution Surface Change Detection: Data Collection and Validation of Measurements from UAS at the Nevada National Security Site, Nevada, USA

The use of uncrewed aerial systems (UAS) increases the opportunities for detecting surface changes in remote areas and in challenging terrain. Detecting surface topographic changes offers an important constraint for understanding earthquake damage, groundwater depletion, effects of mining, and other events. For these purposes, changes on the order of 5–10 cm are readily detected, but sometimes it is necessary to detect smaller changes. An example is the surface changes that result from underground explosions, which can be as small as 3 cm. Previous studies that described change detection methodologies were generally not aimed at detecting sub-5-cm changes. Additionally, studies focused on high-fidelity accuracy were either computationally modeled or did not fully provide the necessary examples to highlight the usability of these workflows. Detecting changes at this threshold may be critical in certain applications, such as global security research and monitoring for high-consequence natural hazards, including landslides. Here we provide a detailed description of the methodology we used to detect 2–3 cm changes in an important applied research setting—surface changes related to underground explosions. This methodology improves the accuracy of change detection data collection and analysis through the optimization of pre-field planning, surveying, flight operations, and post-processing the collected data, all of which are critical to obtaining the highest output data resolution possible. We applied this methodology to a field study location, collecting 1.4 Tb of images over the course of 30 flights, and location data for 239 ground control points (GCPs). We independently verified changes with orthoimagery, and found that structure-from-motion, software-reported root mean square errors (RMSEs) for both control and check points underestimated the actual error. We found that 3 cm changes are detectable with this methodology, thereby improving our knowledge of a rock’s response to underground explosions.

47 OTHER INSTRUMENTATION↗

DOE OSTI2026

Development of Advanced, Radiation Resistant, Optical-based Detector Technology for Future Experiments.

The primary objective of this project has been to advance the design of high-performance electromagnetic (EM) calorimeters for future particle physics experiments, to identify and measure the timing, position and energy of electrons, positrons and gamma rays, particularly in high-luminosity environments with intense radiation and pileup conditions. To meet such challenges, the proposed research has focused on the development of ultra-compact, radiation-hard calorimeter modules, to provide excellent timing, spatial, and energy resolution. The work aligns with the DOE’s Basic Research Needs (BRN) for High Energy Physics (HEP) Instrumentation and the research team contributes actively to the Coordinating Panel on Advanced Detectors (CPAD) RDC9 calorimetry collaboration in the USA and the European Committee on Future Accelerators (ECFA) DRD-CALO calorimetry collaboration at CERN, the European Laboratory for Particle Physics located in Geneva, Switzerland. The research builds on the RADiCAL (radiation-hard, ultra-compact) modular sampling calorimeter approach, developed by the research team, which employs dense and very bright optical materials such as LYSO:Ce scintillator plates that are interleaved with very dense tungsten plates to minimize detector size while optimizing performance. The modules are comparable in size to a human index finger, dimensionally 14 mm x 14 mm in cross section and 135 mm in length. And despite the small size, the structure is capable of providing excellent timing and energy resolution. This is facilitated through the use of specialized quartz capillaries filled with wavelength-shifting filaments, positioned at various depths along the length of a module, to collect and guide light signals to silicon photomultipliers (SiPMs) which detect and convert the optical signals to electronic signals for analysis. The primary goals of this project have been: (1) Achieve a timing resolution to σ t ≤ 30 ps for high-energy electrons and photons, important for their association with specific events produced in colliding-beam experiments and for the detection of decays-in-flight of long-lived particles. The project has achieved this goal in beam tests of a single RADiCAL module at CERN, during which a timing resolution of σ t = 27 ps was measured for electrons of energy E = 150 GeV. Based upon a mathematical fit to the data measured over a broad energy range from low energy to high energy, a resolution of σ t ≤ 18 ps has been estimated for electrons of very high (TeV) energy. From these measurements and with further expected technical improvements, the timing resolution should reach σ t ≤ 10 ps, important for searches for discovery physics in upcoming and future experiments. (2) Achieve an energy resolution of σ E / E ≤ 10% / $\sqrt{E}$. The project has yet to achieve this goal, but is close to it, having measured a value of σ E / E ≤ 15.9% / $\sqrt{E}$ using a modular array. Ultimately, the resolution goal is expected to be reached by adjustments to material thicknesses within the modules, which will improve the sampling fraction to measure more precisely the shower energy for lower energy particles. The versatility of the modular RADiCAL approach enables the testing of advanced materials, photosensors and electronics, developed in collaboration with CPAD RDC and ECFA DRD-CALO groups. The structure can distinguish electrons, positrons and gamma rays from hadrons and muons and beam-induced backgrounds, making it a valuable tool in a variety of detector environments, including future circular colliders (FCC-ee, FCC-hh) proposed for the European Laboratory for Particle Physics (CERN), the muon-collider proposed for Fermi National Accelerator Laboratory (Fermilab), and searches for new physics in beam-dump, fixed target and forward-physics experiments. And, while designed with particle physics applications in mind, the technologies developed in this project have the potential for application more broadly in particle and nuclear physics, materials science, and medical physics, underscoring the far-reaching potential of this line of instrumentation research and development.

47 OTHER INSTRUMENTATION↗

DOE OSTI2020

Evaluating factors influencing infrasonic signal detection and automatic processing performance utilizing a regional network

Physical and deployment factors that influence infrasound signal detection and assess automatic detection performance for a regional infrasound network of arrays in the Western U.S. are explored using signatures of ground truth (GT) explosions (yields). Despite these repeated known sources, published infrasound event bulletins contain few GT events. Arrays are primarily distributed toward the south-southeast and south-southwest at distances between 84 and 458 km of the source with one array offering azimuthal resolution toward the northeast. Events occurred throughout the spring, summer, and fall of 2012 with the majority occurring during the summer months. Depending upon the array, automatic detection, which utilizes the adaptive F-detector successfully, identifies between 14% and 80% of the GT events, whereas a subsequent analyst review increases successful detection to 24%–90%. Combined background noise quantification, atmospheric propagation analyses, and comparison of spectral amplitudes determine the mechanisms that contribute to missed detections across the network. This analysis provides an estimate of detector performance across the network, as well as a qualitative assessment of conditions that impact infrasound monitoring capabilities. Finally, the mechanisms that lead to missed detections at individual arrays contribute to network-level estimates of detection capabilities and provide a basis for deployment decisions for regional infrasound arrays in areas of interest.

58 GEOSCIENCES↗

DOE OSTI2023

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗