Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “visualization techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Land-use analysis using infrastructure representations and high-resolution flood inundation mapping techniques

In the face of climate change and population growth in coastal regions, land-use analysis efforts are more challenging than ever. Land-use decision-makers in coastal communities are burdened with the difficult choices of where to place new homes versus other assets. While there has been an increased focus on hazard mitigation and disaster resilience in the field of planning, evidence points towards continued development in risk-prone areas including flood zones. Residential development within flood zones specifically continues to be a major issue. To help counter this trend, this study introduces a novel land-use analysis method, coupling topographic flood inundation mapping techniques with digital elevation model (DEM) adaptations. This Topographic Model Scenario Generation workflow can be used by planners early in the land-use decision making process and provides an alternative to high-computational hydraulic models. The analysis also includes the identification of strengths and weaknesses of topographic models' recognition of built infrastructure assets, adding to a limited body of knowledge addressing recommended uses of such models. Levees and canals prove particularly functional in this context while detention ponds less so, likely due to a lack of total water mass accountability. Lastly, we provide a functional demonstration in Southeast Texas to illustrate the workflow's ability to create multiple infrastructure scenarios and visualize their effects across different flood events.

42 ENGINEERING↗
DOE OSTI2021

Magnetism in metastable and annealed compositionally complex alloys

Compositionally complex materials (CCMs) present a potential paradigm shift in the design of magnetic materials. These alloys exhibit long-range structural order coupled with limited or no chemical order. As a result, extreme local environments exist with a large variations in the magnetic energy terms, which can manifest large changes in the magnetic behavior. In the current work, the magnetic properties of (Cr, Mn, Fe, Ni) alloys are presented. These materials were prepared by room-temperature combinatorial sputtering, resulting in a range of compositions with a single bcc structural phase and no chemical ordering. The combinatorial growth technique allows CCMs to be prepared outside of their thermodynamically stable phase, enabling the exploration of otherwise inaccessible order. The mixed ferromagnetic and antiferromagnetic interactions in these alloys causes frustrated magnetic behavior, which results in an extremely low coercivity (<1mT), which increases rapidly at 50 K. At low temperatures, the coercivity achieves values of nearly 500 mT, which is comparable to some high-anisotropy magnetic materials. Further, commensurate with the divergent coercivity is an atypical drop in the temperature dependent magnetization. These effects are explained by a mixed magnetic phase model, consisting of ferro-, antiferro-, and frustrated magnetic regions, and are rationalized by simulations. A machine-learning algorithm is employed to visualize the parameter space and inform the development of subsequent compositions. Annealing the samples at 600 °C orders the sample, more-than doubling the Curie temperature and increasing the saturation magnetization by as much as 5×. Simultaneously, the large coercivities are suppressed, resulting in magnetic behavior that is largely temperature independent over a range of 350 K. The ability to transform from a hard magnet to a soft magnet over a narrow temperature range makes these materials promising for heat-assisted recording technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Three-dimensional imaging of grain boundaries via quantitative fluorescence X-ray tomography analysis

Abstract Three-dimensional visualization of material composition within multiple grains and across complex networks of grain boundaries at nanoscales can provide new insight into the structure evolution and emerging functional properties of the material for diverse applications. Here, using nanoscale scanning X-ray fluorescence tomography, coupled with an advanced self-absorption correction algorithm developed in this work, we analyze the three-dimensional gain distributions and compositions in a Ce 0.8 Gd 0.2 O 2-δ -CoFe 2 O 4 mixed ionic-electronic conductor system with high accuracy and statistical significance. Our systematic investigation reveals an additional emergent phase and uncovers highly intriguing composition stability ranges for the multiple material phases within this system. The presented visualization of composition variations across complex interfaces, supported by our quantitative composition analysis, discloses mechanistic pathways of the diverse phase transformations occurring in the material synthesis, providing insights for the optimization of transport properties in the mixed ionic-electronic conductor system.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Investigation of Correlation Methods for Use in Criticality Safety

Although their adoption by practitioners has been limited, the introduction of similarity indices in criticality safety was a major step forward in reducing the reliance on expert judgement in discerning applicable experiments for the validation of new appliations in criticality safety analyses. Similarity indices have been successfully employed in bias trending and data assimilation techniques, but it is often unclear which acceptance criteria should be used. In their 2004 paper, Broadhead et al. specify the most widely used similarity parameter, ck, as an acceptance cutoff at 0.9. (Broadhead et al., ”Sensitivity and Uncertainty-Based Criticality Safety Validation Techniques,” Nucl. Sci. Eng. 146, 340–366, 2004). Experiments with a ck < 0.9 are often not considered applicable for code validation. This heuristic is based on quantitative studies and engineering judgement, but in some cases, experiments with ck < 0.9 can be used to accurately estimate computational bias. This suggests that further analysis is needed to determine what components of ck are driving applicability and accuracy in bias estimation. For cases in which applicable experiments may not be available (as is the case with UF6 transport canisters), understanding what distinguishes experiments in providing adequate bias estimates aside from just the similarity index is particularly necessary. To further the goal to better interpret ck values, several visualization tools were developed to assist in the investigation of which components of ck are driving applicability.

ck↗
DOE OSTI2021

Five-second STEM dislocation tomography for 300 nm thick specimen assisted by deep-learning-based noise filtering

Scanning transmission electron microscopy (STEM) is suitable for visualizing the inside of a relatively thick specimen than the conventional transmission electron microscopy, whose resolution is limited by the chromatic aberration of image forming lenses, and thus, the STEM mode has been employed frequently for computed electron tomography based three-dimensional (3D) structural characterization and combined with analytical methods such as annular dark field imaging or spectroscopies. However, the image quality of STEM is severely suffered by noise or artifacts especially when rapid imaging, in the order of millisecond per frame or faster, is pursued. Here we demonstrate a deep-learning-assisted rapid STEM tomography, which visualizes 3D dislocation arrangement only within five-second acquisition of all the tilt-series images even in a 300 nm thick steel specimen. The developed method offers a new platform for various in situ or operando 3D microanalyses in which dealing with relatively thick specimens or covering media like liquid cells are required.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Depth-Resolved X-Ray Nanoimaging of Coherent and Incoherent Energy Transport in Silicon Carbide

Understanding lattice dynamics is crucial for optimizing the process of creating functional structures, such as laser writing of color-center defects. However, existing structural probes have difficulty measuring structural dynamics with submicrometer depth sensitivity. Here, in this study, a depth-resolved ultrafast X-ray nanodiffraction technique is developed to track the lattice dynamics of silicon carbide (SiC) in three dimensions. Upon laser excitation of an aluminum layer that acts as a heat and strain transducer, a specular Bragg peak of SiC shows an overall increase in the X-ray diffraction intensity rather than a peak shift. The relaxation dynamics of the increased intensity are significantly different when probed on and off the Bragg peak. The fast subnanosecond relaxation probed at the maximum of the Bragg peak is a result of the propagation of a coherent strain wave along the depth direction, while a slow relaxation probed at the wings of the Bragg peak reflects a localized incoherent lattice heating. To further visualize these processes, spatiotemporal maps were obtained by scanning the relative position and delay between the laser pump and X-ray probe beams, which capture the propagation of the strain wave, as well as a stationary structural distortion close to the aluminum/SiC interface. These depth-resolved structural measurements disentangle energy dissipation mechanisms in laser-excited SiC, and they open opportunities for finer control of, for example, the formation of optically addressable defect complexes central to quantum information applications.

X-ray nanodiffraction↗
DOE OSTI2023

Elucidating a dissolution–deposition reaction mechanism by multimodal synchrotron X-ray characterization in aqueous Zn/MnO 2 batteries

Aqueous Zn/MnO 2 batteries with their environmental sustainability and competitive cost, are becoming a promising, safe alternative for grid-scale electrochemical energy storage. Here, presented as a promising design principle to deliver a higher theoretical capacity, this work offers fundamental understanding of the dissolution–deposition mechanism of Zn/β-MnO 2 . A multimodal synchrotron characterization approach including three operando X-ray techniques (powder diffraction, absorption spectroscopy, and fluorescence microscopy) is coupled with elementally resolved synchrotron X-ray nano-tomography. Together they provide a direct correlation between structural evolution, reaction chemistry, and 3D morphological changes. Operando synchrotron X-ray diffraction and spectroscopy show a crystalline-to-amorphous phase transition. Quantitative modeling of the operando data by Rietveld refinement for X-ray diffraction and multivariate curve resolution (MCR) for X-ray absorption spectroscopy are used in a complementary fashion to track the structural and chemical transitions of both the long-range (crystalline phases) and short-range (including amorphous phases) ordering upon cycling. Scanning X-ray microscopy and full-field nano-tomography visualizes the morphology of electrodes at different electrochemical states with elemental sensitivity to spatially resolve the formation of the Zn- and Mn-containing phases. Overall, this work critically indicates that for Zn/MnO 2 aqueous batteries, the reaction pathways involving Zn–Mn complex formation upon cycling become independent of the polymorphs of the initial electrode and sheds light on the interplay among structural, chemical, and morphological evolution for electrochemically driven phase transitions.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Development of an end state vision to implement digital monitoring in nuclear plants

Transitioning from an onsite Maintenance & Diagnostics Center to cloud-based services offers many new opportunities with computing power and storage, but also new challenges in terms of networking and security. This report will cover everything required for that transition including data processing and uploading to cloud services, feature selection, model creation, and result visualization for decision making. Although there are several other cloud-based services (e.g. Amazon Web Services and Google Cloud), this report explores Microsoft Azure to simplify nomenclature and maintain a consistent focus. Many of the services offered by Microsoft Azure are also available in the other cloud-based services, and their differences have been recorded in other literature. The Azure services most important to a nuclear power plant including networking & security, storage & databases, and Artificial Intelligence (AI) are reviewed here. Networking covers all aspects related to communication to Azure resources including security, privacy, and redundancy. Storage & databases includes data storage, upgrading, patching, backups, and monitoring. The AI services allows the user access to the machine learning (ML) techniques developed with Azure including automated ML, anomaly detection, computer vision, and natural language processing. This report summaries the features, capabilities, and challenges when using cloud-based services in a user-friendly manner.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Ghost imaging second harmonic generation microscopy

Second harmonic generation (SHG) microscopy is useful for visualizing interfaces and sub-structures within a wide range of materials due to the propensity for SHG to occur in non-centrosymmetric environments. However, since SHG is a nonlinear process generally necessitating small focal sizes for higher peak powers, a raster scanning approach is usually needed to build an SHG image over a significant sample size. While raster scanning is effective, there is a cost in terms of the time needed to acquire the image and, also, some materials cannot withstand the higher optical intensities within the small focal volume. As such, we describe a SHG microscopy approach based on ghost imaging (GI), which enables imaging data to be collected in parallel rather than sequentially as in raster scanning techniques. We experimentally demonstrate the approach and combine GI-SHG with compressive sensing to make further substantial gains in reducing the amount of sampling required for image reconstruction. Furthermore, GI-SHG is shown to have significant advantages for imaging in highly scattering environments, partly because GI is a background-free approach requiring spatial correlations between photons that travel two paths, with one path entirely devoid of sample interaction. This basic property of GI means that only the photons that travel unimpeded through the sample preserve the spatial correlations needed to reconstruct the image, while more scattered photons do not contribute to the overall GI signal. Finally, we compare the image quality and sampling properties of three different reconstruction algorithms used for compressive sensing.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Disentangling Coexisting Structural Order Through Phase Lock-In Analysis of Atomic-Resolution STEM Data

As a real-space technique, atomic-resolution STEM imaging contains both amplitude and geometric phase information about structural order in materials, with the latter encoding important information about local variations and heterogeneities present in crystalline lattices. Such phase information can be extracted using geometric phase analysis (GPA), a method which has generally focused on spatially mapping elastic strain. Here we demonstrate an alternative phase demodulation technique and its application to reveal complex structural phenomena in correlated quantum materials. As with other methods of image phase analysis, the phase lock-in approach can be implemented to extract detailed information about structural order and disorder, including dislocations and compound defects in crystals. Extending the application of this phase analysis to Fourier components that encode periodic modulations of the crystalline lattice, such as superlattice or secondary frequency peaks, we extract the behavior of multiple distinct order parameters within the same image, yielding insights into not only the crystalline heterogeneity but also subtle emergent order parameters such as antipolar displacements. When applied to atomic-resolution images spanning large (~0.5 × 0.5 μm 2 ) fields of view, this approach enables vivid visualizations of the spatial interplay between various structural orders in novel materials.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Raman Microscopy Analysis of Wyoming CarbonSAFE Pilot Well Thin Sections for Mineralogy and Organic Matter Characterization

Scanning confocal Raman microspectroscopy (RMS) was used to analyze thin sections for inorganic mineral content and for evaluation of organic material (OM). Thin sections were selected from a stratigraphic test well core in the Powder River Basin, Wyoming under the Wyoming CarbonSAFE project. The well was analyzed with a variety of rock and fluid characterization techniques to determine the feasibility of a commercial-scale CO2 storage site. RMS complements other analyses, including traditional petrography, SEM, porosity and permeability, and XRD For this aspect of the study, RMS is especially important in the evaluation of OM in sealing lithologies. At prospective geologic CO2 storage sites, it is imperative to assess the unconventional oil and gas potential of seals to ensure that any future development would not compromise the integrity of the seals. Whole-slide mineralogical surveys were performed on thin sections from various shale and sandstone formations. Surveys were analyzed with Direct Classical Least Squares to identify and quantify minerals. The location and concentration of minerals was color-coded and overlaid on optical images for visualization of the distribution of minerals. Dense hyperspectral Raman mapping of OM was performed on five thin sections. Eleven spectral parameters diagnostic of organic type and thermal maturity were used to train a Partial Least Squares (PLS) calibration against a set of artificially matured samples spanning the pre- to mid-oil window. The PLS was applied to the study set and a post-mature set. Additionally, the PLS was applied to each point in hyperspectral maps for visualization of trends in maturity across sample sets and discrimination of organic matter types within a given map. In inorganic surveys on thin sections, a total of 14 unique inorganic minerals were identified in Raman spectra including quartz, dolomite, calcite, hematite and anhydrite. Shale thin sections tended to be dominated by organic material. OM was often observed mixed with inorganic minerals. Sand- and mudstones were dominated by inorganic minerals. The PLS extrapolated the post-mature set to reflectances >1.2%. The study set ranged from very immature to postmature in the median of map fit-peak parameters. However, point maturity maps indicate that matrix OM in all study samples is immature and that discreet organic particles selected for mapping, which may be inertinites, bias medians towards more-mature. The work demonstrates the capabilities of RMS to perform both whole-slide mineralogy and OM analysis with applications to formation evaluation in oil & gas, carbon sequestration and mining. Here, analysis of sealing formations in the well indicates high levels of immature organic matter that would not be a viable target for future oil production that could compromise the CO2 storage site. The work brings together diverse disciplines from geology and petrography to analytical chemistry, big data and microscopy.

Myers, Grant↗
DOE OSTI2020

Deep Learning for Intelligent Bubble Size Detection in the Spallation Neutron Source Visual Target

The Spallation Neutron Source (SNS) at Oak Ridge National Laboratory (ORNL) will undergo proton power upgrade (PPU), increasing the proton beam power from 1.4 MW to 2.8 MW. From 2.8 MW, 2.0 MW will go to the current First Target Station and the rest will go to the future Second Target Station (STS). The First Target Station uses a liquid mercury target that is contained in a 316L stainless steel vessel. The proton beam is pulsed at 60 Hz, with a pulse of about 0.7μs. When the proton beam hits the target, the intense energy deposition leads to a rapid rise in temperature in the mercury. This temperature rise creates pressure waves that propagate through the mercury and cause cavitation erosion. The power upgrade will cause stronger pressure waves that will further increase damage because of cavitation. Injecting small helium bubbles in the mercury has been an efficient method of mitigating the pressure wave at 1.4 MW. However, at higher power, additional mitigation is necessary. Therefore, the 2 MW target vessel will be equipped with swirl bubblers and an additional gas injection port near the nose to inject more gas in the target. To develop a gas injection strategy and design, flow visualization in water with a transparent prototypical target (“visual target”) was performed. Bubble sizes and their spatial distribution in the flow loop are crucial to understanding the effectiveness of the bubbles in mitigating pressure waves. Bubbles were generated in the visual target under varied conditions of input pressures with helium and air. Images were captured using a high-speed camera at varied frame rates at different positions away from the swirl bubbler and different depths in the flow loop under varying lighting conditions. Initially, methods such as circular Hough transforms were applied after a series of images processing to obtain a general distribution of bubble sizes. Bubbles smaller than 500 μm are preferred to effectively mitigate the effect of pressure waves, which demands an accurate bubble detection and sizing system. Intelligent detection and identification of bubble sizes alleviate misdetection and improves accuracies. Employing neural networks, intelligent detection of bubble sizes and their distribution was developed and provides a robust alternative to traditional techniques. Human intervention was employed to label in-focus and out-of-focus bubbles in the set of training images. An object detection network using a pretrained convolutional neural network was created that extracted the features from the training images. Data augmentation was used to improve network accuracy through a random transformation of the original data.

Rasheed, Fayaz↗
DOE OSTI2020

Direct Visualization of Current-Stimulated Oxygen Migration in YBa 2 Cu 3 O 7–δ Thin Films

The past years have witnessed major advancements in all-electrical doping control on cuprates. In the vast majority of cases, the tuning of charge carrier density has been achieved via electric field effect by means of either a ferroelectric polarization or using a dielectric or electrolyte gating. Unfortunately, these approaches are constrained to rather thin superconducting layers and require large electric fields in order to ensure sizable carrier modulations. In this work, we focus on the investigation of oxygen doping in an extended region through current-stimulated oxygen migration in YBa 2 Cu 3 O 7–δ superconducting bridges. The underlying methodology is rather simple and avoids sophisticated nanofabrication process steps and complex electronics. Here, a patterned multiterminal transport bridge configuration allows us to electrically assess the directional counterflow of oxygen atoms and vacancies. Importantly, the emerging propagating front of current-dependent doping δ is probed in situ by optical microscopy and scanning electron microscopy. The resulting imaging techniques, together with photoinduced conductivity and Raman scattering investigations, reveal an inhomogeneous oxygen vacancy distribution with a controllable propagation speed permitting us to estimate the oxygen diffusivity. These findings provide direct evidence that the microscopic mechanism at play in electrical doping of cuprates involves diffusion of oxygen atoms with the applied current. The resulting fine control of the oxygen content would permit a systematic study of complex phase diagrams and the design of electrically addressable devices.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Examination of Intergranular Crack on SC0234

The microstructure of the SC0234 weld was evaluated per DA-SPEC-01067, ''General Purpose Heat Source (GPHS) Plutonium-238 Dioxide Fueled Clad Product Specification'' and it was determined that the weld microstructure conformed to the quality requirements. During the examination of weld cross section, the metallographic analysis revealed the presence of a surface crack. Prior visual examination per RPS-TA55-303 had not revealed the crack, nor did the surface photography performed per RPS-TA55-302. The presence of the crack does not violate any of the product specifications called out on DA-SPEC-01067. Although there is a well-documented history of weld hot crack susceptibility in the early development of the DOP-26 iridium alloy, a surface crack such as this had not been documented during welding at Los Alamos. In the interest of better understanding the cause; the product engineer/program recommended further examination of the crack. This report discusses the examination of the crack found in the weld cross section of simulant clad SC0234. Metallography, Scanning Electron Microscopy (SEM) and Energy Dispersive Spectroscopy techniques (EDS) where used in the analysis of the crack and the surrounding weld microstructure.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Installation protocol for charge transfer dissociation mass spectrometry on ion trapping mass spectrometers

Abstract Rationale Charge transfer dissociation (CTD) is a novel fragmentation technique that demonstrates enhanced structural characterization for a wide variety of molecules compared to standard fragmentation techniques like collision‐induced dissociation (CID). Alternative fragmentation techniques, such as electron transfer dissociation, electron capture dissociation, and ultraviolet photodissociation, also overcome many of the shortfalls of CID, but none of them are a silver bullet that can adequately characterize a wide variety of structures and charge states of target compounds. Given the diversity of structural classes and their occasional obstinance towards certain activation techniques, alternative fragmentation techniques are required that rely on novel or alternative modes of activation. Methods Herein, we present a step‐by‐step protocol for the installation of CTD on a quadrupole ion trap mass spectrometer and best practices for optimizing the signal‐to‐noise ratio and acquisition times for CTD mass spectra. Results In addition to two CTD installations in the Jackson laboratory, CTD has also been installed, and is currently in operation, on two 3D ion trap mass spectrometers in France: one in the laboratory of Dr. David Ropartz and Dr. Hélène Rogneaux at INRAE in Nantes, and the other in the laboratory of Dr. Jean‐Yves Salpin at Université d'Évry Val‐d'Essonne, part of the Paris‐Saclay University system. Conclusions Here, we provide a visual protocol to help others accomplish the instrument modification.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Synchrotron-based diffraction-enhanced imaging and diffraction-enhanced imaging combined with CT X-ray imaging systems to image seeds at 30 keV

Utilized the upgraded Synchrotron-based non-destructive Diffraction-enhanced imaging and Diffraction-enhanced imaging coupled with CT X-ray imaging systems to image the chickpea seeds, to enhance the contrast in plant root architecture, visibility of fine structures of root architecture growth and some aspects of physiology at acceptable level. DEI-CT images were acquired with 30 keV synchrotron X-rays. A series of DEI-CT slices were assembled together, to form a 3D data set. DEI-CT images explored more structural information and morphology. Noticed detailed anatomical, physiological observations, and contrast mechanisms. Furthermore, with these systems, some of the complex plant traits, root morphology, growth of laterals and subsequent laterals can be visualized directly.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Correlative single-cell hard X-ray computed tomography and X-ray fluorescence imaging

Abstract X-ray computed tomography (XCT) and X-ray fluorescence (XRF) imaging are two non-invasive imaging techniques to study cellular structures and chemical element distributions, respectively. However, correlative X-ray computed tomography and fluorescence imaging for the same cell have yet to be routinely realized due to challenges in sample preparation and X-ray radiation damage. Here we report an integrated experimental and computational workflow for achieving correlative multi-modality X-ray imaging of a single cell. The method consists of the preparation of radiation-resistant single-cell samples using live-cell imaging-assisted chemical fixation and freeze-drying procedures, targeting and labeling cells for correlative XCT and XRF measurement, and computational reconstruction of the correlative and multi-modality images. With XCT, cellular structures including the overall structure and intracellular organelles are visualized, while XRF imaging reveals the distribution of multiple chemical elements within the same cell. Our correlative method demonstrates the feasibility and broad applicability of using X-rays to understand cellular structures and the roles of chemical elements and related proteins in signaling and other biological processes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Hindsight logging for model training

In modern Machine Learning, model training is an iterative, experimental process that can consume enormous computation resources and developer time. To aid in that process, experienced model developers log and visualize program variables during training runs. Exhaustive logging of all variables is infeasible, so developers are left to choose between slowing down training via extensive conservative logging, or letting training run fast via minimalist optimistic logging that may omit key information. As a compromise, optimistic logging can be accompanied by program checkpoints; this allows developers to add log statements post-hoc, and "replay" desired log statements from checkpoint---a process we refer to as hindsight logging. Unfortunately, hindsight logging raises tricky problems in data management and software engineering. Done poorly, hindsight logging can waste resources and generate technical debt embodied in multiple variants of training code. In this paper, we present methodologies for efficient and effective logging practices for model training, with a focus on techniques for hindsight logging. Our goal is for experienced model developers to learn and adopt these practices. To make this easier, we provide an open-source suite of tools for Fast Low-Overhead Recovery (flor) that embodies our design across three tasks: (i) efficient background logging in Python, (ii) adaptive periodic checkpointing, and (iii) an instrumentation library that codifies hindsight logging for efficient and automatic record-replay of model-training. Model developers can use each flor tool separately as they see fit, or they can use flor in hands-free mode, entrusting it to instrument their code end-to-end for efficient record-replay. Our solutions leverage techniques from physiological transaction logs and recovery in database systems. Evaluations on modern ML benchmarks demonstrate that flor can produce fast checkpointing with small user-specifiable overheads (e.g. 7%), and still provide hindsight log replay times orders of magnitude faster than restarting training from scratch.

Computer Science↗