Hades: A Context-Aware Active Storage Framework for Accelerating Large-Scale Data Analysis
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Performing a variety of numerical computations efficiently and, at the same time, in a portable fashion requires both an overarching design followed by a number of implementation strategies. All of these are exemplified below as we present transitioning the PLASMA numerical library from relying on dependence-driven large tasks to achieving utilization of fine grain tasking and offload to hardware accelerators while keeping its core dependence sets: OpenMP source code pragmas and runtime for most system-level functionality and basic low-level numerical kernels provided directly by hardware vendors or open source projects with vendor contributions. We also present new algorithmic methods and their efficient parallel implementations including fine grained tasking for eigen-spectrum slicing and offload for mixed-precision eigenvalue refinement. We provide performance, scaling, and numerical results showing sizable gains over the available solutions from either the open source and vendor-provided packages.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Test variables discussed graphically include the effects of: (1) ambient temperature; (2) depth of the discharge; (3) discharge rate; (4) percent of recgargel (5) KOH concentration; (6) KOH volume; (7) charge rate and (8) negative per charge.
Teflon samples returned from the Solar Max Satellite (SMS) suffered noticeable damage such as cracking and yellowing. This is in contrast to teflon exposed aboard STS-5 and STS-8 which showed no detectable changes. Selected teflon tape samples from SMS were studied to evaluate the extent and mechanism of degradation. ESCA studies revealed that these teflon samples contain hydrocarbon segments which were susceptible to oxygen atom degradation. Mechanical measurements also showed bulk property changes as a result of LEO exposure. A molecular model of material and energetic oxygen atom interaction was proposed. SMS data and the importance of developing correlation between accelerated exposure data from STS and ground-based testing and real time data will be presented.
Explore the source record for details and available documents.
This work develops new techniques within Horovod, a generic communication library supporting data parallel training across deep learning frameworks. In particular, we improve the Horovod control plane by implementing a new coordination scheme that takes advantage of the characteristics of the typical data parallel training paradigm, namely the repeated execution of collectives on the gradients of a fixed set of tensors. Using a caching strategy, we execute Horovod’s existing coordinator-worker logic only once during a typical training run, replacing it with a more efficient decentralized orchestration strategy using the cached data and a global intersection of a bitvector for the remaining training duration. Next, we introduce a feature for end users to explicitly group collective operations, enabling finer grained control over the communication buffer sizes. To evaluate our proposed strategies, we conduct experiments on a world-class supercomputer — Summit. We compare our proposals to Horovod’s original design and observe 2x performance improvement at a scale of 6000 GPUs; we also compare them against tf.distribute and torch.DDP and achieve 12% better and comparable performance, respectively, using up to 1536 GPUs; we compare our solution against BytePS in typical HPC settings and achieve about 20% better performance on a scale of 768 GPUs. Finally, we test our strategies on a scientific application (STEMDL) using up to 27,600 GPUs (the entire Summit) and show that we achieve a near-linear scaling of 0.93 with a sustained performance of 1.54 exaflops (with standard error +- 0.02) in FP16 precision.
Explore the source record for details and available documents.
An electronic communication-and-control system has been developed as a prototype of advanced means of automatically modifying the switching of traffic lights to give priority to emergency vehicles. This system could be used alternatively or in addition to other emergency traffic-light-preemption systems, including a variety of systems now in use as well as two proposed systems described in "Systems Would Preempt Traffic Lights for Emergency Vehicles" (NPO-30573), NASA Tech Briefs, Vol. 28, No. 10 (October 2004), page 36. Unlike those prior systems that depend on detection of sounds and/or lights emitted by emergency vehicles, this system is not subject to severe range limitations. This system can be retrofitted into any pre-existing traffic-light-control system, without need to modify that system other than to make a minimal number of wire connections between the two systems. This system comprises several subsystems, including a transponder and interface circuitry on each emergency vehicle, a monitoring and control unit at each intersection equipped with traffic lights, and a wide-area two-way radio communication network that connects the emergency vehicles and intersection units. Computers in the various intersections and vehicle units run special-purpose software that implements the traffic- light-preemption scheme. The operations of the intersection and vehicle units are synchronized by use of Global Positioning System (GPS) timing signals. The transponder in each vehicle estimates its own position and velocity by use of GPS signals, deductive ("dead") reckoning, data from the onboard diagnostic (OBD) computer of the vehicle, and/or triangulation of beacon signals. When the operator of an emergency vehicle turns on its flashing lights and sirens in response to a request for an emergency response, the transponder unit goes into action, reading the OBD data to determine speed and acceleration, and reading and gathering further navigational data as described above. The position, velocity, and acceleration data are combined with vehicle-identification data in a prescribed format, and the resulting set of data is transmitted to the intersections within communication range of the transponder.
Recent observations of GeV /TeV photon emission from several X-ray binaries have sparked a renewed interest in these objects as galactic particle accelerators. In spite of the available multi-wavelength data, their acceleration mechanisms are not determined, and the nature of the accelerated particles (hadrons or leptons) is unknown. While much evidence favors leptonic emission, it is very likely that a hadronic component is also accelerated in the jets of these binary systems. The observation of neutrino emission would be clear evidence for the presence of a hadronic component in the outflow of these sources. In this paper we look for periodic neutrino emission from binary systems. Such modulation, observed in the photon flux, would be caused by the geometry of these systems. The results of two searches are presented that differ in the treatment of the spectral shape and phase of the emission. The 'generic' search allows parameters to vary freely and best fit values, in a 'model-dependent' search, predictions are used to constrain these parameters. We use the IceCube data taken from May 31, 2007 to April 5, 2008 with its 22-string configuration, and from April 5, 2008 and May 20, 2009 with its 40-string configuration. For the generic search and the 40 string sample, we find that the most significant source in the catalog of 7 binary stars is Cygnus X-3 with a 1.8% probability after trials (2.10" sigma one-sided) of being produced by statistical fluctuations of the background. The model-dependent method tested a range of system geometries - the inclination and the massive star's disk size - for LS I+61 deg 303, no significant excess was found.
J-2X engine testing on the A-2 test stand at the NASA John C. Stennis Space Center (SSC) has recently concluded. As part of that test campaign, the engine was operated at lower power levels in support of expanding the use of J-2X to other missions. However, the A-2 diffuser was not designed for engine testing at the proposed low power levels. To evaluate the risk of damage to the diffuser, computer simulations were created of the rocket engine exhaust plume inside the 50ft long, water-cooled, altitude-simulating diffuser. The simulations predicted that low power level testing would cause the plume to oscillate in the lower sections of the diffuser. This can possibly cause excessive vibrations, stress, and heat transfer from the plume to the diffuser walls. To understand and assess the performance of the diffuser during low power level engine testing, nine accelerometers and four strain gages were installed around the outer surface of the diffuser. The added instrumentation also allowed for the verification of the rocket exhaust plume computational model. Prior to engine hot-fire testing, a diffuser water-flow test was conducted to verify the proper operation of the newly installed instrumentation. Subsequently, two J-2X engine hot-fire tests were completed. Hot-Fire Test 1 was 11.5 seconds in duration, and accelerometer and strain data verified that the rocket engine plume oscillated in the lower sections of the diffuser. The accelerometers showed very different results dependent upon location. The diffuser consists of four sections, with Section 1 being closest to the engine nozzle and Section 4 being farthest from the engine nozzle. Section 1 accelerometers showed increased amplitudes at startup and shutdown, but low amplitudes while the diffuser was started. Section 3 accelerometers showed the opposite results with near zero G amplitudes prior to and after diffuser start and peak amplitudes to +/- 100G while the diffuser was started. Hot-Fire Test 1 strain gages showed different data dependent on section. Section 1 strains were small, and were in the range of 50 to 150 microstrain, which would result in stresses from 1.45 to 4.35 ksi. The yield stress of the material, A-285 Grade C Steel, is 29.7 ksi. Section 4 strain gages showed much higher values with strains peaking at 1600 microstrain. This strain corresponds to a stress of 46.41 ksi, which is in excess of the yield stress, but below the ultimate stress of 55 to 75 ksi. The decreased accelerations and strain in Section 1, and the increased accelerations and strain in Sections 3 and 4 verified the computer simulation prediction of increased plume oscillations in the lower sections of the diffuser. Hot-Fire Test 2 ran for a duration of 125 seconds. The engine operated at a slightly higher power level than Hot-Fire Test 1 for the initial 35 seconds of the test. After 35 seconds the power level was lowered to Hot-Fire Test 1 levels. The acceleration and strain data for Hot-Fire Test 2 was similar during the initial part of the test. However, just prior to the engine being lowered to the Hot-Fire Test 1 power level, the strain gage data in Section 4 showed a large decrease to strains near zero microstrain from their peak at 1500 microstrain. Future work includes further strain and acceleration data analysis and evaluation.
A Technology Computer Aided Design (TCAD) simulation-based method is developed to evaluate whether derating of high-energy heavy-ion accelerator test data bounds the risk for single-event gate rupture (SEGR) from much higher energy on-orbit ions for a mission linear energy transfer (LET) requirement. It is shown that a typical derating factor of 0.75 applied to a single-event effect (SEE) response curve defined by high-energy accelerator SEGR test data provides reasonable on-orbit hardness assurance, although in a high-voltage power MOSFET, it did not bound the risk of failure.
The author has identified the following significant results. The applications of ERTS-1 imagery for geological fracture mapping regardless of season has been repeatedly confirmed. The enhancement provided by a differential cover of snow increases the number and length of fracture-lineaments which can be detected with ERTS-1 data and accelerates the fracture mapping process for a variety of practical applications. The geological mapping benefits of the program will be realized in geographic areas where data are most needed - complex glaciated terrain and areas of deep residual soils. ERTS-1 derived fracture-lineament maps which provide detail well in excess of existing geological maps are not available in the Massachusetts-Connecticut area. The large quantity of new data provided by ERTS-1 may accelerate and improve field mapping now in progress in the area. Numerous other user groups have requested data on the techniques. This represents a major change in operating philosophy for groups who to data judged that snow obscured geological detail.
Inelastic neutron scattering (INS) experiments utilizing modern time-of-flight spectrometers enable the comprehensive mapping of the energy (E)- and momentum (Q)-resolved dynamical structure factor of single crystals, probing both the lattice and magnetic excitations. Yet, the large size and complexity of four-dimensional INS data are challenging current analysis workflows, often resulting in an underutilization of the measured information. To help address this issue, this paper introduces new software interfaced with the Mantid framework, pathSQE, designed to streamline the processing, analysis and interpretation of 4D single-crystal INS data. By automating key tasks such as 1D/2D slicing, symmetrization, Brillouin zone folding, data visualization, prioritization and filtering, and comparisons with simulations, pathSQE facilitates and accelerates INS data analysis workflows. Here, this paper outlines the features and implementation and provides several illustrations of the use of pathSQE on data collected on single crystals using direct-geometry time-of-flight spectrometers at the Spallation Neutron Source, including Ge, FeSi, MnO and SnS single-crystal measurements on the ARCS, HYSPEC and CNCS neutron spectrometers. Beyond streamlining post-experiment data processing, pathSQE establishes an automated and modular processing pipeline that could support future real-time experiment steering.
In addressing one of the fundamental problems in solar physics establishing the mechanism(s) responsible for the solar wind acceleration and the corona heating - it is essential to have a reliable knowledge of the heliocentric radial dependence of the solar wind properties. Adequate data are available for small solar distances R less than 4 R(solar mass) from coronal white light and EUV observations and at distances R greater than 60 R(solar mass) from in situ measurements. One of the few methods available to fill in the gap between these boundaries is the radio scintillation technique. Taking the example of the solar wind velocity, the most reliable such measurements are obtained when phase fluctuation observations of scattered radio waves, which are not susceptible to saturation effects, are recorded at two or more widely-spaced ground stations. Two extensive observation campaigns of this type were carried out with the Venus-orbiting satellites Venera 10 in 1976 and Venera 15/16 in 1984. The observations were performed over the course of three months near superior conjunction at solar offset distances R approximately 6-80 R(solar mass). The main results from the subsequent analysis of these data are: (1) velocities vary between 250 and 380 km s(exp -1) for R greater than 20 R(solar mass), agreeing with similar measurements using natural sources (IPS); (2) velocities derived from two-station phase fluctuation observations varv between 70 and 120 km s(exp -1) for R less than 12 R(solar mass), i.e. values substantially lower than those derived from conventional IPS data; and (3) it is suggested that the different velocity profiles derived from the two data sets at small R may be due to the effects of magnetosonic and Alfvenic waves on radio wave scattering. Further analysis of additional radio sounding data should help resolve the apparent discrepancy.
The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.
This white paper presents a multi-year initiative to develop practical, secure, and scalable digital twin capabilities for engineered systems in aging and surveillance contexts—an approach pioneered at the National Nuclear Security Administration (NNSA) Lawrence Livermore National Laboratory (LLNL) that maps directly onto the needs and ambitions of the Navy for ship- and fleet-level digital twins. LLNL’s work in building part- and process-level digital twins for advanced manufacturing, with a vision to scale up to entire factory floors and, ultimately, enterprise-wide digital twins, offers an adaptable pathway for the Navy as it seeks to modernize lifecycle management, readiness, and predictive maintenance across ships and fleets. For our application, we integrate physics-based modeling with automated data ingestion, processing, and AI-driven calibration, creating hybrid models that are both interpretable and data responsive. We modernized legacy workflows, established centralized data infrastructure, automated experimental pipelines, and demonstrated end-to-end coupling of accelerated aging data with finite element simulations via optimization and surrogate modeling. The result is a generalizable framework that supports part-level digital twins today and lays the groundwork for future system-level twins suitable for Navy applications.