SEARCH · Engineering Papers

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,675 records · Page 93

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Effectively Using Remote I/O For Work Composition in Distributed Workflows

Distributed scientific workflows are becoming more important with the interest in incorporating AI into their loops. A critical programming and performance question is how to compose workflow tasks when data is produced on one system but must be consumed on another. Since the dominant technique is composition with remote I/O, this paper explores its performance expectations. We describe BigFlowSim, a workflow I/O simulator that captures key implementation choices for remote I/O, including intensity, reuse, locality, access pattern, and data movement.With BigFlowSim, we generate a synthetic benchmark. We quantify the effects of each parameter with a performance sensitivity study. We explain trends in terms of data movement reduction and show that, under certain conditions, it is possible to establish a total order among most parameters. We apply these insights to a high energy physics workflow, Belle II Monte Carlo and simulate several I/O optimizations. Speedups range from 5% to 2×, without changing compute time.

Friese, Ryan D.↗

DOE OSTI2017

Epidemic failure detection and consensus for extreme parallelism

Future extreme-scale high-performance computing systems will be required to work under frequent component failures. The MPI Forum s User Level Failure Mitigation proposal has introduced an operation, MPI Comm shrink, to synchronize the alive processes on the list of failed processes, so that applications can continue to execute even in the presence of failures by adopting algorithm-based fault tolerance techniques. This MPI Comm shrink operation requires a failure detection and consensus algorithm. This paper presents three novel failure detection and consensus algorithms using Gossiping. The proposed algorithms were implemented and tested using the Extreme-scale Simulator. The results show that in all algorithms the number of Gossip cycles to achieve global consensus scales logarithmically with system size. The second algorithm also shows better scalability in terms of memory and network bandwidth usage and a perfect synchronization in achieving global consensus. The third approach is a three-phase distributed failure detection and consensus algorithm and provides consistency guarantees even in very large and extreme-scale systems while at the same time being memory and bandwidth efficient.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Evaluating Emulation-based Models of Distributed Computing Systems

Emulation-based models of distributed computing systems are collections of virtual ma- chines, virtual networks, and other emulation components configured to stand in for oper- ational systems when performing experimental science, training, analysis of design alterna- tives, test and evaluation, or idea generation. As with any tool, we should carefully evaluate whether our uses of emulation-based models are appropriate and justified. Otherwise, we run the risk of using a model incorrectly and creating meaningless results. The variety of uses of emulation-based models each have their own goals and deserve thoughtful evaluation. In this paper, we enumerate some of these uses and describe approaches that one can take to build an evidence-based case that a use of an emulation-based model is credible. Predictive uses of emulation-based models, where we expect a model to tell us something true about the real world, set the bar especially high and the principal evaluation method, called validation , is comensurately rigorous. We spend the majority of our time describing and demonstrating the validation of a simple predictive model using a well-established methodology inherited from decades of development in the compuational science and engineering community.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Leveraging FPGA Advantages for Quicker Data Processing for LBNF

The Long Baseline Neutrino Facility (LBNF) will deliver a 2.4 MW muon neutrino beam from Fermilab to the Deep Underground Neutrino Experiment (DUNE), requiring unprecedented precision in beamline alignment to achieve DUNE's neutrino oscillation measurement goals. Vertical misalignments of beamline components as small as 0.5 mm can contribute 6-7\% uncertainty in predicted neutrino flux, necessitating sub-0.1 mm alignment monitoring capabilities. The Horn Location Sensor (HLS) system employs frequency sweep interferometry (FSI) in a distributed hydrostatic leveling network to achieve the required precision under harsh radiation conditions up to 5000 kRad/year. Traditional FSI implementations suffer from laser sweep nonlinearities that degrade resolution and require computationally intensive post-processing corrections using gas reference cells. This work presents a real-time FPGA-based implementation of the HLS data acquisition and processing system using a sweep tracker interferometer for dynamic sweep linearization. The system utilizes a PYNQ-Z2 FPGA with programmable logic implementing parallel 16k-point FFT processing across four channels, synchronized by the sweep tracker signal to eliminate post-processing requirements. Spectral performance testing demonstrates significant improvements in peak sharpness compared to traditional fixed-frequency digitization. The FPGA implementation enables real-time displacement monitoring with processing speeds orders of magnitude faster than software-based approaches, essential for the operational requirements of LBNF's eventual distributed sensor network. This advancement in real-time FSI processing directly supports DUNE's precision neutrino physics program by providing the rapid feedback necessary for maintaining stringent beamline alignment tolerances during high-power beam operations.

Rossel, Jacob↗

DOE OSTI2025

Real-Time FPGA Implementation For Frequency Sweep Interferometry In The LBNF Complex

Rossel, A. Jacob [Fermilab; Unlisted]↗

DOE OSTI2015

Design of the SLAC RCE Platform: A General Purpose ATCA Based Data Acquisition System

The SLAC RCE platform is a general purpose clustered data acquisition system implemented on a custom ATCA compliant blade, called the Cluster On Board (COB). The core of the system is the Reconfigurable Cluster Element (RCE), which is a system-on-chip design based upon the Xilinx Zynq family of FPGAs, mounted on custom COB daughter-boards. The Zynq architecture couples a dual core ARM Cortex A9 based processor with a high performance 28nm FPGA. The RCE has 12 external general purpose bi-directional high speed links, each supporting serial rates of up to 12Gbps. 8 RCE nodes are included on a COB, each with a 10Gbps connection to an on-board 24-port Ethernet switch integrated circuit. The COB is designed to be used with a standard full-mesh ATCA backplane allowing multiple RCE nodes to be tightly interconnected with minimal interconnect latency. Multiple shelves can be clustered using the front panel 10-gbps connections. The COB also supports local and inter-blade timing and trigger distribution. An experiment specific Rear Transition Module adapts the 96 high speed serial links to specific experiments and allows an experiment-specific timing and busy feedback connection. This coupling of processors with a high performance FPGA fabric in a low latency, multiple node cluster allows high speed data processing that can be easily adapted to any physics experiment. RTEMS and Linux are both ported to the module. The RCE has been used or is the baseline for several current and proposed experiments (LCLS, HPS, LSST, ATLAS-CSC, LBNE, DarkSide, ILC-SiD, etc).

43 PARTICLE ACCELERATORS↗

DOE OSTI2018

Modeling error PDF optimization based wavelet neural network modeling of dynamic system and its application in blast furnace ironmaking

In general, the modeling errors of dynamic system model are a set of random variables. The traditional performance index of modeling such as means square error (MSE) and root means square error (RMSE) can not fully express the connotation of modeling errors with stochastic characteristics both in the dimension of time domain and space domain. Therefore, the probability density function (PDF) is introduced to completely describe the modeling errors in both time scales and space scales. Based on it, a novel wavelet neural network (WNN) modeling method is proposed by minimizing the two-dimensional (2D) PDF shaping of modeling errors. First, the modeling error PDF by the tradional WNN is estimated using data-driven kernel density estimation (KDE) technique. Then, the quadratic sum of 2D deviation between the modeling error PDF and the target PDF is utilized as performance index to optimize the WNN model parameters by gradient descent method. Since the WNN has strong nonlinear approximation and adaptive capability, and all the parameters are well optimized by the proposed method, the developed WNN model can make the modeling error PDF track the target PDF, eventually. Simulation example and application in a blast furnace ironmaking process show that the proposed method has a higher modeling precision and better generalization ability compared with the conventional WNN modeling based on MSE criteria. Furthermore, the proposed method has more desirable estimation for modeling error PDF that approximates to a Gaussian distribution whose shape is high and narrow.

Modelling↗

DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing↗

DOE OSTI2025

ViBRANT Applied as a Closed Loop Flux and Thermal Surrogate

Surrogate testing platforms provide a means to develop a target aspect of a larger system. Some systems target single aspects while others are more wholistic. ViBRANT (Visual Benign Reactor as Analog for Nuclear Testing) aims to be a wholistic system by representing the primary reactor behaviors through an accessible LED driven reactor core, photodiodes, thermocouples, and high-performance software interfaces being developed to the standards required for actual microreactor deployment such as MARVEL. With the ability to drive control drums enabled with full reactor capability (e.g. speed/angle/torque capability/limits for operation and clutch/spring scram capability), drive ViBRANT’s high fidelity 36 pin hexagon core to represent phenomena via LEDs, and capture the analog response of distributed photodiodes/thermocouples, this paper will show the system’s ability to operate in a closed-loop fashion and do well at matching reactor power/temperature distribution in an accessible reactor surrogate environment.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2022

Back-to-back inclusive dijets in DIS at small x : Sudakov suppression and gluon saturation at NLO

Back-to-back dijet cross-sections in deeply inelastic scattering (DIS) at small xBj are suppressed by many-body multiple scattering and screening effects arising from gluon saturation at high parton densities. They are similarly sensitive in these kinematics to large Sudakov logarithms from soft gluon radiation. Uncovering novel physics in this DIS channel therefore requires understanding the interplay of the two phenomena. In this work, we compute the small $x_\text{Bj}$ inclusive dijet DIS cross-section in back-to-back kinematics at next-to-leading order (NLO) in the Color Glass Condensate effective field theory (CGC EFT). Our result includes, for the first time, all real and virtual NLO contributions to the impact factor. These include all Sudakov double and single logarithm contributions, as well as all other finite $\mathcal{O}(α_s)$ terms that contribute at this order. We demonstrate explicitly that resummations of small x and Sudakov logarithms can be performed simultaneously in the CGC EFT. This requires that the JIMWLK kernel for small x evolution of the Weizsäcker-Williams (WW) gluon distribution satisfies a kinematic constraint imposed by lifetime ordering of successive gluon emissions; the corresponding modifications to the kernel, corresponding to resummations of large double transverse logarithms, are precisely of the type required to stabilize JIMWLK evolution beyond leading logarithmicaccuracy. We compute the azimuthal harmonics of the NLO back-to-back distributions and show their sensitivity to both the unpolarized and linearly polarized WW gluon distributions. Finally, we discuss how TMD factorization is broken by an emergent saturation scale at small $x$.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1992

Real-time demonstration hardware for enhanced DPCM video compression algorithm

The lack of available wideband digital links as well as the complexity of implementation of bandwidth efficient digital video CODECs (encoder/decoder) has worked to keep the cost of digital television transmission too high to compete with analog methods. Terrestrial and satellite video service providers, however, are now recognizing the potential gains that digital video compression offers and are proposing to incorporate compression systems to increase the number of available program channels. NASA is similarly recognizing the benefits of and trend toward digital video compression techniques for transmission of high quality video from space and therefore, has developed a digital television bandwidth compression algorithm to process standard National Television Systems Committee (NTSC) composite color television signals. The algorithm is based on differential pulse code modulation (DPCM), but additionally utilizes a non-adaptive predictor, non-uniform quantizer and multilevel Huffman coder to reduce the data rate substantially below that achievable with straight DPCM. The non-adaptive predictor and multilevel Huffman coder combine to set this technique apart from other DPCM encoding algorithms. All processing is done on a intra-field basis to prevent motion degradation and minimize hardware complexity. Computer simulations have shown the algorithm will produce broadcast quality reconstructed video at an average transmission rate of 1.8 bits/pixel. Hardware implementation of the DPCM circuit, non-adaptive predictor and non-uniform quantizer has been completed, providing realtime demonstration of the image quality at full video rates. Video sampling/reconstruction circuits have also been constructed to accomplish the analog video processing necessary for the real-time demonstration. Performance results for the completed hardware compare favorably with simulation results. Hardware implementation of the multilevel Huffman encoder/decoder is currently under development along with implementation of a buffer control algorithm to accommodate the variable data rate output of the multilevel Huffman encoder. A video CODEC of this type could be used to compress NTSC color television signals where high quality reconstruction is desirable (e.g., Space Station video transmission, transmission direct-to-the-home via direct broadcast satellite systems or cable television distribution to system headends and direct-to-the-home).

Bizon, Thomas P.↗

DOE OSTI2023

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗

NASA NTRS2015

A Metric to Quantify Shared Visual Attention in Two-Person Teams

Introduction: Critical tasks in high-risk environments are often performed by teams, the members of which must work together efficiently. In some situations, the team members may have to work together to solve a particular problem, while in others it may be better for them to divide the work into separate tasks that can be completed in parallel. We hypothesize that these two team strategies can be differentiated on the basis of shared visual attention, measured by gaze tracking. 2) Methods: Gaze recordings were obtained for two-person flight crews flying a high-fidelity simulator (Gontar, Hoermann, 2014). Gaze was categorized with respect to 12 areas of interest (AOIs). We used these data to construct time series of 12 dimensional vectors, with each vector component representing one of the AOIs. At each time step, each vector component was set to 0, except for the one corresponding to the currently fixated AOI, which was set to 1. This time series could then be averaged in time, with the averaging window time (t) as a variable parameter. For example, when we average with a t of one minute, each vector component represents the proportion of time that the corresponding AOI was fixated within the corresponding one minute interval. We then computed the Pearson product-moment correlation coefficient between the gaze proportion vectors for each of the two crew members, at each point in time, resulting in a signal representing the time-varying correlation between gaze behaviors. We determined criteria for concluding correlated gaze behavior using two methods: first, a permutation test was applied to the subjects' data. When one crew member's gaze proportion vector is correlated with a random time sample from the other crewmember's data, a distribution of correlation values is obtained that differs markedly from the distribution obtained from temporally aligned samples. In addition to validating that the gaze tracker was functioning reasonably well, this also allows us to compute probabilities of coordinated behavior for each value of the correlation. As an alternative, we also tabulated distributions of correlation coefficients for synthetic data sets, in which the behavior was modeled as a first-order Markov process, and compared correlation distributions for identical processes with those for disparate processes, allowing us to choose criteria and estimate error rates. 3) Discussion: Our method of gaze correlation is able to measure shared visual attention, and can distinguish between activities involving different instruments. We plan to analyze whether pilots strategies of sharing visual attention can predict performance. Possible measurements of performance include expert ratings from instructors, fuel consumption, total task time, and failure rate. While developed for two-person crews, our approach can be applied to larger groups, using intra-class correlation coefficients instead of the Pearson product-moment correlation.

Gontar, Patrick↗

DOE OSTI2017

SCIDAC Center for simulation of wave particle interactions CompX participation

Harnessing the energy that is released in fusion reactions would provide a safe and abundant source of power to meet the growing energy needs of the world population. The next step toward the development of fusion as a practical energy source is the construction of ITER, a device capable of producing and controlling the high performance plasma required for self-sustaining fusion reactions, or “burning” plasma. The input power required to drive the ITER plasma into the burning regime will be supplied primarily with a combination of external power from radio frequency waves in the ion cyclotron range of frequencies and energetic ions from neutral beam injection sources, in addition to internally generated Ohmic heating from the induced plasma current that also serves to create the magnetic equilibrium for the discharge. The ITER project is a large multi-billion dollar international project in which the US participates. The success of the ITER project depends critically on the ability to create and maintain burning plasma conditions, it is absolutely necessary to have physics-based models that can accurately simulate the RF processes that affect the dynamical evolution of the ITER discharge. The Center for Simulation of WavePlasma Interactions (CSWPI), also known as RF-SciDAC, is a multi-institutional collaboration that has conducted ongoing research aimed at developing: (1) Coupled core-to-edge simulations that will lead to an increased understanding of parasitic losses of the applied RF power in the boundary plasma between the RF antenna and the core plasma; (2) Development of models for core interactions of RF waves with energetic electrons and ions (including fusion alpha particles and fast neutral beam ions) that include a more accurate representation of the particle dynamics in the combined equilibrium and wave fields; and (3) Development of improved algorithms that will take advantage of massively parallel computing platforms at the petascale level and beyond to achieve the needed physics, resolution, and/or statistics to address these issues. CompX provides computer codes and analysis for the calculation of the electron and ion distributions in velocity-space and plasma radius which are necessary for reliable calculations of power deposition and toroidal current drive due to combined radiofrequency and neutral beam at high injected powers. It has also contributed to ray tracing modeling of injected radiofrequency powers, and to coupling between full-wave radiofrequency wave models and the distribution function calculations. In the course of this research, the Fokker-Planck distribution function calculation was made substantially more realistic by inclusion of finite-width drift-orbit effects (FOW). FOW effects were also implemented in a calculation of the phase-space diffusion resulting from radiofrequency full-wave models. Average level of funding for CompX was approximately three man-months per year.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS2000

Solar UV Radiation and the Origin of Life on Earth

We have started a comprehensive, interdisciplinary study of the influence of solar ultraviolet radiation on the atmosphere of of the early Earth. We plan to model the chemistry of the Earth atmosphere during its evolution, using observed UV flux distributions of early solar analogs as boundary conditions in photochemical models of the Earth's atmosphere. The study has four distinct but interlinked parts: (1) Establishing the radiation of the early Sun; (2) Determining the photochemistry of the early Earth's atmosphere; (3) Estimating the rates of H2 loss from the atmosphere; and (4) Ascertaining how sensitive is the photochemistry to the metallicity of the Sun. We are currently using STIS and EUVE to obtain high-quality far-UV and extreme-UV observations of three early-solar analogs. We will perform a detailed non-LTE study of each stars, and construct theoretical model photosphere, and an empirical model chromospheres, which can be used to extrapolate the continuum to the Lyman continuum region. Given a realistic flux distribution of the early Sun, we will perform photochemical modeling of weakly reducing primitive atmospheres to determine the lifetime and photochemistry of CH4. In particular, we will make estimates of the amount of CH4 present in the prebiotic atmosphere, and estimate the atmospheric CH4 concentration during the Late Archean (2.5-3.0 b.y. ago) and determine whether it would have been sufficiently abundant to help offset reduced solar luminosity at that time. Having obtained a photochemical model, we will solve for the concentrations of greenhouse gasses and important pre-biotic molecules, and perform a detailed radiative transfer calculations to compute the UV flux reaching the surface.

Heap, Sara R.↗

DOE OSTI2014

SU-E-T-587: Monte Carlo Versus Ray-Tracing for Treatment Planning Involving CNS Tumors On the MultiPlan System for CyberKnife Radiosurgery

Purpose: MultiPlan, the treatment planning system for the CyberKnife Robotic Radiosurgery system offers two approaches to dose computation, namely Ray-Tracing (RT), the default technique and Monte Carlo (MC), an option. RT is deterministic, however it accounts for primary heterogeneity only. MC on the other hand has an uncertainty associated with the calculation results. The advantage is that in addition, it accounts for heterogeneity effects on the scattered dose. Not all sites will benefit from MC. The goal of this work was to focus on central nervous system (CNS) tumors and compare dosimetrically, treatment plans computed with RT versus MC. Methods: Treatment plans were computed using both RT and MC for sites covering (a) the brain (b) C-spine (c) upper T-spine (d) lower T-spine (e) L-spine and (f) sacrum. RT was first used to compute clinically valid treatment plans. Then the same treatment parameters, monitor units, beam weights, etc., were used in the MC algorithm to compute the dose distribution. The plans were then compared for tumor coverage to illustrate the difference if any. All MC calculations were performed at a 1% uncertainty. Results: Using the RT technique, the tumor coverage for the brain, C-spine (C3–C7), upper T-spine (T4–T6), lower T-spine (T10), Lspine (L2) and sacrum were 96.8%, 93.1%, 97.2%, 87.3%, 91.1%, and 95.3%. The corresponding tumor coverage based on the MC approach was 98.2%, 95.3%, 87.55%, 88.2%, 92.5%, and 95.3%. It should be noted that the acceptable planning target coverage for our clinical practice is >95%. The coverage can be compromised for spine tumors to spare normal tissues such as the spinal cord. Conclusion: For treatment planning involving the CNS, RT and MC appear to be similar for most sites but for the T-spine area where most of the beams traverse lung tissue. In this case, MC is highly recommended.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2014

Map Matching and Real World Integrated Sensor Data Warehousing (Presentation)

The inclusion of interlinked temporal and spatial elements within integrated sensor data enables a tremendous degree of flexibility when analyzing multi-component datasets. The presentation illustrates how to warehouse, process, and analyze high-resolution integrated sensor datasets to support complex system analysis at the entity and system levels. The example cases presented utilizes in-vehicle sensor system data to assess vehicle performance, while integrating a map matching algorithm to link vehicle data to roads to demonstrate the enhanced analysis possible via interlinking data elements. Furthermore, in addition to the flexibility provided, the examples presented illustrate concepts of maintaining proprietary operational information (Fleet DNA) and privacy of study participants (Transportation Secure Data Center) while producing widely distributed data products. Should real-time operational data be logged at high resolution across multiple infrastructure types, map matched to their associated infrastructure, and distributed employing a similar approach; dependencies between urban environment infrastructures components could be better understood. This understanding is especially crucial for the cities of the future where transportation will rely more on grid infrastructure to support its energy demands.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2018

Stochastic Generation Capacity Expansion Planning with Approximate Dynamic Programming

Failing to consider the many long-term uncertainties that affect the performance of a power generation portfolio can result in suboptimal generation expansion plans. Further, traditional deterministic approaches can omit flexible plans that are able to adapt to future events. In this paper, we explore the use of approximate dynamic programming (ADP) to create forward-looking generation expansion plans. A case study is included with three sequential decision periods; three generation technologies; and four uncertainties: demand growth, natural gas prices, renewable portfolio standards, and the adoption of customer-driven solar generation. The flexible plans found through ADP show a 3 % reduction in total expected cost when compared to myopic planning heuristics while circumventing the computational burdens that accompany high-dimensional dynamic programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗