Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Streaming Readout and Data-Stream Processing With ERSAP

With the exponential growth in the volume and complexity of data generated at high-energy physics and nuclear physics research facilities, there is an imperative demand for innovative strategies to process this data in real or near-real-time. Given the surge in the requirement for high-performance computing, it becomes pivotal to reassess the adaptability of current data processing architectures in integrating new technologies and managing streaming data. This paper introduces the ERSAP framework, a modern solution that synergizes flow-based programming with the reactive actor model, paving the way for distributed, reactive, and high performance in data stream processing applications. Additionally, we unveil a novel algorithm focused on time-based clustering and event identification in data streams. The efficacy of this approach is further exemplified through the data-stream processing outcomes obtained from the recent beam tests of the EIC prototype calorimeter at DESY.

Vardan, Gyurjyan↗
DOE OSTI2017

High Performance Data Transfer for Distributed Data Intensive Sciences

We report on the development of ZX software providing high performance data transfer and encryption. The design scales in: computation power, network interfaces, and IOPS while carefully balancing the available resources. Two U.S. patent-pending algorithms help tackle data sets containing lots of small files and very large files, and provide insensitivity to network latency. It has a cluster-oriented architecture, using peer-to-peer technologies to ease deployment, operation, usage, and resource discovery. Its unique optimizations enable effective use of flash memory. Using a pair of existing data transfer nodes at SLAC and NERSC, we compared its performance to that of bbcp and GridFTP and determined that they were comparable. With a proof of concept created using two four-node clusters with multiple distributed multi-core CPUs, network interfaces and flash memory, we achieved 155Gbps memory-to-memory over a 2x100Gbps link aggregated channel and 70Gbps file-to-file with encryption over a 5000 mile 100Gbps link.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Frontier Job-Centric Telemetry Dataset

Comprehensive analysis of high-performance computing (HPC) systems requires linking workload execution to system behavior. This kind of analysis is vital for diagnosing performance issues, managing capacity, detecting anomalous workloads, and understanding how applications interact with system hardware. This job-centric telemetry dataset unifies scheduler job records with node-level measurements, enabling direct association between workloads and their corresponding power, thermal, and performance characteristics. It contains sanitized, scheduler related metadata for 152,400 individual jobs that ran on the Frontier supercomputer and ended on selected days throughout 2024 and 2025, a subpopulation of ~6.8% of the total number of allocated jobs with non-zero run time on the system over that same period. Each is linked with files that contain telemetry time series records of the power utilization and temperature behavior of its allocated nodes and their processors during the run time of the job. Where available, a portion of the job files also contain network performance time series. Jobs are sampled from select days that reflect normal levels of user activity and possess job size distributions with large numbers of leadership class jobs (>20% of Frontier nodes). Jobs in this dataset attempt to best represent successful user workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2019

Data automation at light sources

Rapidly growing data volumes at light sources demand increasingly automated data collection, distribution, and analysis processes, in order to enable new scientific discoveries while not overwhelming finite human capabilities. We present here the case for automating and outsourcing light source science using cloud-hosted data automation and enrichment services, institutional computing resources, and high- performance computing facilities to provide cost-effective, scalable, and reliable implementations of such processes. We discuss three specific services that accomplish these goals for data distribution, automation, and transformation. In the first, Globus cloud-hosted data automation services are used to implement data capture, distribution, and analysis workflows for Advanced Photon Source and Advanced Light Source beamlines, leveraging institutional storage and computing. In the second, such services are combined with cloud-hosted data indexing and institutional storage to create a collaborative data publication, indexing, and discovery service, the Materials Data Facility (MDF), built to support a host of informatics applications in materials science. The third integrates components of the previous two projects with machine learning capabilities provided by the Data and Learning Hub for science (DLHub) to enable on-demand access to machine learning models from light source data capture and analysis workflows, and provides simplified interfaces to train new models on data from sources such as MDF on leadership scale computing resources. We draw conclusions about best practices for building next-generation data automation systems for future light sources.

Blaiszik, Ben↗
DOE OSTI2023

TAMM: Tensor algebra for many-body methods

Tensor algebra operations such as contractions in computational chemistry consume a significant fraction of the computing time on large-scale computing platforms. The widespread use of tensor contractions between large multi-dimensional tensors in describing electronic structure theory has motivated the development of multiple tensor algebra frameworks targeting heterogeneous computing platforms. In this paper, we present Tensor Algebra for Many-body Methods (TAMM), a framework for productive and performance-portable development of scalable computational chemistry methods. TAMM decouples the specification of the computation from the execution of these operations on available high-performance computing systems. With this design choice, the scientific application developers (domain scientists) can focus on the algorithmic requirements using the tensor algebra interface provided by TAMM, whereas high-performance computing developers can direct their attention to various optimizations on the underlying constructs, such as efficient data distribution, optimized scheduling algorithms, and efficient use of intra-node resources (e.g., graphics processing units). The modular structure of TAMM allows it to support different hardware architectures and incorporate new algorithmic advances. We describe the TAMM framework and our approach to the sustainable development of scalable ground- and excited-state electronic structure methods. We present case studies highlighting the ease of use, including the performance and productivity gains compared to other frameworks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1992

Uncertainties in obtaining high reliability from stress-strength models

There has been a recent interest in determining high statistical reliability in risk assessment of aircraft components. The potential consequences are identified of incorrectly assuming a particular statistical distribution for stress or strength data used in obtaining the high reliability values. The computation of the reliability is defined as the probability of the strength being greater than the stress over the range of stress values. This method is often referred to as the stress-strength model. A sensitivity analysis was performed involving a comparison of reliability results in order to evaluate the effects of assuming specific statistical distributions. Both known population distributions, and those that differed slightly from the known, were considered. Results showed substantial differences in reliability estimates even for almost nondetectable differences in the assumed distributions. These differences represent a potential problem in using the stress-strength model for high reliability computations, since in practice it is impossible to ever know the exact (population) distribution. An alternative reliability computation procedure is examined involving determination of a lower bound on the reliability values using extreme value distributions. This procedure reduces the possibility of obtaining nonconservative reliability estimates. Results indicated the method can provide conservative bounds when computing high reliability. An alternative reliability computation procedure is examined involving determination of a lower bound on the reliability values using extreme value distributions. This procedure reduces the possibility of obtaining nonconservative reliability estimates. Results indicated the method can provide conservative bounds when computing high reliability.

Neal, Donald M.↗
DOE OSTI2023

Evaluation of distributed process-based hydrologic model performance using only a priori information to define model inputs

Fully distributed, integrated surface–subsurface hydrological models (ISSHMs) have seen renewed interest due to availability of better software, high performance computing facilities, and high-resolution, spatially extensive data products. ISSHMs are valuable as tools for advancing system understanding as they can resolve multiple processes defined on the plot scale including three-dimensional interaction of surface water and groundwater. Here, we evaluated the performance of an ISSHM, the Advanced Terrestrial Simulator (ATS), on seven diverse catchments across the continental US using widely available data products to define model inputs without calibration. We compare the ATS-simulated streamflow and evapotranspiration with gauge observations and MODIS-derived evapotranspiration, respectively. Using the Kling-Gupta Efficiency (KGE) as metric, ATS with default data products performed reasonably well at 6 of 7 catchments for streamflow. However, in one of those 6 catchments ATS had poor performance on baseflow and ATS’s overall performance was thus judged to be inadequate despite the acceptable KGE. ATS performance for evapotranspiration was good in all 7 catchments using default data products. In the two catchments where ATS streamflow performance using default data products was not acceptable, the performance was significantly improved by using local information on subsurface properties below the soil. We also compare the model-simulated streamflow and evapotranspiration with the Sacramento soil moisture accounting (SAC-SMA) model, a semi-distributed model that was calibrated on a catchment-by-catchment basis. Uncalibrated ATS performance is comparable to the calibrated SAC-SMA model in terms of streamflow while ATS performance is similar to or better (much better in certain catchments) in reproducing MODIS-derived evapotranspiration. Reasonably good performance of ATS without catchment-specific calibration provides new confidence in the ISSHM class of models and community data products as tools for advancing understanding of watershed function in a changing environment.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Reflections on the Shifting Experiences of Scientific Infrastructure

Infrastructure of all types is fundamental to modern work and life. Computing for scientific work, especially, extends from distributed local research sites, often at the edges of other major systems, outward into globally connected high-performance facilities and infrastructures. This commentary reviews longstanding research on the social characteristics of infrastructure. We reflect on social concerns that affect the ongoing development, use, and maintenance of a wide range of scientific computing and data resources. Reflecting on the social nature of infrastructure is timely for Computing in Science & Engineering readers, given continued emphasis on developing even more expansive platforms for data and artificial intelligence work in science (e.g., the United States’ Genesis Mission). We assert that, regardless of technological advances, the complex nature of scientific research and data will require continued understanding of longstanding and nascent social practices across varied communities. This is fundamentally necessary to build and sustain usable infrastructure or platforms that can productively advance scientific research.

Paine, Drew [Lawrence Berkeley National Laboratory↗
DOE OSTI2026

HydraGNN_OPF_GFM_2026 - Ensemble of predictive graph foundation models for power grid applications

This dataset supports research on graph foundation models for optimal power flow (OPF) on electric grids using HydraGNN. It contains heterogeneous graph representations of PGLib-OPF cases spanning systems from 14 to 13,659 buses, together with packed HDF5 datasets for pretraining, feasibility classification, and N-1 contingency analysis. The release includes OPF solution data, downstream fine-tuning datasets, pretrained HeteroSAGE and HeteroHEAT model checkpoints, hyperparameter-optimization summaries across multiple heterogeneous GNN architectures, and aggregated fine-tuning results for sample-efficiency studies. The dataset is designed to enable scalable training, evaluation, and transfer-learning studies for OPF surrogate modeling, including node-level AC-OPF solution prediction, graph-level prediction, feasibility classification, operating-condition generalization, and contingency-response tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1995

Constructing Scientific Applications from Heterogeneous Resources

A new model for high-performance scientific applications in which such applications are implemented as heterogeneous distributed programs or, equivalently, meta-computations, is investigated. The specific focus of this grant was a collaborative effort with researchers at NASA and the University of Toledo to test and improve Schooner, a software interconnection system, and to explore the benefits of increased user interaction with existing scientific applications.

Schichting, Richard D.↗
NASA NTRS2018

NASA GSFC Avionics Architectures and Future Directions

NASA Goddard Spaceflight Center (GSFC) employs standard avionics hardware and software architectures. Hardware architectures include GMSA (Goddard Modular Smallsat Architecture), MUSTANG (Modular Unified Space Technology Avionics for Next Generation), SpaceCube. The spaceflight software architecture is based on cFS (Core Flight System). Future driving requirements for avionics architectures include increase sensor data rates, increased onboard processing, autonomous applications, and distributed space missions. Architectural concepts that can meet these requirements include the use of the High Performance Spaceflight Computing (HPSC) Chiplet, employing hybrid computing architectures, and increasing network bandwidths.

architectures↗
NASA NTRS1980

Performance of computer-optimized tapered-roller bearings to 2.4 million DN

The temperature distribution and bearing heat generation of 120.65 mm bore high-speed tapered roller bearings was determined at shaft speeds of 20,000 rpm under simultaneous thrust and radial loads. The temperatures and thermal outputs were computed as functions of shaft speed, loading lubricant flow rates, and lubricant inlet temperatures. Bearing temperatures and heat generation were considerably lower than in standard bearings; cup cooling was effective in lowering cup temperatures to levels of cone temperatures.

Parker, R. J.↗
NASA NTRS1993

High-performance mass storage system for workstations

Reduced Instruction Set Computer (RISC) workstations and Personnel Computers (PC) are very popular tools for office automation, command and control, scientific analysis, database management, and many other applications. However, when using Input/Output (I/O) intensive applications, the RISC workstations and PC's are often overburdened with the tasks of collecting, staging, storing, and distributing data. Also, by using standard high-performance peripherals and storage devices, the I/O function can still be a common bottleneck process. Therefore, the high-performance mass storage system, developed by Loral AeroSys' Independent Research and Development (IR&D) engineers, can offload a RISC workstation of I/O related functions and provide high-performance I/O functions and external interfaces. The high-performance mass storage system has the capabilities to ingest high-speed real-time data, perform signal or image processing, and stage, archive, and distribute the data. This mass storage system uses a hierarchical storage structure, thus reducing the total data storage cost, while maintaining high-I/O performance. The high-performance mass storage system is a network of low-cost parallel processors and storage devices. The nodes in the network have special I/O functions such as: SCSI controller, Ethernet controller, gateway controller, RS232 controller, IEEE488 controller, and digital/analog converter. The nodes are interconnected through high-speed direct memory access links to form a network. The topology of the network is easily reconfigurable to maximize system throughput for various applications. This high-performance mass storage system takes advantage of a 'busless' architecture for maximum expandability. The mass storage system consists of magnetic disks, a WORM optical disk jukebox, and an 8mm helical scan tape to form a hierarchical storage structure. Commonly used files are kept in the magnetic disk for fast retrieval. The optical disks are used as archive media, and the tapes are used as backup media. The storage system is managed by the IEEE mass storage reference model-based UniTree software package. UniTree software will keep track of all files in the system, will automatically migrate the lesser used files to archive media, and will stage the files when needed by the system. The user can access the files without knowledge of their physical location. The high-performance mass storage system developed by Loral AeroSys will significantly boost the system I/O performance and reduce the overall data storage cost. This storage system provides a highly flexible and cost-effective architecture for a variety of applications (e.g., realtime data acquisition with a signal and image processing requirement, long-term data archiving and distribution, and image analysis and enhancement).

Chiang, T.↗
DOE OSTI2020

Distributed Computing for the Project 8 Experiment

The Project 8 collaboration aims to measure the absolute neutrino mass or improve on the current limit by measuring the tritium beta decay electron spectrum. We present the current distributed computing model for the Project 8 experiment. Project 8 is in its second phase of data taking with a near continuous data rate of 1Gbps. The current computing model uses DIRAC (Distributed Infrastructure with Remote Agent Control) for its workflow and data management. A detailed meta-data assignment using the DIRAC File Catalog is used to automate raw data transfers and subsequent stages of data processing. The DIRAC system is deployed on containers managed using a Kubernetes cluster to provide a scalable infrastructure. A modified DIRAC Site Director provides the ability to submit jobs using Singularity on opportunistic High-Performance Computing (HPC) sites.

Distributed Computing, Kubernetes, DIRAC, Project ↗
DOE OSTI2017

ESIF 2016: Modernizing Our Grid and Energy System

This 2016 annual report highlights work conducted at the Energy Systems Integration Facility (ESIF) in FY 2016, including grid modernization, high-performance computing and visualization, and INTEGRATE projects.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

58 GEOSCIENCES↗
DOE OSTI2018

Evaluating the Feasibility of Isolated Pancreatic Perfusion for Chemotherapy Using Computed Tomography: An Experimental Study in Pig Models

PurposePercutaneous isolated pancreatic perfusion (PIPP) is performed along with interventional radiology techniques to obtain high drug concentration by occluding the arterial inlet and venous outlet of the pancreas. The experimental study aimed to evaluate the contrast distribution in PIPP under different flow rates with or without anterior mesenteric artery (AMA) occlusion.Materials and MethodsThis study was approved by a local animal experiment ethics committee. Nine pigs were divided into Groups 1, 2, and 3, by infusion rates of 12, 24, and 36 mL/min. Groups 4 and 5 (3 pigs each) and Group 6 (2 pigs) underwent PIPP at the same respective infusion rates with and without AMA occlusion. Computed tomography (CT) arteriography was performed during PIPP with nonionic contrast media. The enhanced volume was calculated by adding the enhanced area in each slice using 1.25-mm axial images. The percent enhanced volume to the whole pancreas (%eV) was used to simulate drug distribution; the result was compared among groups.ResultsWithout AMA occlusion, a larger %eV was obtained with high infusion rates (P = 0.039). The median %eV in Groups 1, 2, and 3 were 57.7, 74.2, and 90.5%, respectively. With AMA occlusion, CT demonstrated duodenal enhancement at an infusion rate of 36 mL/min, and the median %eV in Groups 4, 5, and 6 were 92.8, 95.4, and 98.5%, respectively. A significantly larger %eV was obtained after AMA occlusion (P = 0.031).ConclusionA higher infusion rate or AMA occlusion increases the enhanced volume in PIPP in pig models. Level of evidence: No level of evidence.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2021

Experimental and computational analysis of bending fatigue failure in chopped carbon fiber chip reinforced composites

With a better balance among good mechanical performance, high freedom of design, and low material and manufacturing cost, chopped carbon fiber chip reinforced sheet molding compound (SMC) composites show great potential in different engineering applications. Here in this paper, bending fatigue behaviors of SMC composites considering the heterogeneous fiber orientation distributions have been thoroughly investigated utilizing both experimental and computational methods. First, four-point bending fatigue tests are performed with designed SMC composites, and the local modulus is adopted as a metric to represent the local fiber orientation of two opposing sides. Interestingly, SMC composites with and without large discrepancy in local modulus of opposing sides show different fatigue behaviors. Interrupted tests are conducted to explore the bending fatigue failure mechanism, and the damage processes of valid specimens are also closely examined. We find that the fatigue failure of SMC composites under four-point bending is governed by crack propagation instead of crack initiation. Because of this, the heterogeneous local fiber orientations of both sides of the specimen influence fatigue life. The microstructure of the lower side shows a direct influence while that of the upper side also exhibiting influence which becomes more prominent for high cycle fatigue cases. Furthermore, a hybrid micro–macro computational model is proposed to efficiently study the cyclic bending behavior of SMC composites. The region of interest is reconstructed with a modified random sequential absorption algorithm to conserve all the microstructural details including the heterogeneous fiber orientation, while the rest of the regions are modeled as homogenized macro-scale continua. Combined with a framework to capture the progressive fatigue damage under cyclic bending, the bending fatigue behaviors of SMC composites are accurately captured by the hybrid computational model comparing with our experimental analysis.

36 MATERIALS SCIENCE↗