SEARCH · Engineering Papers

Results for “Data Intensive Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

All sources NASA NTRS DOE OSTI

NASA NTRS2011

Accelerated Adaptive MGS Phase Retrieval

The Modified Gerchberg-Saxton (MGS) algorithm is an image-based wavefront-sensing method that can turn any science instrument focal plane into a wavefront sensor. MGS characterizes optical systems by estimating the wavefront errors in the exit pupil using only intensity images of a star or other point source of light. This innovative implementation of MGS significantly accelerates the MGS phase retrieval algorithm by using stream-processing hardware on conventional graphics cards. Stream processing is a relatively new, yet powerful, paradigm to allow parallel processing of certain applications that apply single instructions to multiple data (SIMD). These stream processors are designed specifically to support large-scale parallel computing on a single graphics chip. Computationally intensive algorithms, such as the Fast Fourier Transform (FFT), are particularly well suited for this computing environment. This high-speed version of MGS exploits commercially available hardware to accomplish the same objective in a fraction of the original time. The exploit involves performing matrix calculations in nVidia graphic cards. The graphical processor unit (GPU) is hardware that is specialized for computationally intensive, highly parallel computation. From the software perspective, a parallel programming model is used, called CUDA, to transparently scale multicore parallelism in hardware. This technology gives computationally intensive applications access to the processing power of the nVidia GPUs through a C/C++ programming interface. The AAMGS (Accelerated Adaptive MGS) software takes advantage of these advanced technologies, to accelerate the optical phase error characterization. With a single PC that contains four nVidia GTX-280 graphic cards, the new implementation can process four images simultaneously to produce a JWST (James Webb Space Telescope) wavefront measurement 60 times faster than the previous code.

Lam, Raymond K.↗

DOE OSTI2023

High Energy Physics Network Requirements Review: Two-Year Update

The Energy Sciences Network (ESnet) is the high-performance network user facility for the US Department of Energy (DOE) Office of Science (SC) and delivers highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the DOE science mission by connecting all its laboratories and facilities in the US and abroad. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet interconnects DOE national laboratories, user facilities, and major experiments so that scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large datasets, and access distributed data repositories. ESnet is specifically built to provide a range of network services tailored to meet the unique requirements of the DOE’s data-intensive science. In July 2023, the Energy Sciences Network (ESnet) and the High Energy Physics program (HEP) of the DOE SC organized an interim ESnet requirements review of HEP-supported activities, to follow up on the work started during the 2020 HEP Network Requirements Review. Preparation for these events included checking back with the key stakeholders: program and facility management, research groups, and technology providers. Each stakeholder group was asked to prepare updates to their previously submitted case study documents, so that ESnet could update the understanding of any changes to the current, near-term, and long-term status, expectations, and processes that will support the science activities of the program.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

HOSS!

The Hall-D Online Skim System (HOSS) was developed to simultaneously solve two issues for the high intensity GlueX experiment. One was to parallelize the writing of raw data files to disk in order to improve bandwidth. The other was to distribute the raw data across multiple compute nodes in order to produce calibration skims of the data online. The highly configurable system employs RDMA, RAM disks, and zeroMQ driven by Python to simultaneously store and process the full high intensity GlueX data stream.

Lawrence, David↗

DOE OSTI2018

Performance Evaluation and Optimization of HBM-Enabled GPU for Data-Intensive Applications

Not provided.

Computer Science↗

NASA NTRS1977

The moisture budget in relation to convection

An evaluation of the moisture budget in the environment of convective storms is presented by using the unique 3- to 6-h rawinsonde data. Net horizontal and vertical boundary fluxes accounted for most of the large amounts of moisture which were concentrated into convective regions associated with two squall lines that moved through the area during the experiment. The largest values of moisture accumulations were located slightly downwind of the most intense convective activity. Relationships between computed moisture quantities of the moisture budget and radar-observed convection improved when lagging the radar data by 3 h. The residual of moisture which represents all sources and sinks of moisture in the budget equation was largely accounted for by measurements of precipitation.

Scott, R. W.↗

DOE OSTI2023

High-throughput terahertz imaging: progress and challenges

Abstract Many exciting terahertz imaging applications, such as non-destructive evaluation, biomedical diagnosis, and security screening, have been historically limited in practical usage due to the raster-scanning requirement of imaging systems, which impose very low imaging speeds. However, recent advancements in terahertz imaging systems have greatly increased the imaging throughput and brought the promising potential of terahertz radiation from research laboratories closer to real-world applications. Here, we review the development of terahertz imaging technologies from both hardware and computational imaging perspectives. We introduce and compare different types of hardware enabling frequency-domain and time-domain imaging using various thermal, photon, and field image sensor arrays. We discuss how different imaging hardware and computational imaging algorithms provide opportunities for capturing time-of-flight, spectroscopic, phase, and intensity image data at high throughputs. Furthermore, the new prospects and challenges for the development of future high-throughput terahertz imaging systems are briefly introduced.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Optical neural engine for solving scientific partial differential equations

Abstract Solving partial differential equations (PDEs) is the cornerstone of scientific research and development. Data-driven machine learning (ML) approaches are emerging to accelerate time-consuming and computation-intensive numerical simulations of PDEs. Although optical systems offer high-throughput and energy-efficient ML hardware, their demonstration for solving PDEs is limited. Here, we present an optical neural engine (ONE) architecture combining diffractive optical neural networks for Fourier space processing and optical crossbar structures for real space processing to solve time-dependent and time-independent PDEs in diverse disciplines, including Darcy flow equation, the magnetostatic Poisson’s equation in demagnetization, the Navier-Stokes equation in incompressible fluid, Maxwell’s equations in nanophotonic metasurfaces, and coupled PDEs in a multiphysics system. We numerically and experimentally demonstrate the capability of the ONE architecture, which not only leverages the advantages of high-performance dual-space processing for outperforming traditional PDE solvers and being comparable with state-of-the-art ML models but also can be implemented using optical computing hardware with unique features of low-energy and highly parallel constant-time processing irrespective of model scales and real-time reconfigurability for tackling multiple tasks with the same architecture. The demonstrated architecture offers a versatile and powerful platform for large-scale scientific and engineering computations.

Tang, Yingheng (ORCID:0009000153622546)↗

DOE OSTI2023

High-throughput calculations of charged point defect properties with semi-local density functional theory—performance benchmarks for materials screening applications

Abstract Calculations of point defect energetics with Density Functional Theory (DFT) can provide valuable insight into several optoelectronic, thermodynamic, and kinetic properties. These calculations commonly use methods ranging from semi-local functionals with a-posteriori corrections to more computationally intensive hybrid functional approaches. For applications of DFT-based high-throughput computation for data-driven materials discovery, point defect properties are of interest, yet are currently excluded from available materials databases. This work presents a benchmark analysis of automated, semi-local point defect calculations with a-posteriori corrections, compared to 245 “gold standard” hybrid calculations previously published. We consider three different a-posteriori correction sets implemented in an automated workflow, and evaluate the qualitative and quantitative differences among four different categories of defect information: thermodynamic transition levels, formation energies, Fermi levels, and dopability limits. We highlight qualitative information that can be extracted from high-throughput calculations based on semi-local DFT methods, while also demonstrating the limits of quantitative accuracy.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Computational approaches to model X-ray photon correlation spectroscopy from molecular dynamics

X-ray photon correlation spectroscopy (XPCS) allows for the resolution of dynamic processes within a material across a wide range of length and time scales. X-ray speckle visibility spectroscopy (XSVS) is a related method that uses a single diffraction pattern to probe ultrafast dynamics. Interpretation of the XPCS and XSVS data in terms of underlying physical processes is necessary to establish the connection between the macroscopic responses and the microstructural dynamics. To aid the interpretation of the XPCS and XSVS data, we present a computational framework to model these experiments by computing the X-ray scattering intensity directly from the atomic positions obtained from molecular dynamics simulations. We compare the efficiency and accuracy of two alternative computational methods: the direct method computing the intensity at each diffraction vector separately, and a method based on fast Fourier transform that computes the intensities at all diffraction vectors at once. Furthermore, the computed X-ray speckle patterns capture the density fluctuations over a range of length and time scales and are shown to reproduce the known properties and relations of experimental XPCS and XSVS for liquids.

36 MATERIALS SCIENCE↗

DOE OSTI2016

INTEGRATION OF PANDA WORKLOAD MANAGEMENT SYSTEM WITH SUPERCOMPUTERS

Abstract The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the funda- mental nature of matter and the basic forces that shape our universe, and were recently credited for the dis- covery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 140 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data cen- ters are physically scattered all over the world. While PanDA currently uses more than 250000 cores with a peak performance of 0.3+ petaFLOPS, next LHC data taking runs will require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Com- puting Facility (OLCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava, and others). The current approach utilizes a modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single- threaded workloads in parallel on Titan s multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms. We will present our current accom- plishments in running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facility s infrastructure for High Energy and Nuclear Physics, as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

De, K↗

NASA NTRS1982

Ultrasonic Transducer Analyzer

Ultrasonic transducer-beam-intensity distributions are determined by analyzing echoes from a spherical ball. Computers control equipment and process data. Important beam characteristics, such as location of best beam focus and beam diameter at focus, can be determined quickly from extensive set of plots generated by apparatus.

Grounds, M. K.↗

NASA NTRS1986

Influence of bulk turbulence and entrance boundary layer thickness on the curved duct flow field

The objective of this investigation was the experimental evaluation of bulk turbulence and boundary thickness influence on the secondary flow development in a square, 90 deg turning duct. A three dimensional laser velocimetry system was utilized to measure the mean and fluctuating components of velocity in the large curved duct facility. The three dimensional development of the viscous shear layers in the curved duct has a strong influence on the complete flow field. Since ducted three dimensional flows are found in many engineering applications, including gas turbine engines, and contain high turbulence levels and high wall heat transfer rates, they present a difficult challenge to computational fluid mechanics codes. Turbulence modeling remains one of constraints to CFD advance due to inadequate physical understanding and experimental definition of turbulent shear flows. The results of this investigation expand the curved duct data base to higher turbulence levels and thicker entrance boundary layers. The experimental results provide a challenging benchmark data base for computational fluid dynamics code development and validation. The variation of inlet bulk turbulence intensity provides additional information to aid in turbulence model evaluation.

Crawford, Roger A.↗

NASA NTRS1990

Influence of bulk turbulence and entrance boundary layer thickness on the curved duct flow field

Shohadaee, Ahmad A.↗

DOE OSTI2011

On the Origin of Ultra High Energy Cosmic Rays II

We show that accretion disks around Active Galactic Nuclei (AGNs) could account for the enormous power in observed ultra high energy cosmic rays {approx}10{sup 20} eV (UHEs). In our model, cosmic rays are produced by quasi-steady acceleration of ions in magnetic structures previously proposed to explain jets around Active Galactic Nuclei with supermassive black holes. Steady acceleration requires that an AGN accretion disk act as a dynamo, which we show to follow from a modified Standard Model in which the magnetic torque of the dynamo replaces viscosity as the dominant mechanism accounting for angular momentum conservation during accretion. A black hole of mass M{sub BH} produces a steady dynamo voltage V {proportional_to} {radical}M{sub BH} giving V {approx} 10{sup 20} volts for M{sub BH} {approx} 10{sup 8} solar masses. The voltage V reappears as an inductive electric field at the advancing nose of a dynamo-driven jet, where plasma instability inherent in collisionless runaway acceleration allows ions to be steadily accelerated to energies {approx} V, finally ejected as cosmic rays. Transient events can produce much higher energies. The predicted disk radiation is similar to the Standard Model. Unique predictions concern the remarkable collimation of jets and emissions from the jet/radiolobe structure. Given MBH and the accretion rate, the model makes 7 predictions roughly consistent with data: (1) the jet length; (2) the jet radius; (3) the steady-state cosmic ray energy spectrum; (4) the maximum energy in this spectrum; (5) the UHE cosmic ray intensity on Earth; (6) electron synchrotron wavelengths; and (7) the power in synchrotron radiation. These qualitative successes motivate new computer simulations, experiments and data analysis to provide a quantitative verification of the model.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Models, data, and scripts associated with “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning”

This data package is associated with the publication “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning’’ submitted to the Journal of Geophysical Research: Machine Learning and Computation (Scheibe et al. 2024). River sediment respiration observations are expensive and labor intensive to obtain and there is no physical model for predicting this quantity. The Worldwide Hydrobiogeochemisty Observation Network for Dynamic River Systems (WHONDRS) observational data set (Goldman et al.; 2020) is used to train machine learning (ML) models to predict respiration rates at unsampled sites. This repository archives training data, ML models, predictions, and model evaluation results for the purposes of reproducibility of the results in the associated manuscript and community reuse of the ML models trained in this project. One of the key challenges in this work was to find an optimum configuration for machine learning models to work with this feature-rich (i.e. 100+ possible input variables) data set. Here, we used a two-tiered approach to managing the analysis of this complex data set: 1) a stacked ensemble of ML models that can automatically optimize hyperparameters to accelerate the process of model selection and tuning and 2) feature permutation importance to iteratively select the most important features (i.e. inputs) to the ML models. The major elements of this ML workflow are modular, portable, open, and cloud-based, thus making this implementation a potential template for other applications. This data package is associated with the GitHub repository found at Please see the file level metadata (flmd; “sl-archive-whondrs_flmd.csv”) for a list of all files contained in this data package and descriptions for each. Please see the data dictionary (dd; “sl-archive-whondrs_dd.csv”) for a list of all column headers contained within comma separated value (csv) files in this data package and descriptions for each. The GitHub repository is organized into five top-level directories: (1) “input_data” holds the training data for the ML models; (2) “ml_models” holds machine learning models trained on the data in “input_data”; (3) “scripts” contains data preprocessing and postprocessing scripts and intermediate results specific to this data set that bookend the ML workflow; (4) “examples” contains the visualization of the results in this repository including plotting scripts for the manuscript (e.g., model evaluation, FPI results) and scripts for running predictions with the ML models (i.e., reusing the trained ML models); (5) “output_data” holds the overall results of the ML model on that branch. Each trained ML model resides on its own branch in the repository; this means that inputs and outputs can be different branch-to-branch. Furthermore, depending on the number of features used to train the ML models, the preprocessing and postprocessing scripts, and their intermediate results, can also be different branch-to-branch. The “main-*” branches are meant to be starting points (i.e. trunks) for each model branch (i.e. sprouts). Please see the Branch Navigation section in the top-level README.md in the GitHub repository for more details. There is also one hidden directory “.github/workflows”. This hidden directory contains information for how to run the ML workflow as an end-to-end automated GitHub Action but it is not needed for reusing the ML models archived here. Please the top-level README.md in the GitHub repository for more details on the automation.

13C↗

DOE OSTI2015

Analytic reconstruction algorithms for triple-source CT with horizontal data truncation

Purpose: This paper explores a triple-source imaging method with horizontal data truncation to enlarge the field of view (FOV) for big objects. Methods: The study is conducted by using theoretical analysis, mathematical deduction, and numerical simulations. The proposed algorithms are implemented in c + + and MATLAB. While the basic platform is constructed in MATLAB, the computationally intensive segments are coded in c + +, which are linked via a MEX interface. Results: A triple-source circular scanning configuration with horizontal data truncation is developed, where three pairs of x-ray sources and detectors are unevenly distributed on the same circle to cover the whole imaging object. For this triple-source configuration, a fan-beam filtered backprojection-type algorithm is derived for truncated full-scan projections without data rebinning. The algorithm is also extended for horizontally truncated half-scan projections and cone-beam projections in a Feldkamp-type framework. Using their method, the FOV is enlarged twofold to threefold to scan bigger objects with high speed and quality. The numerical simulation results confirm the correctness and effectiveness of the developed algorithms. Conclusions: The triple-source scanning configuration with horizontal data truncation cannot only keep most of the advantages of a traditional multisource system but also cover a larger FOV for big imaging objects. In addition, because the filtering is shift-invariant, the proposed algorithms are very fast and easily parallelized on graphic processing units.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

cuZ-Checker: A GPU-Based Ultra-Fast Assessment System for Lossy Compressions

Lossy compression is becoming an indispensable technique for the success of today's extreme-scale high-performance computing projects that produce vast volumes of data during scientific simulations or instrument data acquisitions. Comprehensively understanding the compression quality and performance of different lossy compressors is critical to selecting the best-fit compressors and using them properly and efficiently in practice. A few lossy compression assessment tools (e.g., Z-checker) have been developed, but none of them support the execution in a GPU environment. This is a significant gap because many recent extreme-scale applications and lossy compressors (e.g., cuSZ) can run entirely within GPUs. In this work, we develop an efficient lossy compression measuring system (called cuZ-Checker) on the GPU platform, which aims to perform the lossy compression quality and performance assessment completely within the GPU environment. Our contribution is threefold. (1) We develop a novel GPU-based lossy compression measuring framework using a computation pattern-based design approach. This approach classifies the computing-intensive metrics into three categories based on their patterns which creates large opportunities for kernel fusion and data reuse. (2) For each pattern in cuZ-Checker, we develop a CUDA kernel and provide fine-grained optimizations to boost its performance. (3) We thoroughly evaluate our cuZ-checker on a V100 GPU using four real-world scientific application datasets. Experiments show that cuZ-Checker can significantly accelerate the overall lossy compression assessment performance by 23X similar to 31X compared with the OpenMP-based multithreading CPU performance. To the best of our knowledge, this is the first lossy compression measuring system designed for GPU devices.

GPU↗

NASA NTRSDate not supplied

Development of a High-Fidelity CLARREO Pathfinder Simulator

CLARREO Pathfinder is a hyperspectral instrument designed to resolve reflectance of solar radiation from Earth in the wavelength range of 350 nm to 2300 nm with an uncertainty of 0.3% (k=2) or less. Its success will demonstrate CLARREO’s ability to provide on-orbit SI-Traceable calibration of measured spectral reflectance with an advanced accuracy. The CLARREO pathfinder simulator has been developed to generate high-fidelity inter-calibration event data products that can be used for pre-launch inter-calibration algorithm study. A high-fidelity simulation for CLARREO inter-calibration events includes computationally intensive radiative transfer calculations for observations within each individual collocation footprint. A principal component based radiative transfer model is incorporated in the simulator to enable an ultra-fast forward simulation, ensuring a low latency data processing for month- to year-long intercalibration events. This paper introduces the design and development of the CLARREO pathfinder simulator and some of its applications for inter-calibration matching error evaluation.

Wan Wu↗