SEARCH · Engineering Papers

Results for “Data parallelism”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

All sources NASA NTRS DOE OSTI

NASA NTRS1982

Multispectral determination of soil moisture-2

Soil moisture data obtained using scatterometers, modular multispectral scanners and passive microwave radiometers were revised and grouped into four field cover types for statistical anaysis. Guymon data are grouped as alfalfa, bare, milo with rows perpendicular to the field view, and milo viewed parallel to the field of view. Dalhart data are grouped as bare combo, stubble, disked stubble, and corn field. Summary graphs combine selected analyses to compare the effects of field cover. The analysis for each of the cover types is presented in tables and graphs. Other tables show elementary statistics, correlation matrices, and single variable regressions. Selected eigenvectors and factor analyses are included and the highest correlating sensor typs for each location are summarized.

Estes, J. E.↗

DOE OSTI2022

Accelerating shared file checkpoint with local burst buffers

A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.

Gooding, Thomas↗

NASA NTRS1991

Parallel Processing System

In order to process very high resolution image data from spacecraft sensors, Goddard Space Flight Center commissioned the development of a Massively Parallel Processor (MPP) based upon simultaneous processing of image picture elements (pixels) rather than serial processing. It resulted in a considerable increase in computational speed. MasPar Computer Corporation's MasPar MP-1 incorporates this technology, allowing users to attack a variety of computationally-intensive problems. The MP-1 is no longer manufactured but has been replaced by the MP-2, a more advanced model.

Source record↗

DOE OSTI2021

DeepGRN: prediction of transcription factor binding site across cell-types using attention-based deep neural networks

Abstract Background Due to the complexity of the biological systems, the prediction of the potential DNA binding sites for transcription factors remains a difficult problem in computational biology. Genomic DNA sequences and experimental results from parallel sequencing provide available information about the affinity and accessibility of genome and are commonly used features in binding sites prediction. The attention mechanism in deep learning has shown its capability to learn long-range dependencies from sequential data, such as sentences and voices. Until now, no study has applied this approach in binding site inference from massively parallel sequencing data. The successful applications of attention mechanism in similar input contexts motivate us to build and test new methods that can accurately determine the binding sites of transcription factors. Results In this study, we propose a novel tool (named DeepGRN) for transcription factors binding site prediction based on the combination of two components: single attention module and pairwise attention module. The performance of our methods is evaluated on the ENCODE-DREAM in vivo Transcription Factor Binding Site Prediction Challenge datasets. The results show that DeepGRN achieves higher unified scores in 6 of 13 targets than any of the top four methods in the DREAM challenge. We also demonstrate that the attention weights learned by the model are correlated with potential informative inputs, such as DNase-Seq coverage and motifs, which provide possible explanations for the predictive improvements in DeepGRN. Conclusions DeepGRN can automatically and effectively predict transcription factor binding sites from DNA sequences and DNase-Seq coverage. Furthermore, the visualization techniques we developed for the attention modules help to interpret how critical patterns from different types of input features are recognized by our model.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Optimizing inference of segmentation on high-resolution images in MLExchange

MLExchange is a machine learning (ML) operations platform providing web user-interfaces (UIs) for data visualization and analysis pipelines at synchrotron facilities. Among these UIs is the segmentation app which helps synchrotron users utilize ML algorithms to automatically segment high-resolution scientific images with minimal manual annotation effort. In this work, we share code optimizations that significantly speed up the segmentation inference workflow of large data in short time. By optimizing the sequence of CPU-GPU data transfers and introducing CPU parallelization to key operations, we improve the per-device, per-image frame computational efficiency and observe close to 3×$$\times$$ speedup over the original segmentation inference workflow run time when utilizing a single GPU. Further adaptations enabling multi-GPU inference yield more than 40×$$\times$$ speedup with 100 GPUs compared to the optimized single GPU inference workflow. This acceleration of the segmentation inference workflow will provide MLExchange users with easy access to segmentation results with little wait time.

Lu, Shizhao↗

DOE OSTI2023

Hyperspectral and Nanosecond Temporal Resolution Widefield Infrared Photothermal Heterodyne Imaging

Label-free, bond-selective imaging offers new opportunities for fundamental and applied studies in chemistry, biology, and materials science. Preventing its broader application to investigating spectrally congested specimens are issues related to low sensitivity as well as low spatial and temporal resolution. Here, we demonstrate a widefield, mid-infrared (MIR) photothermal imaging technique, called widefield Infrared Photothermal Heterodyne imaging (wIR-PHI), that massively parallelizes acquisition of MIR absorption data through use of a high-speed complementary metal-oxide-semiconductor camera. wIR-PHI possesses notable features that include: spatial resolution significantly below the MIR diffraction limit, hyperspectral imaging capabilities, high sensitivity, and ~100 ns temporal resolution. Here, the first two features are highlighted by hyperspectral imaging of proximally close poly(methyl methacrylate) (PMMA) and polystyrene (PS) nanoparticles where clear, bond-specific imaging of nanoparticles, separated by less than the MIR diffraction limit, is demonstrated. Sensitivity is highlighted by imaging individual PMMA and PS nanoparticles with radii between r = 97-556 nm. This leads to a current, peak absorption cross-section limit-of-detection of σ abs = 1.9× 10 –16 cm 2 . wIR-PHI's 100 ns temporal resolution is simultaneously demonstrated by observing the decay of photothermal contrast on individual nanoparticles on a ~200-6000 ns timescale. In whole, wIR-PHI’s dramatic increase in acquisition speed opens opportunities for future MIR kinetic imaging and spectroscopic studies of important chemical, biological, and material processes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

HOSS!

The Hall-D Online Skim System (HOSS) was developed to simultaneously solve two issues for the high intensity GlueX experiment. One was to parallelize the writing of raw data files to disk in order to improve bandwidth. The other was to distribute the raw data across multiple compute nodes in order to produce calibration skims of the data online. The highly configurable system employs RDMA, RAM disks, and zeroMQ driven by Python to simultaneously store and process the full high intensity GlueX data stream.

Lawrence, David↗

DOE OSTI2020

Ghost imaging second harmonic generation microscopy

Second harmonic generation (SHG) microscopy is useful for visualizing interfaces and sub-structures within a wide range of materials due to the propensity for SHG to occur in non-centrosymmetric environments. However, since SHG is a nonlinear process generally necessitating small focal sizes for higher peak powers, a raster scanning approach is usually needed to build an SHG image over a significant sample size. While raster scanning is effective, there is a cost in terms of the time needed to acquire the image and, also, some materials cannot withstand the higher optical intensities within the small focal volume. As such, we describe a SHG microscopy approach based on ghost imaging (GI), which enables imaging data to be collected in parallel rather than sequentially as in raster scanning techniques. We experimentally demonstrate the approach and combine GI-SHG with compressive sensing to make further substantial gains in reducing the amount of sampling required for image reconstruction. Furthermore, GI-SHG is shown to have significant advantages for imaging in highly scattering environments, partly because GI is a background-free approach requiring spatial correlations between photons that travel two paths, with one path entirely devoid of sample interaction. This basic property of GI means that only the photons that travel unimpeded through the sample preserve the spatial correlations needed to reconstruct the image, while more scattered photons do not contribute to the overall GI signal. Finally, we compare the image quality and sampling properties of three different reconstruction algorithms used for compressive sensing.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

A lightweight, user-configurable detector ASIC digital architecture with on-chip data compression for MHz X-ray coherent diffraction imaging

Today, most X-ray pixel detectors used at light sources transmit raw pixel data off the detector ASIC. With the availability of more advanced ASIC technology nodes for scientific application, more digital functionalities from the computing domains (e.g., compression) can be integrated directly into a detector ASIC to increase data velocity. In this paper, we describe a lightweight, user-configurable detector ASIC digital architecture with on-chip compression which can be implemented in 130 nm technologies in a reasonable area on the ASIC periphery. In addition, we present a design to efficiently handle the variable data from the stream of parallel compressors. The architecture includes user-selectable lossy and lossless compression blocks. The impact of lossy compression algorithms is evaluated on simulated and experimental X-ray ptychography datasets. This architecture is a practical approach to increase pixel detector frame rates towards the continuous 1 MHz regime for not only coherent imaging techniques such as ptychography, but also for other diffraction techniques at X-ray light sources.

47 OTHER INSTRUMENTATION↗

DOE OSTI2025

Scaling Ultrahigh-Resolution E3SM Land Model for Leadership-Class Supercomputers

This paper presents advancements in scaling the ultrahigh-resolution E3SM Land Model (uELM) for deployment on leadership-class supercomputers, addressing the increased demand for km-scale Earth system modeling. By focusing on km-scale ELM simulations, we enhance predictive capabilities for climate interactions, facilitating improved responses to climate change impacts on energy systems, agriculture, and water resources. Our approach leverages innovative software architecture optimizations, sophisticated data handling techniques, and advanced parallel processing, achieving strong scalability on two leadership supercomputers (2400 nodes (105,600 cores) on Summit, and 1200 nodes (76,800 cores) on Frontier). Results from extensive scalability assessments on the Summit and Frontier also demonstrate outstanding I/O performance (close to 400 GB/s write throughput) and the model's ability to efficiently handle increasing computational demands. This study not only establishes uELM's capability for high-resolution simulations over vast geographical domains, but also sets a foundation for future Earth system modeling breakthroughs.

Wang, Dali [ORNL] (ORCID:0000000168065108)↗

DOE OSTI2020

A Sparse Tensor Benchmark Suite for CPUs and GPUs

Tensor computations present significant performance chal- lenges that impact a wide spectrum of applications ranging from machine learning, healthcare analytics, social network analysis, data mining to quantum chemistry and signal processing. Efforts to improve the perfor- mance of tensor computations include exploring data layout, execution scheduling, and parallelism in common tensor kernels. This work presents a benchmark suite for arbitrary-order sparse tensor kernels using state- of-the-art tensor formats: coordinate (COO) and hierarchical coordinate (HiCOO) on CPUs and GPUs. It presents a set of reference tensor kernel implementations that are compatible with real-world tensors and power law tensors extended from synthetic graph generation techniques. We also propose Roofline performance models for these kernels to provide insights of computer platforms from sparse tensor view. This benchmark suite along with the synthetic tensor generator is publicly available.

Li, Jiajia↗

DOE OSTI2020

kokkosSZ (kSZ) A Portable Cellerator Implementation of SZ using Kokkos Programming Model

kSZ is a Kokkos-based implementation of the world-widely used SZ lossy compressor. We use Kokkos because it provides abstractions for both parallel execution of code and data management, which can be used to support portable implementation across different accelerator technologies. Kokkos can support OpenMP/OpenMPTarget, oneAPI, Pthreads, and CUDA as backend programming models.

ECP↗

DOE OSTI2021

CUDAMicroBench

CUDAMicroBench is a collection of fourteen microbenchmarks that demonstrate performance challenges in CUDA programming and techniques to optimize the CUDA programs to address these challenges. It also includes examples and techniques for using advanced CUDA features such as data shuffling between threads, dynamic parallelism, etc that can help users optimize the CUDA program for performance.

Liao, Chunhua↗

DOE OSTI2022

ornladios/ADIOS2

ADIOS 2: The Adaptable Input Output (I/O) System version 2 is an open-source framework that addresses scientific data management challenges, e.g. scalable parallel I/O, as we approach the exascale era in high-performance computing (HPC). ADIOS 2 bindings are available in C++, C, Fortran, Python and can be used on supercomputers, personal computers, and cloud systems running on Linux, macOS and Windows. ADIOS 2 has out-of-the-box support for MPI and serial environments.

ECP↗

DOE OSTI2025

Electro-optic sampling of classical and quantum light

Full characterization of electric-field waveforms in amplitude and phase is achieved across the terahertz to visible spectral range through interaction with an optical pulse shorter than a half-cycle period via the Pockels (linear electro-optic) effect. This technique of electro-optic sampling has become an indispensable tool in various areas, including ultrafast pump-probe, time-domain and frequency-comb spectroscopies, quantum optics, high-harmonic generation, and attosecond science, and holds great promise for further advances. Not only does it enable spectroscopic measurements with record dynamic range and temporal resolution, along with massively parallel real-time spectral data acquisition, but its remarkable sensitivity also allows the detection of vacuum fluctuations, i.e., “zero-point motion” of electric fields, profoundly impacting our understanding of the fundamental laws of nature.

Benea-Chelmus, Ileana-Cristina (ORCID:000000024814↗

DOE OSTI2021

VTK-m User's' Guide (V.1.7)

High-performance computing relies on ever finer threading. Advances in processor technology include ever greater numbers of cores, hyperthreading, accelerators with integrated blocks of cores, and special vectorized instructions, all of which require more software parallelism to achieve peak performance. Traditional visualization solutions cannot support this extreme level of concurrency. Extreme scale systems require a new programming model and a fundamental change in how we design algorithms. To address these issues we created VTK-m: the visualization toolkit for multi-/many-core architectures. VTK-m supports a number of algorithms and the ability to design further algorithms through a top-down design with an emphasis on extreme parallelism. VTK-m also provides support for finding and building links across topologies, making it possible to perform operations that determine manifold surfaces, interpolate generated values, and find adjacencies. Although VTK-m provides a simplified high-level interface for programming, its template-based code removes the overhead of abstraction. VTK-m simplifies the development of parallel scientific visualization algorithms by providing a framework of supporting functionality that allows developers to focus on visualization operations. Consider the listings in Figure 1.1 that compares the size of the implementation for the Marching Cubes algorithm in VTK-m with the equivalent reference implementation in the CUDA software development kit. Because VTK-m internally manages the parallel distribution of work and data, the VTK-m implementation is shorter and easier to maintain. Additionally, VTK-m provides data abstractions not provided by other libraries that make code written in VTK-m more versatile.This book includes contributions from the VTK-m community including the VTK-m development team and the user community.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Sierra/SolidMechanics 5.6 User's Manual

Sierra/SolidMechanics (Sierra/SM) is a three-dimensional solid mechanics code with a versatile element library, nonlinear material models, large deformation capabilities, and contact. It is built on the SIERRA Framework. SIERRA provides a data management framework in a parallel computing environment that allows the addition of capabilities in a modular fashion. Contact capabilities are parallel and scalable. This document provides information about the functionality in Sierra/SM and the command structure required to access this functionality in a user input file. This document is divided into chapters based primarily on functionality. For example, the command structure related to the use of various element types is grouped in one chapter; descriptions of material models are grouped in another chapter. Sierra/SM provides both explicit transient dynamics and implicit quasistatics and dynamics capabilities. Both the explicit and implicit modules are highly scalable in a parallel computing environment. In the past, the explicit and implicit capabilities were provided by two separate codes, known as Presto and Adagio, respectively. These capabilities have been consolidated into a single code. The executable is named Adagio, but it provides the full suite of solid mechanics capabilities, for both implicit and explicit. The Presto executable has been disabled as a consequence of this consolidation.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

The VTK-m Users' Guide (V.1.9)

97 MATHEMATICS AND COMPUTING↗