Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data parallelism”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Hierarchical Image Segmentation of Remotely Sensed Data using Massively Parallel GNU-LINUX Software

A hierarchical set of image segmentations is a set of several image segmentations of the same image at different levels of detail in which the segmentations at coarser levels of detail can be produced from simple merges of regions at finer levels of detail. In [1], Tilton, et a1 describes an approach for producing hierarchical segmentations (called HSEG) and gave a progress report on exploiting these hierarchical segmentations for image information mining. The HSEG algorithm is a hybrid of region growing and constrained spectral clustering that produces a hierarchical set of image segmentations based on detected convergence points. In the main, HSEG employs the hierarchical stepwise optimization (HSWO) approach to region growing, which was described as early as 1989 by Beaulieu and Goldberg. The HSWO approach seeks to produce segmentations that are more optimized than those produced by more classic approaches to region growing (e.g. Horowitz and T. Pavlidis, [3]). In addition, HSEG optionally interjects between HSWO region growing iterations, merges between spatially non-adjacent regions (i.e., spectrally based merging or clustering) constrained by a threshold derived from the previous HSWO region growing iteration. While the addition of constrained spectral clustering improves the utility of the segmentation results, especially for larger images, it also significantly increases HSEG s computational requirements. To counteract this, a computationally efficient recursive, divide-and-conquer, implementation of HSEG (RHSEG) was devised, which includes special code to avoid processing artifacts caused by RHSEG s recursive subdivision of the image data. The recursive nature of RHSEG makes for a straightforward parallel implementation. This paper describes the HSEG algorithm, its recursive formulation (referred to as RHSEG), and the implementation of RHSEG using massively parallel GNU-LINUX software. Results with Landsat TM data are included comparing RHSEG with classic region growing.

Tilton, James C.↗
NASA NTRS1988

Adapting high-level language programs for parallel processing using data flow

EASY-FLOW, a very high-level data flow language, is introduced for the purpose of adapting programs written in a conventional high-level language to a parallel environment. The level of parallelism provided is of the large-grained variety in which parallel activities take place between subprograms or processes. A program written in EASY-FLOW is a set of subprogram calls as units, structured by iteration, branching, and distribution constructs. A data flow graph may be deduced from an EASY-FLOW program.

Standley, Hilda M.↗
DOE OSTI2020

Portable interactive visualization of large-scale simulations in geotechnical engineering using Unity3D

Development in large-scale geotechnical engineering simulation places tremendous demand for efficient visualization of such simulation data. This study presents a lightweight software tool, i.e. Geotechnical Interactive Visualization (GIV), as a solution to this challenge, which achieves efficient interactive visualization of large-scale simulations data in geotechnical engineering. Visualization data flow and algorithms specifically optimized for common geotechnical engineering applications are implemented in GIV. GIV can visualize geotechnical structure models with time-varying attributes attached to mesh with fixed topology, and also models with time-varying mesh topologies but no attributes attached, the two most common visualization tasks in geotechnical engineering. Furthermore, challenges for large-scale simulation data visualization, including parallel simulation data redundancy, massive data size, dynamic user interaction, and portability are overcome via specifically designed algorithms for simulation data preprocessing and optimized visualization modules using the powerful 3D rendering and interactive game engine Unity3D. Comparison of GIV with several widely used visualization tools for the visualization of large-scale idealized datasets and realistic geotechnical simulations highlights the visualization efficiency, smooth interactivity, and lightweight features of GIV.

42 ENGINEERING↗
NASA NTRS1999

Data Structure and Parallel Decomposition Considerations on a Fibonacci Grid

The Fibonacci grid, proposed by Swinbank and Purser (see companion abstract), provides attractive properties for global numerical atmospheric prediction by offering an optimally homogeneous, geometrically regular, and approximately isotropic discretization, with only the polar regions requiring special numerical treatment. It is a mathematical idealization, applied to the sphere, of the multi-spiral patterns often found in botanical structures, such as in pine cones and sunflower heads. Computationally, it is natural to organize the domain, into zones, in each of which the same pair, or triple, of "Fibonacci spirals" dominate. But the further subdivision of such zones into "tiles" of a shape and size suitable for distribution to the processors of a massively parallel computer requires very careful consideration if the subsequent spatial computations along the respective spirals, especially those computations (such as compact differencing schemes) that involve recursion, can be implemented in an efficient "load-balanced "manner without requiring excessive amounts of inter-processor communications. In this paper we show how certain "number theoretic" properties of the Fibonacci sequence (whose numbers prescribe the multiplicity of successive spirals) may be exploited in the decomposition of grid zones into tidy arrangements of triangular grid tiles, each tile possessing one side approximately parallel to the constant-latitude zone boundary. We also describe how the spatially recursive processes may be decomposed across such a tiling, and the directionality of the recursions reversed on alternate grid lines, to ensure a very high degree of load balancing throughout the execution of the computations required for one time step of a global model.

Michalakes, John↗
DOE OSTI2025

Parallel sorting algorithm classification: is manual instrumentation necessary?

Understanding parallel algorithms is crucial for accelerating scientific simulations on complex, distributed memory, high-performance computers. Modern algorithm classification approaches learn semantics directly from source code to differentiate between algorithms, however, accessing source code is not always possible. We can learn about parallel algorithms from observing their performance, as programs running the same algorithms and using the same hardware should exhibit similar performance characteristics. We present an approach to learn algorithm classes from parallel performance data directly in order to classify algorithms without access to the source code. We extend previous work to enable classifying parallel sorting algorithms using automatic instrumentation instead of requiring manual region annotations in the source code. In this work, we design and demonstrate a study for classification of parallel sorting algorithms using parallel performance data collected from automatic instrumentation, and evaluate the performance of our new methodology on classification. We leverage Caliper to collect the performance data, Thicket for our exploratory data analysis (EDA), and PyTorch and Scikit-learn to evaluate the effectiveness of random forests, support vector machines (SVMs), decision trees, neural networks, and logistic regressions on parallel performance data. Additionally, we study noise in parallel performance data, whether the removal of noise and pre-processing of the data is necessary to accurately classify parallel sorting algorithms, and determine the effectiveness of features created from performance data. In conclusion, we demonstrate classification accuracy for these five different models of up to 97.7% across four different parallel algorithm classes.

Algorithm Classification↗
NASA NTRS2013

Parallel Plate System for Collecting Data Used to Determine Viscosity

A parallel-plate system collects data used to determine viscosity. A first plate is coupled to a translator so that the first plate can be moved along a first direction. A second plate has a pendulum device coupled thereto such that the second plate is suspended above and parallel to the first plate. The pendulum device constrains movement of the second plate to a second direction that is aligned with the first direction and is substantially parallel thereto. A force measuring device is coupled to the second plate for measuring force along the second direction caused by movement of the second plate.

Kaukler, William↗
NASA NTRS2012

Simulated Wake Characteristics Data for Closely Spaced Parallel Runway Operations Analysis

A simulation experiment was performed to generate and compile wake characteristics data relevant to the evaluation and feasibility analysis of closely spaced parallel runway (CSPR) operational concepts. While the experiment in this work is not tailored to any particular operational concept, the generated data applies to the broader class of CSPR concepts, where a trailing aircraft on a CSPR approach is required to stay ahead of the wake vortices generated by a lead aircraft on an adjacent CSPR. Data for wake age, circulation strength, and wake altitude change, at various lateral offset distances from the wake-generating lead aircraft approach path were compiled for a set of nine aircraft spanning the full range of FAA and ICAO wake classifications. A total of 54 scenarios were simulated to generate data related to key parameters that determine wake behavior. Of particular interest are wake age characteristics that can be used to evaluate both time- and distance- based in-trail separation concepts for all aircraft wake-class combinations. A simple first-order difference model was developed to enable the computation of wake parameter estimates for aircraft models having weight, wingspan and speed characteristics similar to those of the nine aircraft modeled in this work.

Guerreiro, Nelson M.↗
DOE OSTI2023

LLNL/thicket

Thicket is a python-based toolkit for Exploratory Data Analysis (EDA) of parallel performance data that enables performance optimization and understanding of applications’ performance on supercomputers. It bridges the performance tool gap between being able to consider only a single instance of a simulation run (e.g., single platform, single measurement tool, or single scale) and finding actionable insights in multi-dimensional, multi-scale, multi-architecture, and multi-tool performance datasets.

Brink, Stephanie Labasan↗
NASA NTRS1986

Data reduction, management, and analysis software for CID

In an overview of the Data Reduction System, three major steps are examined. First, the raw data tapes were selected from the onboard recorders. These tapes should provide the best quality data for the data reduction software system. These tapes contained 352 channels of data, plus the monitor channels recorded in 8 bit Pulsed Coded Modulation (PCM) words. The next step consists of transcribing the PCM tapes from 8 bit serial digital data to 8 bit parallel digital data. This puts the data in the correct format for processing. The transcription process was accomplished here at LaRC in the Central Data Transportation Facility (CDTF). The last step in this 3 step process is to process the data through the reduction system developed for the Impact Dynamic Research Facility in the early part of 1980. Processing system criteria, system interface routines, and engineering units program that reads digitized data from tapes, and file management programs are discussed.

Davis, C. W.↗
DOE OSTI2023

A Ceph S3 Object Data Store for HEP

We present a novel data format design that obviates the need for data tiers by storing individual event data products in column objects. The objects are stored and retrieved through Ceph S3 technology, with a layout designed to minimize metadata volume and maximize data processing parallelism. Performance benchmarks of data storage and retrieval are presented.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1968

Parallel-to-serial biphase-data converter

Data converter produces a serial biphase output signal from parallel input data. Alternate bits are loaded into a shift register in complement form so that the bits appear at the end of the shift register in a true-complement form sequence.

Truelove, R. D.↗
NASA NTRS1990

An efficient data dependence analysis for parallelizing compilers

A novel algorithm, called the lambda test, is presented for an efficient and accurate data dependence analysis of multidimensional array references. It extends the numerical methods to allow all dimensions of array references to be tested simultaneously. Hence, it combines the efficiency and the accuracy of the both approaches. This algorithm has been implemented in PARAFRASE, a FORTRAN program parallelization restructurer developed at the University of Illinois at Urbana-Champaign. Some experimental results are presented to show its effectiveness.

Li, Zhiyuan↗
NASA NTRS2002

Emission Line Galaxies in the STIS Parallel Survey: Observations and Data Analysis - 1

In the first three years of operation STIS obtained slitless spectra of approximately 2500 fields in parallel to prime HST observations as part of the STIS Parallel Survey (SPS). The archive contains approximately 300 fields at high galactic latitude (|b| greater than 30) with spectroscopic exposure times greater than 3000 seconds. This sample contains 220 fields (excluding special regions and requiring a consistent grating angle) observed between 6 June 1997 and 21 September 2000, with a total survey area of approximately 160 square arcminutes. At this depth, the SPS detects an average of one emission line galaxy per three fields. We present the analysis of these data, and the identification of 131 low to intermediate redshift galaxies detected by optical emission lines. The sample contains 78 objects with emission lines that we infer to be redshifted [OII]3727 emission at 0.43 < z < 1.7. The comoving number density of these objects is comparable to that of Halpha-emitting galaxies in the NICMOS parallel observations. One quasar and three probable Seyfert galaxies are detected. Many of the emission-line objects show morphologies suggestive of mergers or interactions. The reduced data are available upon request from the authors.

Teplitz, Harry I.↗
DOE OSTI2020

The parallelism motifs of genomic data analysis

Genomic datasets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share these data with the research community, but some of these genomic data analysis problems require large-scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high-end parallel systems today and place different requirements on programming support, software libraries and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high-performance genomics analysis, including alignment, profiling, clustering and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or ‘motifs’ that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

59 BASIC BIOLOGICAL SCIENCES↗