Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel programming patterns”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Metadata compression

A computer-executable method, system, and computer program product for managing a Parallel-Logged File System (PLFS) on a data storage system, wherein the data storage system is in communication with an application that updates the data storage system with checkpoint data, the computer-executable method, system, and computer program product comprising receiving a first checkpoint including data from the application; analyzing the data in the first checkpoint to determine whether a pattern exists in the checkpoint data; and upon a positive determination, adding the pattern into PLFS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Exploring MPI Communication Models for Graph Applications Using Graph Matching as a Case Study

Traditional implementations of parallel graph operations on distributed memory platforms are written using Message Passing Interface (MPI) point to point communication primitives such as Send-Recv (blocking and nonblocking). Apart from this classical model, the MPI community has over the years added other communication models; however, their suitability for handling the irregular traffic workloads typical of graph operations remain comparatively less explored. Our aim in this paper is to study these relatively underutilized communication models of MPI for graph applications. More specifically, we evaluate MPI’s one-sided programming, or Remote Memory Access (RMA), and nearest neighborhood collectives using a process graph topology. There are features in these newer models that are intended to better map to irregular communication patterns, as exemplified in graph algorithms. As a concrete application for our case study, we use distributed memory implementations of a weighted graph matching algorithm to investigate performances of MPI-3 RMA and neighborhood collective operations compared to nonblocking Send- Recv. A matching in a graph is a subset of edges such that no two matched edges are incident on the same vertex. A maximum weight matching is a matching of maximum weight computed as the sum of the weights of matched edges. Algorithms for optimal matching have quadratic complexity and are difficult to parallelize. More importantly, execution of graph matching is dominated by high volume of irregular memory accesses, making it an ideal candidate for studying the effects of various MPI communication models on graph applications at scale. Our neighborhood collectives and RMA implementations yields up to 6× speedup over traditional optimized nonblocking Send-Recv implementations, on thousands of cores of the NERSC Cori supercomputer. We believe the lessons learned from this case study can be adopted to benefit a wider range of graph application codes.

Ghosh, Sayan↗
DOE OSTI2017

Aspen-based performance and energy modeling frameworks

Now with the anticipation of exascale architectures, energy consumption is becoming one of the critical design parameters, especially in light of the energy budget of 20-30 Megawatts set by the U.S. Department of Energy. Understanding an application’s execution pattern and its energy footprint is critical to improving application operation on a diverse heterogeneous architecture. Applying application-specific performance optimization can consequently improve energy consumption. However, this approach is only applicable to current systems. As we enter a new era of exascale architectures that is projected to contain more complex memory hierarchies, increased levels of parallelism, heterogeneity in hardware, and complex programming models and techniques, energy and performance management is getting more cumbersome. We therefore propose techniques that predict the energy consumption beforehand or at runtime to enable proactive tuning. Such energy prediction approaches must be generic and adapt themselves at runtime to changing application and hardware configurations. Most existing energy estimation and prediction approaches are empirical in nature and thus tied to current systems. To overcome this limitation, we propose two energy estimation techniques: ACEE (Algorithmic and Categorical Energy Estimation), which uses a combination of analytical and empirical modeling techniques; and AEEM (Aspen’s Embedded Energy Estimation), a system-level analytical energy estimation technique. Both of these models incorporate the Aspen domain specific language for performance modeling. We present the methodologies of these two models and test their accuracy using five proxy applications. We also describe three use cases.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

Highlights of X-Stack ExM Deliverable Swift/T

Swift/T is a key success from the ExM: System support for extreme-scale, many-task applications1 X-Stack project, which proposed to use concurrent dataflow as an innovative programming model to exploit extreme parallelism in exascale computers. The Swift/T component of the project reimplemented the Swift language from scratch to allow applications that compose scientific modules together to be build and run on available petascale computers (Blue Gene, Cray). Swift/T does this via a new compiler and runtime that generates and executes the application as an MPI program. We assume that mission-critical emerging exascale applications will be composed as scalable applications using existing software components, connected by data dependencies. Developers wrap native code fragments using a higherlevel language, then build composite applications to form a computational experiment. This exemplifies hierarchical concurrency: lower-level messaging libraries are used for fine-grained parallelism; highlevel control is used for inter-task coordination. These patterns are best expressed with dataflow, but static DAGs (i.e., other workflow languages) limit the applications that can be built; they do not provide the expressiveness of Swift, such as conditional execution, iteration, and recursive functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Combining RAVEN, RELAP5-3D, and PHISICS for Fuel Cycle and Core Design Analysis for New Cladding Criteria

The Nuclear Regulatory Commission (NRC) has considered revision of 10 CFR 50.46C rule to account for the effects of burn-up rate in future analysis of reactor accident scenarios so that safety margins may evolve with the operation and reloading of the reactor as dynamic limits. In order to find these limiting conditions, both cladding oxidation and maximum temperature must be cast as functions of fuel exposure. It is computationally intensive to run a plant model through a long duration operational transient to the time of fuel reload, and this must be repeated for each reload until the time of the accident scenario. Moreover, fuel reload patterns may differ in many ways so that calculations with many different core configurations must be considered. To perform such new analyses in a reasonable amount of computational time with good accuracy, Idaho National Laboratory (INL) has developed new multi-physics tools by combining existing codes and adding new capabilities. The Parallel Highly Innovative Simulation INL Code System (PHISICS) toolkit for neutronic and reactor physics is coupled with the Reactor Excursion and Leak Analysis Program (RELAP5-3D) for the Loss of Coolant Accident (LOCA) analysis and RAVEN for the Probabilistic Risk Assessment (PRA) and margin characterization analysis. Expansion of code capability was also necessary. For RELAP5-3D to process a single sequence of cores in a continuous run required modification of RELAP5-3D to restart a sequence of input decks, each with a slightly modified neutronics model or thermal-hydraulic flow region and culminating in an accident scenario. A new multi-deck input processing capability was developed and verified for this analysis. In conclusion, the combined RAVEN/PHISICS/RELAP5-3D tool is used to analyze a typical Pressurized Water Reactor (PWR).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

Interactive Web Application for Traffic Simulation Data Management and Visualization

As traffic simulation software becomes more effective for realistically simulating and analyzing traffic dynamics and vehicle interactions on the mesoscopic and microscopic level, the management, dissemination, and collaborative visualization of traffic simulation results produced by individual transportation planners presents a significant challenge. Existing online content management systems have a very limited capability in allowing users to query specific traffic simulation scenarios and geospatially visualize simulation results through shareable and interactive web interfaces. This paper presents a web-based application for promoting the archiving, sharing, and visualization of large-scale traffic simulation outputs. The application is developed to enhance cyber-physical controls, communications, and public education for collaborative transportation planning. Unique features of the web application include: (a) allowing users to upload their new traffic simulation scenarios (parameters and outputs), as well as search existing scenarios using easily accessible interfaces; (b) optimizing simulation output files with heterogeneous data formats and projected coordinate systems for web-based storage and management using a scalable and searchable data/metadata standard; (c) standardizing user-uploaded simulation outputs using web interfaces and data processing libraries with parallel computing capacity; and (d) providing shareable web visual interfaces for visualizing the traffic flow and signal information stored in simulation outputs (e.g., regional traffic patterns and individual vehicle interactions) and visually comparing multiple simulation outputs both spatially and temporally. Furthermore, the paper presents the conceptual design and implementation of this application, and demonstrates the application’s performance for sharing, comparing, and visualizing simulation outputs from VISSIM and SUMO, two commonly used traffic simulation software programs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Analyzing the Energy and Power Consumption of Remote Memory Accesses in the OpenSHMEM Model

PGAS models like OpenSHMEM provide interfaces to explicitly initiate one-sided remote memory accesses among processes. In addition, the model also provides synchronizing barriers to ensure a consistent view of the distributed memory at different phases of an application. The incorrect use of such interfaces affects the scalability achievable while using a parallel programming model. This study aims at understanding the effects of these constructs on the energy and power consumption behavior of OpenSHMEM applications. Our experiments show that cost incurred in terms of the total energy and power consumed depends on multiple factors across the software and hardware stack. We conclude that there is a significant impact on the power consumed by the CPU and DRAM due to multiple factors including the design of the data transfer patterns within an application, the design of the communication protocols within a middleware, the architectural constraints laid by the interconnect solutions, and also the levels of memory hierarchy within a compute node. This work motivates treating energy and power consumption as important factors while designing compute solutions for current and future distributed systems.

Jana, Siddhartha↗
DOE OSTI2014

Ordering Traces Logically to Identify Lateness in Parallel Programs

Event traces are a valuable tool for understanding the behavior of parallel programs. Automatically analyzing a large trace, however, especially without a specific objective, is difficult. We aid this process by extracting a trace’s logical structure, an ordering of events derived from their happened before relationships. From this perspective, we can calculate delays relative to peers rather than duration and determine where they begin and how they propagate. The logical structure also acts as a platform for comparing and clustering processes as well as showing communication patterns in a trace visualization. We present an algorithm for determining this idealized logical structure from a trace and develop metrics to quantify delays and differences among processes. We implement this in our novel trace visualizer, Ravel, and apply our approach to several applications, demonstrating the accuracy of our extracted structure and its utility in analyzing these codes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Analyzing the Energy and Power Consumption of Remote Memory Accesses in the OpenSHMEM Model

PGAS models like OpenSHMEM provide interfaces to explicitly initiate one-sided remote memory accesses among processes. In addition, the model also provides synchronizing barriers to ensure a consistent view of the distributed memory at different phases of an application. The incorrect use of such interfaces affects the scalability achievable while using a parallel programming model. This study aims at understanding the effects of these constructs on the energy and power consumption behavior of OpenSHMEM applications. Our experiments show that cost incurred in terms of the total energy and power consumed depends on multiple factors across the software and hardware stack. We conclude that there is a significant impact on the power consumed by the CPU and DRAM due to multiple factors including the design of the data transfer patterns within an application, the design of the communication protocols within a middleware, the architectural constraints laid by the interconnect solutions, and also the levels of memory hierarchy within a compute node. This work motivates treating energy and power consumption as important factors while designing compute solutions for current and future distributed systems.

Jana, Siddhartha↗
DOE OSTI2013

SDS: A Framework for Scientific Data Services

Large-scale scientific applications typically write their data to parallel file systems with organizations designed to achieve fast write speeds. Analysis tasks frequently read the data in a pattern that is different from the write pattern, and therefore experience poor I/O performance. In this paper, we introduce a prototype framework for bridging the performance gap between write and read stages of data access from parallel file systems. We call this framework Scientific Data Services, or SDS for short. This initial implementation of SDS focuses on reorganizing previously written files into data layouts that benefit read patterns, and transparently directs read calls to the reorganized data. SDS follows a client-server architecture. The SDS Server manages partial or full replicas of reorganized datasets and serves SDS Clients' requests for data. The current version of the SDS client library supports HDF5 programming interface for reading data. The client library intercepts HDF5 calls and transparently redirects them to the reorganized data. The SDS client library also provides a querying interface for reading part of the data based on user-specified selective criteria. We describe the design and implementation of the SDS client-server architecture, and evaluate the response time of the SDS Server and the performance benefits of SDS.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2015

Experiences in extending parallware to support OpenACC. In: WACCPD '15 Proceedings of the Second Workshop on Accelerator Programming using Directives, Article No. 4

Porting scientific codes to accelerator-based computers using OpenACC and OpenMP is an important topic for the HPC community. Programmability, performance portability and developer productivity are key issues for the widespread use of these systems. In the scope of general-purpose parallel computing, Parallware is a new commercial OpenMP-enabling source-to-source compiler that automatically adds OpenMP capabilities in scientific programs. Thus, extending Parallware with OpenACC or OpenMP 4.x support would contribute to improve programmability and developer productivity. In contrast, the performance portability of such approach needs to be demonstrated in practice. This paper presents a preliminary study to extend Parallware with OpenACC support for GPU devices. A simple benchmark suite has been designed to mimic important features and computational patterns of real scientific applications. Handcoded OpenACC versions are compared to OpenMP versions automatically generated by Parallware. Performance is evaluated with the PGI OpenACC compiler on systems accelerated with NVIDIA GPUs.

Lobeiras, Jacobo↗
DOE OSTI2015

A Multiwavenumber Theory for Eddy Diffusivities and Its Application to the Southeast Pacific (DIMES) Region

A multiwavenumber theory is formulated to represent eddy diffusivities. It expands on earlier single-wavenumber theories and includes the wide range of wavenumbers encompassed in eddy motions. In the limiting case in which ocean eddies are only composed of a single wavenumber, the multiwavenumber theory is equivalent to the single-wavenumber theory and both show mixing suppression by the eddy propagation relative to the mean flow. The multiwavenumber theory was tested in a region of the Southern Ocean (70°–45°S, 110°–20°W) that covers the Drake Passage and includes the tracer/float release locations during the Diapycnal and Isopycnal Mixing Experiment in the Southern Ocean (DIMES). Cross-stream eddy diffusivities and mixing lengths were estimated in this region from the single-wavenumber theory, from the multiwavenumber theory, and from floats deployed in a global K 0 ° Parallel Ocean Program (POP) simulation. Compared to the single-wavenumber theory, the horizontal structures of cross-stream mixing lengths from the multiwavenumber theory agree better with the simulated float-based estimates at almost all depth levels. The multiwavenumber theory better represents the vertical structure of cross-stream mixing lengths both inside and outside the Antarctica Circumpolar Current (ACC). Finally, both the single-wavenumber and multiwavenumber theories represent the horizontal structures of cross-stream diffusivities, which resemble the eddy kinetic energy patterns.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Mechanisms of heat flux across the Southern Greenland continental shelf in 1/10° and 1/12° ocean/sea ice simulations

The increased presence of warm Atlantic water on the Greenland continental shelf has been connected to the accelerated melting of the Greenland Ice Sheet, particularly in the southwest and southeast shelf regions. Results from two high-resolution coupled ocean-sea ice simulations that utilized either the 1/10-degree Parallel Ocean Program (POP) or the 1/12-degree HYbrid Coordinate Ocean Model (HYCOM) are used to understand the flux of heat on and off the southern Greenland shelf. The analysis reveals that the region of greatest heat flux onto the shelf is southeast Greenland. On the southwestern shelf, heat is mainly exported from the shelf to the interior basins. We identify differences in the shelf break current structure and on-shelf heat content between the two simulations. Just south of the Denmark strait, there is a seasonally persistent pattern of multi-day variability in the cross-shelf heat flux in both simulations. In the POP simulation, this high-frequency signal results in net on-shore heat flux. In the HYCOM simulation, the signal is weaker and results in net off-shelf heat flux. This variability is consistent with Denmark Strait Overflow eddies traveling along the shelf break.

58 GEOSCIENCES↗
DOE OSTI2015

Design and testing of the first 2D Prototype Vertically Integrated Pattern Recognition Associative Memory

An associative memory-based track finding approach has been proposed for a Level 1 tracking trigger to cope with increasing luminosities at the LHC. The associative memory uses a massively parallel architecture to tackle the intrinsically complex combinatorics of track finding algorithms, thus avoiding the typical power law dependence of execution time on occupancy and solving the pattern recognition in times roughly proportional to the number of hits. This is of crucial importance given the large occupancies typical of hadronic collisions. The design of an associative memory system capable of dealing with the complexity of HL-LHC collisions and with the short latency required by Level 1 triggering poses significant, as yet unsolved, technical challenges. For this reason, an aggressive R&D program has been launched at Fermilab to advance state of-the-art associative memory technology, the so called VIPRAM (Vertically Integrated Pattern Recognition Associative Memory) project. The VIPRAM leverages emerging 3D vertical integration technology to build faster and denser Associative Memory devices. The first step is to implement in conventional VLSI the associative memory building blocks that can be used in 3D stacking, in other words, the building blocks are laid out as if it is a 3D design. In this paper, we report on the first successful implementation of a 2D VIPRAM demonstrator chip (protoVIPRAM00). The results show that these building blocks are ready for 3D stacking.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Determining Compression Characteristics of Honeycomb Material - 19674

The objective is to determine compression test characteristics of stainless steel honeycomb material to be able to represent honeycomb structures accurately in analytical models used to simulate hypothetical accident scenarios of shipping packages. Honeycomb is a material used primary in the aerospace industry due to its high strength-to-weight ratio. Because it doesn't have a shelf life and can withstand high heats, it is an excellent candidate for a structural material in package designs. Honeycomb has orthotropic material properties. The honeycomb currently being evaluated is made of metal ribbons spot welded together to form hexagon pattern between metal plates. The hexagon ribbons are brazed to the metal plates. The complexity of the honeycomb significantly lengthens the simulation time, which is further complicated when the brazing, welding, and imperfections of the material is considered. This means to affectively represent the honeycomb in simulation programs, such as Abaqus, the material needs to be approximated as a uniform orthotropic material. This requires material properties in each of the three directions, T, W, and L. The T direction is defined as the 'strong' direction, perpendicular to the honeycomb sheet. The L direction is parallel the ribbon and the W direction is perpendicular to the ribbon. Honeycomb has 3 different compression stages. Stage 1 is the initial compression where the honeycomb maintains its structural integrity and does not permanently deform from loads from normal operations. Stage 2 is when the initial buckling causes irreversible damage to the honeycomb. Stage 2 is the one we are most interested in because it absorbs the most energy from a hypothetical accident scenario. During Stage 2, the honeycomb fails layer-by-layer, indicating that the more layers, the longer material crushing is sustained, and thus the more energy that is absorbed. Stage 3 is final compression, similar to compressing solid metal. During stage 3 the stress increases with a diminishing rate of elongation and the honeycomb is completely failed where the plates between the honeycomb core sandwich the crushed honeycomb ribbon. The stress-strain graph bellow shows compression test of 3 different samples in the T direction. The red lines divide the three stages. The far left is Stage 1, the middle stage 2, and the right stage 3. The data collected is displayed in the table below. The expressions in the table represent the regression formula that represents each stage of the compression on a stress-strain graph. Stage 1 is assumed to intersect with the origin; data for stage 3 for the crush in W and L directions where unable to be gathered due to the nature of failure for those directions. Stage 1 and 2 are linear regressions while stage 3 is a degree 2, to best match the curve. Linear regression is used on stage 2 because ultimately that represents the energy absorbed. With more data, a degree n x 2 regression would be a more appropriate regression for stage 2, where n is the number of layers. The table below is the approximation of each scenario and stage. Each stage starts and ends at the intersection of the next stage. There where two main objectives for these tests: firstly to understand how honeycomb performs under extreme compression, and secondly to be able to numerically represent the honeycomb structure. Both of these where completed to varying degrees. It is important understood how the honeycomb fails. If it is crushed in the T direction it retains its integrity even after being crushed, however when crushed in the L and W crush direction, if it fails, it disintegrates, and loses all integrity. Also, the more layers the more time the material spends in stage 2. Failure is started by buckling, thus if there is any imperfection, the stress will not spike but transition straight into stage 2. The data collected and aggregated can be used for initial simulation of honeycomb used in packages. The initial testing has set the ground work for more data to be collected in order to verify results and to allow more confidence in the simulation results. This is only the initial data collected. The next steps is to continue to collect more data to verify results and to test more variants of honeycomb, with different brazing, layers, and shape.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗
DOE OSTI2023

High-throughput feedback-enabled optogenetic stimulation and spectroscopy in microwell plates

Abstract The ability to perform sophisticated, high-throughput optogenetic experiments has been greatly enhanced by recent open-source illumination devices that allow independent programming of light patterns in single wells of microwell plates. However, there is currently a lack of instrumentation to monitor such experiments in real time, necessitating repeated transfers of the samples to stand-alone analytical instruments, thus limiting the types of experiments that could be performed. Here we address this gap with the development of the optoPlateReader (oPR), an open-source, solid-state, compact device that allows automated optogenetic stimulation and spectroscopy in each well of a 96-well plate. The oPR integrates an optoPlate illumination module with a module called the optoReader, an array of 96 photodiodes and LEDs that allows 96 parallel light measurements. The oPR was optimized for stimulation with blue light and for measurements of optical density and fluorescence. After calibration of all device components, we used the oPR to measure growth and to induce and measure fluorescent protein expression in E. coli . We further demonstrated how the optical read/write capabilities of the oPR permit computer-in-the-loop feedback control, where the current state of the sample can be used to adjust the optical stimulation parameters of the sample according to pre-defined feedback algorithms. The oPR will thus help realize an untapped potential for optogenetic experiments by enabling automated reading, writing, and feedback in microwell plates through open-source hardware that is accessible, customizable, and inexpensive.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

ATDM LANL FleCSI: Topology and Execution Framework

FleCSI is a compile-time configurable C++ framework designed to support multi-physics application development. As such, FleCSI attempts to provide a very general set of infrastructure design patterns that can be specialized and extended to suit the needs of a broad variety of solver and data requirements. This means that FleCSI is potentially useful to many different ECP projects. Current support includes multidimensional mesh topology, mesh geometry, and mesh adjacency information, n-dimensional hashed-tree data structures, graph partitioning interfaces, and dependency closures (to identify data dependencies between distributed-memory address spaces). FleCSI introduces a functional programming model with control, execution, and data abstractions that are consistent with state-of-the-art task-based runtimes such as Legion and Charm++. The model also provides support for fine-grained, data-parallel execution with backend support for runtimes such as OpenMP and C++17. The FleCSI abstraction layer provides the developer with insulation from the underlying runtimes, while allowing support for multiple runtime systems, including conventional models like asynchronous MPI. The intent is to give developers a concrete set of user-friendly programming tools that can be used now, while allowing flexibility in choosing runtime implementations and optimizations that can be applied to architectures and runtimes that arise in the future. This project is essential to the ECP Ristra Next-Generation Code project, part of ASC ATDM, because it provides a hierarchically parallel programming model that is consistent with the design of modern system architectures, but which allows for the straightforward expression of algorithmic parallelism in a portably performant manner.

97 MATHEMATICS AND COMPUTING↗