Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “High Performance Computers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Earth Science Deep Learning: Applications and Lessons Learned

Deep learning has revolutionized computer vision and natural language processing with various algorithms scaled using high-performance computing. At the NASA Marshall Space Flight Center (MSFC), the Data Science and Informatics Group (DSIG) has been using deep learning for a variety of Earth science applications. This paper provides examples of the applications and also addresses some of the challenges that were encountered.

Training↗
NASA NTRS2021

The CHIMAERA System for Retrievals of Cloud Top, Optical and Microphysical Properties from Imaging Sensors

Continuity and consistency of geophysical retrieval products obtained from different Earth-observing spaceborne or airborne atmospheric multispectral imagers can be challenging due to inherent differences in the instruments and/or the use of different retrieval algorithms. The Cross-platform HIgh resolution Multi-instrument AtmosphEric Retrieval Algorithms (CHIMAERA) system addresses the latter aspect of the inter-sensor continuity problem for cloud property retrievals by removing retrieval methodology and implementation as a source of inconsistency when applied to instruments that share common measurement capabilities. Transferring an existing retrieval algorithm to a new sensor oftentimes is a nontrivial task, as it is common for an algorithm code to be tightly coupled to the sensor for which it was developed. By creating a clear division between the science algorithm and the instrument I/O codes, CHIMAERA allows easy migration of science algorithms to different sensors. CHIMAERA is built from C and FORTRAN source code, and can operate in a variety of environments ranging from a personal laptop to a high-performance computing environment for near real-time satellite data production. It is highly adaptable, low-maintenance and allows for easy expansion such that adding new instruments into the system requires only instrument-specific I/O and provision of any external lookup tables specific to the instrument's spectral characteristics. CHIMAERA currently supports 14 spaceborne and airborne atmospheric imagers from a single code base, and has been in use since 2007. In this paper we describe the engineering aspects of CHIMAERA and briefly discuss a few examples from its many current applications.

Galina (Gala) Wind↗
DOE OSTI2020

In-Depth Optimization with the OpenACC-to-FPGA Framework on an Arria 10 FPGA

The reconfigurable computing paradigm that uses field programmable gate arrays (FPGAs) has received renewed interest in the high-performance computing field due to FPGAs' unique combination of performance and energy efficiency. However, difficulties in programming and optimizing FPGAs have prevented them from being widely accepted as general-purpose computing devices. In accelerator-based heterogeneous computing, portability across diverse heterogeneous devices is also an important issue, but the unique architectural features in FPGAs make this difficult to achieve. To address these issues, a directive-based, high-level FPGA programming and optimization framework was previously developed. In this work, developed optimizations were combined holistically using the directive-based approach to show that each individual benchmark requires a unique set of optimizations to maximize performance. The relationships between FPGA resource usages and runtime performance were also explored.

Lambert, Jacob↗
NASA NTRS2005

10 Gigabit Ethernet Performance on SGI Altix and Origin Systems

As the state of high performance computing continues to advance, the size of datasets continue to grow, driving a need for high bandwidth data networks. family of networks. 10 Gigabit Ethernet is the latest step in the popular Ethernet We have evaluated the S2io Xframe 10 Gigabit Ethernet adapter on 512p SGI Altix systems running ProPack 3, and Origin systems running Irix 6.5.24 and 6.5.26 in our production supercomputing environment. We encountered a number of performance and stability issues, which were promptly dealt with by SGI and S2io. Using nttcp we tested TCP performance for single and multiple streams, and we tested file transfer using NFS and bbftp. We will present the results of our testing, including the effects of various tuning options on throughput and CPU utilization, and offer suggestions for configuring and tuning S2io 10 Gigabit Ethernet cards in an Altix/Linux or Origin/Irix environment.

Meyer, Andy↗
DOE OSTI2024

3D modeling of the Staged Z-pinch with the FLASH code

The Staged Z-pinch (SZP) fusion concept is a magneto-inertial compression scheme developed by Magneto-Inertial Fusion Technologies, Inc. (MIFTI), in which small amounts of fusion fuel are brought to fusion-relevant conditions by passing multi-million amperes strong current through a cylindrical shell of high atomic number material. One- and two-dimensional modeling performed by MIFTI with the MACH2 code, suggests that net fusion energy gain can be achieved when currents in the 10 million amperes range compress a 50%-50% mixture of deuterium and tritium gas. In this project we enhance and use the FLASH code to execute high-fidelity, simulations of various SZP configurations, a collaboration between the Flash Center for Computational Science at the University of Rochester and the Laboratory for Laser Energetics, and MIFTI. FLASH is a high-performance computing, multi-physics, radiation magnetohydrodynamic (MHD) code with extended physics capabilities, which is developed at the Flash Center. The goal of this project is to assess the shell/fuel stability of the pinch to two- and three-dimensional MHD instabilities and to utilize FLASH’s extended physics capabilities to understand how extended-MHD effects impact implosion dynamics and plasma conditions at stagnation. The project is a natural extension of an ongoing collaborative effort between MIFTI and the Flash Center through the U.S. DOE Advanced Research Projects Agency-Energy (ARPA-E) BETHE program and can provide MIFTI with simulation capabilities that are currently beyond their reach with MACH2. Ultimately, MIFTI wants FLASH to become one of their simulation workhorses for reliable, high-fidelity SZP platform design. During the project, the Flash Center team developed a suite of in one- and two-dimensional FLASH simulations to model different variants of the SZP platform, performed code-to-code comparisons with MACH2, and used the FLASH code to design and model SZP experiments like the Double Eagle experiments that MIFTI recently performed. Also, the Flash Center team developed the capabilities of FLASH to be able to do three-dimensional simulations of pulsed-power experiments with the code for the first time. Two publications from this effort are currently under review and two more are in preparation. MIFTI has also committed to the use of FLASH for future SZP simulation efforts.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1993

NASA Langley Research Center's distributed mass storage system

There is a trend in institutions with high performance computing and data management requirements to explore mass storage systems with peripherals directly attached to a high speed network. The Distributed Mass Storage System (DMSS) Project at NASA LaRC is building such a system and expects to put it into production use by the end of 1993. This paper presents the design of the DMSS, some experiences in its development and use, and a performance analysis of its capabilities. The special features of this system are: (1) workstation class file servers running UniTree software; (2) third party I/O; (3) HIPPI network; (4) HIPPI/IPI3 disk array systems; (5) Storage Technology Corporation (STK) ACS 4400 automatic cartridge system; (6) CRAY Research Incorporated (CRI) CRAY Y-MP and CRAY-2 clients; (7) file server redundancy provision; and (8) a transition mechanism from the existent mass storage system to the DMSS.

Pao, Juliet Z.↗
DOE OSTI2024

Shifting Between Compute and Memory Bounds: A Compression-Enabled Roofline Model

In the evolving landscape of high-performance computing, especially to fight the end of Moore’s Law and Dennard’s Scaling, the ability to shift between compute-bound and memory-bound states is critical for enhancing adaptability and flexibility to diverse system and domain-specific architectures. Such capability is vital for optimizing performance across distinguished hardware configurations, such as accelerators, memory hierarchies, and cache systems. Despite that ad hoc optimization techniques, such as compressed/approximate computation, have been enabled for compute-/data-intensive computing for improved performance in distinct hardware settings, there lacks an understanding of 1) the rational behind performance improvement; 2) capability of different optimizations; 3) what optimization to respond to specific computational and memory demands. This work proposes a compression-enabled roofline model to facilitate this adaptability with data compression techniques to balance and transform between computational and memory demands. This model enables applications to adjust in response to the specific strengths and limitations of the underlying hardware and system to optimize resource utilization. The effectiveness of this approach is demonstrated with matrix multiplication kernels on different input sizes, with turning on/off various compression techniques, including 1) low-precision floating point; 2) sparse matrix formulation; and 3) compressed arrays with ZFP. By reducing memory transfer volumes and cache misses and increasing data locality and computational intensity through compression, the specific roofline model can transform between compute and memory bounds to align more efficiently with system capabilities. This advancement not only improves overall performance but also maximizes adaptability in diverse computing environments.

Naraparaju, Ramasoumya [University of Washington]↗
DOE OSTI2024

CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. In conclusion, we demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes.

Algorithmic portability↗
DOE OSTI2022

Data-model files associated with the manuscript "The Effects of Spatial and Temporal Resolution of Gridded Meteorological Forcing on Watershed Hydrological Responses" (Shuai et al., 2022 HESS)

This data package contains the model inputs and outputs used in "The Effects of Spatial and Temporal Resolution of Gridded Meteorological Forcing on Watershed Hydrological Responses" (Shuai et al., 2022 HESS). The data.zip file contains the data used to drive the model simulations. The model.zip file contains the XML input file for ATS. The notebook.zip file contains the Jupyter notebooks for pre- and post- processing model results. The figures.zip file contains the raw figures associated with the manuscript.Meteorological forcing plays a critical role in accurately simulating the watershed hydrological cycle. With the advancement of high-performance computing and the development of integrated watershed models, simulating the watershed hydrological cycle at high temporal (hourly to daily) and spatial resolution (10s of meters) has become efficient and computationally affordable. These hyperresolution watershed models require high resolution of meteorological forcing as model input to ensure the fidelity and accuracy of simulated responses. In this study, we utilized the Advanced Terrestrial Simulator (ATS), an integrated watershed model, to simulate surface and subsurface flow and land surface processes using unstructured meshes at the Coal Creek Watershed near Crested Butte (Colorado). We compared simulated watershed hydrologic responses including streamflow, and distributed variables such as evapotranspiration, snow water equivalent (SWE), and groundwater table driven by three publicly available, gridded meteorological forcings (GMFs) -- Daily Surface Weather and Climatological Summaries (Daymet), Parameter-elevation Regressions on Independent Slopes Model (PRISM), and North American Land Data Assimilation System (NLDAS). By comparing various spatial resolutions (ranging from 400 m to 4 km) of PRISM, the simulated streamflow only becomes marginally worse when spatial resolution of meteorological forcing is coarsened to 4 km (or 30% of the watershed area). However, the 4 km resolution has much worse performance than finer resolution in spatially distributed variables such as SWE. Using temporally disaggregated PRISM, we compared models forced by different temporal resolutions (hourly to daily), sub-daily resolution preserves the dynamic watershed responses (e.g., diurnal fluctuation of streamflow) that are absent in results forced by daily resolution. Conversely, the simulated streamflow shows better performance using daily resolution compared to that using sub-daily resolution. Our findings suggest that the choice of GMF and its spatiotemporal resolution depends on the quantity of interest and its spatial and temporal scale, which may have important implications on model calibration and watershed management decisions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

The effects of spatial and temporal resolution of gridded meteorological forcing on watershed hydrological responses

Abstract. Meteorological forcing plays a critical role in accurately simulating the watershed hydrological cycle. With the advancement of high-performance computing and the development of integrated watershed models, simulating the watershed hydrological cycle at high temporal (hourly to daily) and spatial resolution (tens of meters) has become efficient and computationally affordable. These hyperresolution watershed models require high resolution of meteorological forcing as model input to ensure the fidelity and accuracy of simulated responses. In this study, we utilized the Advanced Terrestrial Simulator (ATS), an integrated watershed model, to simulate surface and subsurface flow and land surface processes using unstructured meshes at the Coal Creek Watershed near Crested Butte (Colorado). We compared simulated watershed hydrologic responses including streamflow and distributed variables such as evapotranspiration, snow water equivalent (SWE), and groundwater table driven by three publicly available, gridded meteorological forcings (GMFs) – Daily Surface Weather and Climatological Summaries (Daymet), the Parameter-elevation Regressions on Independent Slopes Model (PRISM), and the North American Land Data Assimilation System (NLDAS). By comparing various spatial resolutions (ranging from 400 m to 4 km) of PRISM, the simulated streamflow only becomes marginally worse when spatial resolution of meteorological forcing is coarsened to 4 km (or 30 % of the watershed area). However, the 4 km-resolution has much worse performance than finer resolution in spatially distributed variables such as SWE. Using the temporally disaggregated PRISM, we compared models forced by different temporal resolutions (hourly to daily), and sub-daily resolution preserves the dynamic watershed responses (e.g., diurnal fluctuation of streamflow) that are absent in results forced by daily resolution. Conversely, the simulated streamflow shows better performance using daily resolution compared to that using sub-daily resolution. Our findings suggest that the choice of GMF and its spatiotemporal resolution depends on the quantity of interest and its spatial and temporal scale, which may have important implications for model calibration and watershed management decisions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Computational Seismology Workshop Trains Early-Career Scientists

The Computational Infrastructure for Geodynamics (CIG), in collaboration with Lawrence Livermore National Laboratory (LLNL), held a workshop last year that focused on computational seismology. The workshop provided training in seismic waveform processing, visualization, and high-performance computing (HPC) waveform simulation. Fifty-five predominantly early-career participants—graduate students and postdocs—from the United States and 16 other countries attended. The workshop was the first of its kind to feature full access to HPC resources for research-grade example problems.

58 GEOSCIENCES↗
DOE OSTI2022

Scientific machine learning benchmarks

Deep learning has transformed the use of machine learning technologies for the analysis of large experimental datasets. In science, such datasets are typically generated by large-scale experimental facilities, and machine learning focuses on the identification of patterns, trends and anomalies to extract meaningful scientific insights from the data. In upcoming experimental facilities, such as the Extreme Photonics Application Centre (EPAC) in the UK or the international Square Kilometre Array (SKA), the rate of data generation and the scale of data volumes will increasingly require the use of more automated data analysis. Furthermore, at present, identifying the most appropriate machine learning algorithm for the analysis of any given scientific dataset is a challenge due to the potential applicability of many different machine learning frameworks, computer architectures and machine learning models. Historically, for modelling and simulation on high-performance computing systems, these issues have been addressed through benchmarking computer applications, algorithms and architectures. Extending such a benchmarking approach and identifying metrics for the application of machine learning methods to open, curated scientific datasets is a new challenge for both scientists and computer scientists. Here, we introduce the concept of machine learning benchmarks for science and review existing approaches. As an example, we describe the SciMLBench suite of scientific machine learning benchmarks.

42 ENGINEERING↗
DOE OSTI2022

Accelerating Scientific Workflows on HPC Platforms with In Situ Processing

Scientific workflows drive most modern large-scale science breakthroughs by allowing scientists to define their computations as a set of jobs executed in a given order based on their data dependencies. Workflow management systems (WMSs) have become key to automating scientific workflows-executing computational jobs and orchestrating data transfers between those jobs running on complex high-performance computing (HPC) platforms. Traditionally, WMSs use files to communicate between jobs: a job writes out files that are read by other jobs. However, HPC machines face a growing gap between their storage and compute capabilities. To address that concern, the scientific community has adopted a new approach called in situ, which bypasses costly parallel filesystem I/O operations with faster in-memory or in-network communications. When using in situ approaches, communication and computations can be interleaved. In this work, we leverage the Decaf in situ dataflow framework to accelerate task-based scientific workflows managed by the Pegasus WMS, by replacing file communications with faster MPI messaging. We propose a new execution engine that uses Decaf to manage communications within a sub-workflow (i.e., set of jobs) to optimize inter-job communications. We consider two workflows in this study: (i) a synthetic workflow that benchmarks and compares file- and MPI-based communication; and (ii) a realistic bioinformatics workflow that computes mu-tational overlaps in the human genome. Experiments show that in situ communication can improve the bioinformatics workflow execution time by 22% to 30% compared with file communication. Our results motivate further opportunities and challenges for bridging traditional WMSs with in situ frameworks.

Decaf↗
DOE OSTI2017

rvGAHP: push-based job submission using reverse SSH connections. In: WORKS '17 Proceedings of the 12th Workshop on Workflows in Support of Large-Scale Science, Article No. 3

Computational science researchers running large-scale scientific workflow applications often want to run their workflows on the largest available compute systems to improve time to solution. Workflow tools used in distributed, heterogeneous, high performance computing environments typically rely on either a push-based or a pull-based approach for resource provisioning from these compute systems. However, many large clusters have moved to two-factor authentication for job submission, making traditional automated push-based job submission impossible. On the other hand, pull-based approaches such as pilot jobs may lead to increased complexity and a reduction in node-hour efficiency. In this paper, we describe a new, efficient approach based on HTCondor-G called reverse GAHP (rvGAHP) that allows us to push jobs using reverse SSH submissions with better efficiency than pull-based methods. We successfully used this approach to perform a large probabilistic seismic hazard analysis study using SCEC's CyberShake workflow in March 2017 on the Titan Cray XK7 hybrid system at Oak Ridge National Laboratory.

Callaghan, Scott↗
DOE OSTI2019

Umpire: Application-Focused Management and Coordination of Complex Hierarchical Memory

Advanced architectures like Sierra provide a wide range of memory resources that must often be carefully controlled by the user. These resources have varying capacities, access timing rules, and visibility to different compute resources. Applications must intelligently allocate data in these spaces, and depending on the total amount of memory required, applications may also be forced to move data between different parts of the memory hierarchy. Finally, applications using multiple packages must coordinate effectively to ensure that each package can use the memory resources it needs. To address these challenges, we present Umpire, an application-oriented library for managing memory resources. Specifically, Umpire provides support for querying memory resources, provisioning and allocating memory, and memory introspection. It allows computer scientists and computational physicists to efficiently program the memory hierarchies of current and future high-performance computing architectures, without tying their application to specific hardware or software. In this paper, we describe the design and implementation of Umpire and present case studies from the integration of Umpire into applications that are currently running on Sierra.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Visualization at exascale: Making it all work with VTK-m

The VTK-m software library enables scientific visualization on exascale-class supercomputers. Exascale machines are particularly challenging for software development in part because they use GPU accelerators to provide the vast majority of their computational throughput. Algorithmic designs for GPUs and GPU-centric computing often deviate from those that worked well on previous generations of high-performance computers that relied on traditional CPUs. Fortunately, VTK-m provides scientific visualization algorithms for GPUs and other accelerators. VTK-m also provides a framework that simplifies the implementation of new algorithms and adds a porting layer to work across multiple processor types. This paper describes the main challenges encountered when making scientific visualization available at exascale. Here, we document the surprises and obstacles faced when moving from pre-exascale platforms to the final exascale designs and the performance on those systems including scaling studies on Frontier, an exascale machine with over 37,000 AMD GPUs. We also report on the integration of VTK-m with other exascale software technologies. Finally, we show how VTK-m helps scientific discovery for applications such as fusion and particle acceleration that leverage an exascale supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

2020 ETI Annual Summer School: Data Science and Engineering

The Consortium for Enabling Technologies & Innovation (ETI) was established in 2019 to address emerging technologies within the context of nuclear nonproliferation. ETI creates a research and education environment to support cross-cutting technologies across three core disciplines: 1) computer and engineering science research specifically in a form of machine learning and high performance computing (HPC), 2) advanced manufacturing, and 3) nuclear detection technologies. For outreach and development, ETI hosted the first of three summer schools from August 24-28, 2020 with the theme of “Data Science and Engineering”. The school was hosted in an on-line format and had over 200 participants. The recorded content is available on-line as a resource for students. The summer school had four modules: 1) Fundamentals of data Applications, 2) Computational Machine Learning, 3) Bayesian Modeling and Inference, and 4) Data Science for Safeguards. Modules contained both lectures as well as student exercises. Poll Everywhere was utilized in some modules as an on-line method to engage large groups of students. Upcoming ETI Summer Schools include Novel Instrumentation in 2021 and Advanced Manufacturing in 2022.

Biegalski, Steven R.↗
DOE OSTI2018

Opal: A Centralized Memory Manager for Investigating Disaggregated Memory Systems

Many modern applications have memory footprints that are increasingly large, driving system memory capacities higher and higher. Moreover, these systems are often organized where the bulk of the memory is collocated with the compute capability, which necessitates the need for message passing APIs to facilitate information sharing between compute nodes. Due to the diversity of applications that must run on High-Performance Computing (HPC) systems, the memory utilization can fluctuate wildly from one application to another. And, because memory is located in the node, maintenance can become problematic because each node must be taken offline and upgraded individually. To address these issues, vendors are exploring disaggregated, memory-centric, systems. In this type of organization, there are discrete nodes, reserved solely for memory, which are shared across many compute nodes. Due to their capacity, low-power, and non-volatility, Non-Volatile Memories (NVMs) are ideal candidates for these memory nodes. This report discusses a new component for the Structural Simulation Toolkit (SST), Opal, that can be used to study the impact of using NVMs in a disaggregated system in terms of performance, security, and memory management. This page intentionally left blank.

97 MATHEMATICS AND COMPUTING↗