Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “HPC workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

223 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

US Department of Energy, Office of Science, High-Performance Computing Facility: 2023 Operational Assessment Oak Ridge Leadership Computing Facility

The Oak Ridge Leadership Computing Facility (OLCF) was established to accelerate scientific discovery by providing world-leading computational performance and advanced data infrastructure. As a US Department of Energy (DOE) Office of Science user facility, the OLCF has managed the successful deployment and operation of a succession of leadership-class resources dedicated to open science. In addition to these resources, the OLCF staff continually strive to develop innovative processes and technologies, improve security, and empower users through effective allocation management and comprehensive user support and training. These efforts support the advancement of science by the OLCF users and benefit high-performance computing (HPC) facilities around the world. In calendar year (CY) 2023, the OLCF supported 1,676 users and 598 projects and exceeded all targets for user satisfaction. The facility received an average satisfaction score of 4.52 out of 5 on the annual user survey, and 94% of respondents reported a high satisfaction rate with the OLCF overall. Of the 3,619 user tickets submitted in CY 2023, OLCF staff resolved 97% within 3 business days. The facility opened Frontier to full scientific operations this year. Two projects conducted on Frontier received the Association for Computing Machinery (ACM) Gordon Bell Prize and the Gordon Bell Special Prize for Climate Modeling, and a third earned a nomination as a Gordon Bell Prize finalist. The facility’s previous flagship machine, Summit, gained new life and was extended through 2024 in part to help provide resources to the Integrated Research Infrastructure (IRI) projects and the National Artificial Intelligence Research Resource (NAIRR) pilot program. The facility instantiated an Advanced Computing Ecosystem testbed in part to support IRI workflows. OLCF made interactivity easier and more accessible to users than ever through tools like Jupyter notebooks and workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗
DOE OSTI2025

Quantum Computing Technology Roadmaps and Capability Assessment for Scientific Computing - An analysis of use cases from the NERSC workload

The National Energy Research Scientific Computing Center (NERSC), as the high-performance computing (HPC) facility for the Department of Energy’s Office of Science, recognizes the essential role of quantum computing in its future mission. In this report, we analyze the NERSC workload and identify materials science, quantum chemistry, and high-energy physics as the science domains and application areas that stand to benefit most from quantum computers. These domains jointly make up over 50% of the current NERSC production workload, which is illustrative of the impact quantum computing could have on NERSC’s mission going forward. We perform an extensive literature review and determine the quantum resources required to solve classically intractable problems within these science domains. This review also shows that the quantum resources required have consistently decreased over time due to algorithmic improvements and a deeper understanding of the problems. At the same time, public technology roadmaps from a collection of ten quantum computing companies predict a dramatic increase in capabilities over the next five to ten years. Our analysis reveals a significant overlap emerging in this time frame between the technological capabilities and the algorithmic requirements in these three scientific domains. We anticipate that the execution time of large-scale quantum workflows will become a major performance parameter and propose a simple metric, the Sustained Quantum System Performance (SQSP), to compare system-level performance and throughput for a heterogeneous workload.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Simplifying Geospatial Workflows with GeoGridFusion

Growing demands to understand PV deployment and reliability across an expanding range of climates, along with increasing computational power, are driving the need for simplified computing tools that support large-scale analysis and geospatial workflows. While existing open-source libraries and PV system modeling tools offer extensive collections of empirical and physical models, they often lack the ability to scale to large geospatial datasets. The tool presented here, GeoGridFusion, enables PV modelers to store and utilize gridded geospatial data from sources such as NSRDB, PVGIS, and others. GeoGridFusion harmonizes diverse datasets, taxonomies, and nomenclatures, and includes utilities that support intuitive geospatial area selection.

14 SOLAR ENERGY↗
DOE OSTI2020

Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS - Version 0.21 (20200601)

The continental flood inundation mapping (CFIM) framework is a high-performance computing (HPC)-based computational framework for the Height Above Nearest Drainage (HAND)-based inundation mapping methodology. Using the 10m Digital Elevation Model (DEM) data produced by U.S. Geological Survey (USGS) 3DEP (the 3-D Elevation Program) and the NHDPlus hydrography dataset produced by USGS and the U.S. Environmental Protection Agency (EPA), a hydrological terrain raster called HAND is computed for HUC6 units in the conterminous U.S. (CONUS). The value of each raster cell in HAND is an approximation of the relative elevation between the cell and its nearest water stream. Derived from HAND, a hydraulic property table is established to calculate river geometry properties for each of the 2.7 million river reaches covered by NHDPlus (5.5 million kilometers in total length). This table is a lookup table for water depth given an input stream flow value. Such lookup is available between water depth 0m and 25m at 1-foot interval. The flood inundation map is then computed by using HAND and this lookup table based on the near real-time water forecast from the National Water Model (NWM) at the National Oceanic and Atmospheric Administration (NOAA). HAND and the Hydraulic Property Table version v0.21 is an upgrade to v0.2. Version 0.21 applies NHD HR etching to DEMs for all HUC6 units before running the HAND workflow. This new version also fixed the issue of void areas along HUC boundary. This dataset comprises 331 HUC6 units for CONUS (excluding the five great lakes units), each is a downloadable zip file. Version 0.1 was computed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urban-Champaign in 2016 and is currently hosted at the Texas Advanced Computing Center (TACC). Please cite this data DOI and the following publications and data DOIs when you use HAND and the hydraulic property table: Liu, Yan Y. Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS - Version 0.2. (20200301). Oak Ridge Leadership Computing Facility. (2020). DOI: 10.13139/ORNLNCCS/1608331 Liu, Yan Y., David R. Maidment, David G. Tarboton, Xing Zheng, and Shaowen Wang. 'A CyberGIS integration and computation framework for high-resolution continental-scale flood inundation mapping.' JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 770-784. DOI: 10.1111/1752-1688.12660 Zheng, Xing, David G. Tarboton, David R. Maidment, Yan Y. Liu, and Paola Passalacqua. 'River channel geometry and rating curve estimation using height above the nearest drainage.' JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 785-806. DOI: 10.1111/1752-1688.12661 Liu, Y. (2018). Height Above Nearest Drainage (HAND) for CONUS, HydroShare, https://doi.org/10.4211/hs.69f7d237675c4c73938481904358c789 LICENSE FOR USE -- MAPS AND DATA DISCLAIMER This resource is shared under the Creative Commons Attribution CC BY, http://creativecommons.org/licenses/by/4.0/ MAPS AND DATA DISCLAIMER The Oak Ridge National Laboratory (ORNL) shall not be held liable for improper or incorrect use of the data described or information contained on this map or associated series of maps. The data and related map graphics are not legal, land survey or engineering documents and are not intended to be used as such. ORNL gives no warranty, express or implied, as to the accuracy, reliability, utility or completeness of this information. The user of these maps and data assumes all responsibility and risk for the use of the maps and data. ORNL disclaims all warranties, representations or endorsements either express or implied, with regard to the information contained in this map product, including, but not limited to, all implied warranties of merchantability, fitness for a particular purpose or non-infringement. This preliminary map product is for research and review purposes only. It is not intended to be used for emergency management operational or life safety decisions at the local or regional governmental level or by the general public. Users requiring information regarding hazardous conditions or meteorological conditions for specific geographic areas should consult directly with their city or county emergency management office.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A Co-design Framework for Online Data Analysis and Reduction

Science applications preparing for the exascale era are increasingly exploring in situ computations comprising of simulation-analysis-reduction pipelines coupled in-memory. Efficient composition and execution of such complex pipelines for a target platform is a codesign process that evaluates the impact and tradeoffs of various application- and system-specific parameters. In this article, we describe a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for composing parametric studies on coupled applications, and Savanna, a runtime engine for orchestrating and executing campaigns of codesign experiments. Furthermore, this toolset facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage versus compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design of Cheetah and Savanna, and present application examples that use this framework to conduct codesign studies on small clusters as well as leadership class supercomputers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Simulation-based characterization of the variability of earthquake risk to buildings in the near-field

Recent advancements in high performance computing platforms and computational workflow for regional-scale simulations are enabling unprecedented modeling of fault-to-structure earthquake processes. Regional simulations resolving ground motions at frequencies relevant to engineered systems are becoming computationally viable and provide a new capability to improve understanding of the geographical distribution and intensity of risk to buildings and critical infrastructure. As computational capabilities advance, it is essential to move beyond illustrative single rupture realizations for scenario earthquake events towards the development of a full suite of rupture realizations that appropriately characterize the range of risk to building systems. The work described in this article investigates the application of a suite of fault rupture realizations with the objective of assessing near-fault, site-specific seismic demand variability for building structures. A representative high-performance regional-scale computational model is utilized to execute ground motion and building response simulations based on 18 kinematic rupture realizations of an M7 strike-slip scenario earthquake. The fault rupture models for the scenario earthquake are created by systematically perturbing the hypocenter location and stochastically generating rupture parameters (slip, rise time, rake angle) to represent a breadth of ground motion intensities resulting from the spatial and temporal variabilities of an earthquake rupture process. The resulting seismic demand variability for three-story (short period) and forty-story (long period) steel moment-resisting frame buildings is characterized in terms of the median and distribution of peak inter-story drift ratio for a range of near-fault sites. The full suite of 18 fault rupture realizations and approximately 280,000 nonlinear dynamic building simulations indicate that the three-story building undergoes higher median seismic demand and significantly greater variability of demand at a given site than the forty-story building, which has important implications for the level of certainty in predicting building performance during an earthquake. The simulations performed provide deeper insight into the relationship between fault rupture parameterization and building response, which is essential information for developing a representative suite of rupture realizations for specific earthquake scenarios.

58 GEOSCIENCES↗