Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “HPC system”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Preconditioned least‐squares Petrov–Galerkin reduced order models

Abstract In this article, we introduce a methodology for improving the accuracy and efficiency of reduced order models (ROMs) constructed using the least‐squares Petrov–Galerkin (LSPG) projection method through the introduction of preconditioning. Unlike prior related work, which focuses on preconditioning the linear systems arising within the ROM numerical solution procedure to improve linear solver performance, our approach leverages a preconditioning matrix directly within the minimization problem underlying the LSPG formulation. Applying preconditioning in this way has the potential to improve ROM accuracy for several reasons. First, preconditioning the LSPG formulation changes the norm defining the residual minimization, which can improve the residual‐based stability constant bounding the ROM solution's error. The incorporation of a preconditioner into the LSPG formulation can have the additional effect of scaling the components of the residual being minimized to make them roughly of the same magnitude, which can be beneficial when applying the LSPG method to problems with disparate scales (e.g., dimensional equations, multi‐physics problems). Importantly, we demonstrate that an “ideal preconditioned” LSPG ROM (a ROM in which the preconditioner is the inverse of the Jacobian of its corresponding full order model) emulates projection of the full order model solution increment onto the reduced basis. This quantity defines a lower bound on the error of a ROM solution for a given reduced basis. By designing preconditioners that approximate the Jacobian inverse—as is common in designing preconditioners for solving linear systems—it is possible to obtain a ROM whose error approaches this lower bound. The proposed approach is evaluated on several mechanical and thermo‐mechanical problems implemented within the Albany HPC code and run in the predictive regime, with prediction across material parameter space. We demonstrate numerically that the introduction of simple Jacobi, Gauss‐Seidel, and ILU preconditioners into the proper orthogonal decomposition/LSPG formulation reduces significantly the ROM solution error, the reduced Jacobian condition number, the number of nonlinear iterations required to reach convergence, and the wall time (thereby improving efficiency). Moreover, our numerical results reveal that the introduction of preconditioning can deliver a robust and accurate solution for test cases in which the unpreconditioned LSPG method fails to converge.

Lindsay, Payton↗

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

The predominance of Kohn–Sham density functional theory (KS-DFT) for the theoretical treatment of large experimentally relevant systems in molecular chemistry and materials science relies primarily on the existence of efficient software implementations which are capable of leveraging the latest advances in modern high-performance computing (HPC). With recent trends in HPC leading toward increasing reliance on heterogeneous accelerator-based architectures such as graphics processing units (GPU), existing code bases must embrace these architectural advances to maintain the high levels of performance that have come to be expected for these methods. In this work, we purpose a three-level parallelism scheme for the distributed numerical integration of the exchange-correlation (XC) potential in the Gaussian basis set discretization of the Kohn–Sham equations on large computing clusters consisting of multiple GPUs per compute node. In addition, we purpose and demonstrate the efficacy of the use of batched kernels, including batched level-3 BLAS operations, in achieving high levels of performance on the GPU. We demonstrate the performance and scalability of the implementation of the purposed method in the NWChemEx software package by comparing to the existing scalable CPU XC integration in NWChem.

97 MATHEMATICS AND COMPUTING↗

TEAM Project Review, Year 2

This report summarizes our research activities within the TEAM project between December 2020 and December 2021, funded by the ASCR Advanced Research in Quantum Computing program. During the reporting period the LLNL-MSU team has made progress on several fronts. An overarching goal of the team is to provide a comprehensive suite of software tools that can be used for the Characterize-Optimize-Compute loop needed to implement and execute algorithms on quantum devices. We are concurrently developing lightweight solvers that can be used on desktop computers to find optimal control pulses and to characterize small quantum systems (consisting of a few transmons and cavities). However, desktop computers are insufficient for simulating and characterizing larger quantum systems. We have therefore also developed parallel, distributed memory, simulators and optimization solvers, both for open and closed quantum systems. These parallel solvers have, for example, been used to study quantum optimal control for pure-state preparation, utilizing 1000’s of cores on a modern high-performance computing (HPC) platform.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Characterizing the Performance of Executing Many-tasks on Summit

Many scientific workloads are comprised of many tasks, where each task is an independent simulation or analysis of data. The execution of millions of tasks on heterogeneous HPC platforms requires scalable dynamic resource management and multi-level scheduling. RADICAL-Pilot (RP) -- an implementation of the Pilot abstraction, addresses these challenges and serves as an effective runtime system to execute workloads comprised of many tasks. In this paper, we characterize the performance of executing many tasks using RP when interfaced with JSM and PRRTE on Summit: RP is responsible for resource management and task scheduling on acquired resource; JSM or PRRTE enact the placement of launching of scheduled tasks. Our experiments provide lower bounds on the performance of RP when integrated with JSM and PRRTE. Specifically, for workloads comprised of homogeneous single-core, 15 minutes-long tasks we find that: PRRTE scales better than JSM for > O(1000) tasks; PRRTE overheads are negligible; and PRRTE supports optimizations that lower the impact of overheads and enable resource utilization of 63% when executing O(16K), 1-core tasks over 404 compute nodes.

Matteo, Turilli↗

A compute-bound formulation of Galerkin model reduction for linear time-invariant dynamical systems

This work aims to advance computational methods for projection-based reduced-order models (ROMs) of linear time-invariant (LTI) dynamical systems. For such systems, current practice relies on ROM formulations expressing the state as a rank-1 tensor (i.e., a vector), leading to computational kernels that are memory bandwidth bound and, therefore, ill-suited for scalable performance on modern architectures. This weakness can be particularly limiting when tackling many-query studies, where one needs to run a large number of simulations. This work introduces a reformulation, called rank-2 Galerkin, of the Galerkin ROM for LTI dynamical systems which converts the nature of the ROM problem from memory bandwidth to compute bound. We present the details of the formulation and its implementation, and demonstrate its utility through numerical experiments using, as a test case, the simulation of elastic seismic shear waves in an axisymmetric domain. We quantify and analyze performance and scaling results for varying numbers of threads and problem sizes. In conclusion, we present an end-to-end demonstration of using the rank-2 Galerkin ROM for a Monte Carlo sampling study. We show that the rank-2 Galerkin ROM is one order of magnitude more efficient than the rank-1 Galerkin ROM (the current practice) and about 970 times more efficient than the full-order model, while maintaining accuracy in both the mean and statistics of the field.

97 MATHEMATICS AND COMPUTING↗

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗

Extending the Publish/Subscribe Abstraction for High-Performance I/O and Data Management at Extreme Scale

The Adaptable I/O System (ADIOS) represents the culmination of substantial investment in Scientific Data Management, and it has demonstrated success for several important extreme-scale science cases. However, looking towards the exascale and beyond, we see the development of yet more stringent data management requirements that require new abstractions. Therefore, there is an opportunity to attempt to connect the traditional realms of HPC I/O optimization with the Database / Data Management community. As such, in this paper we offer some specific examples from our ongoing work in managing data structures, services, and performance at the extreme scale for scientific computing. Using the publish/subscribe model afforded by ADIOS, we demonstrate a set of services that connect data format, metadata, queries, data reduction, and high-performance delivery. The resulting publish/subscribe framework facilitates connection to on-line workflow systems to enable the dynamic capabilities that will be required for exascale science.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Futureproofing through 2035 for the AI and HPC Power Density Trend

HPC and AI are businesses necessitating growth for providing performance improvement with increased power and cooling. As HPC and AI computers and their supporting facilities approach utility scale with a frequency of technology innovation outpacing utility and construction timelines, understanding and designing for this trend has become critical. This paper will provide historical trend data for facilities and compute racks, relate power trend data to cooling technology capabilities, and reason through constraints impacting anticipated future power densities to aid the reader in futureproofing a facility’s power and cooling systems through the mid-2030’s.

97 MATHEMATICS AND COMPUTING↗

Electric Vehicles at Scale (EVs@Scale) Laboratory Consortium Deep-Dive Technical Meetings: High Power Charging (HPC) Summary Report

Electric vehicle (EV) adoption will change the composition of EV charging load to higher-power charging as more medium- (MD) and heavy-duty (HD) applications are electrified, and as all vocations, including light-duty (LD) vehicles, are capable of faster charging. These shifts provide the opportunity for high-power charging (HPC) and facility equipment to evolve and improve efficiency, cost, and space. High-Power Electric Vehicle Charging Hub Integration Platform (eCHIP) project designs and develops a high-power, interoperable charging experimental platform to research, develop, and demonstrate the integration approaches and technology solutions. The project addresses (1) interconnection and management of a grid-tied inverter; (2) development of a DC distribution system that is responsible for system energy management, interoperability, and DC protection; (3) modular DC/DC conversion for vehicle charging; (4) EV charging interface and DC/DC integration; and (5) smart charge control and vehicle-to-edge (vehicle-to-building [V2B], vehicle-to-everything [V2X]) capability. This summary presentation is the first technical progress output of the project. It provides insight for the first deep-dive technical meeting outputs in terms of research presentations and also includes summary of the discussions occurred in the follow-up breakout sessions. The summary presentation covers three technical areas: (1) HPC: State of the art power architectures and the design of the power electronics, (2) Modeling, energy management, and power control in the HPC station, and (3) Codes and standards work that are in line with the previous two topic areas.

ADVANCED PROPULSION SYSTEMS,POWER TRANSMISSION AND↗

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗

Tech-X contributions to the Center for Tokamak Transients (CTTS) Scientific Discovery through Advanced Computing (SciDAC) project (Final Report)

The Center for Tokamak Transients Simulations (CTTS) consists of computational plasma physicists, computer scientists, and applied mathematicians who are together developing and applying simulation software to improve our understanding of and ability to predict and control transient events in tokamak discharges, particularly those that lead to violent disruptions. We have major simulation activities concerning disruptions that are caused by violating ideal MHD stability criteria, vertical displacement events, resistive wall modes, and tearing modes. We are performing comprehensive modeling of disruption mitigation techniques, particularly by shattered pellet injection (SPI). The Center is built around two mature extended MHD codes: NIMROD and M3D-C1. These codes are being extended in a number of ways as required by the demands of disruption modeling, and are being optimized to work well for larger problem size and modern computer architectures as required to model tokamak plasmas with realistic parameters over the required timescales. The codes will be coupled to the FronTier code for state-of-the-art modeling of SPI. Validation with experimental data on DIII-D, NSTX, and JET is being facilitated by rigorous uncertainty quantification (UQ). The high-level objectives of the center are to: (1) Develop better understanding and improved predictive capability of when crossing a linear stability boundary will likely lead to a disruption, (2) Provide a tool for calculating vessel forces, both axisymmetric and non-axisymmetric, for worst case VDE and other disruptions in tokamaks, (3) Develop increased understanding and improved predictive capability for locked mode disruptions and how best to avoid them, (4) Develop, verify, and validate 3D models for disruption mitigation by shattered pellets that can be used to design and optimize a system for future tokamaks, and (5) Increase the efficiency and scalability of the M3D-C1 and NIMROD codes so that they can model disruptions in ITER scale devices using the most powerful DOE HPC hardware, and (6) Demonstrate code-coupling via FronTier coupling with NIMROD and M3D-C1 and by interfacing these codes within a WDM framework.

97 MATHEMATICS AND COMPUTING↗

Distributed Computing for the Project 8 Experiment

The Project 8 collaboration aims to measure the absolute neutrino mass or improve on the current limit by measuring the tritium beta decay electron spectrum. We present the current distributed computing model for the Project 8 experiment. Project 8 is in its second phase of data taking with a near continuous data rate of 1Gbps. The current computing model uses DIRAC (Distributed Infrastructure with Remote Agent Control) for its workflow and data management. A detailed meta-data assignment using the DIRAC File Catalog is used to automate raw data transfers and subsequent stages of data processing. The DIRAC system is deployed on containers managed using a Kubernetes cluster to provide a scalable infrastructure. A modified DIRAC Site Director provides the ability to submit jobs using Singularity on opportunistic High-Performance Computing (HPC) sites.

Distributed Computing, Kubernetes, DIRAC, Project ↗

Electric Vehicles at Scale (EVs@Scale) Laboratory Consortium Deep-Dive Technical Meetings: High Power Charging (HPC) Summary Report

Electric vehicle (EV) adoption will change the composition of EV charging load to higher-power charging as more medium- (MD) and heavy-duty (HD) applications are electrified, and as all vocations, including light-duty (LD) vehicles, are capable of faster charging. These shifts provide the opportunity for high-power charging (HPC) and facility equipment to evolve and improve efficiency, cost, and space. High-Power Electric Vehicle Charging Hub Integration Platform (eCHIP) project designs and develops a high-power, interoperable charging experimental platform to research, develop, and demonstrate the integration approaches and technology solutions. The project addresses (1) interconnection and management of a grid-tied inverter; (2) development of a DC distribution system that is responsible for system energy management, interoperability, and DC protection; (3) modular DC/DC conversion for vehicle charging; (4) EV charging interface and DC/DC integration; and (5) smart charge control and vehicle-to-edge (vehicle-to-building [V2B], vehicle-to-everything [V2X]) capability. This summary presentation is the first technical progress output of the project. It provides insight for the first deep-dive technical meeting outputs in terms of research presentations and also includes summary of the discussions occurred in the follow-up breakout sessions. The summary presentation covers three technical areas: (1) HPC: State of the art power architectures and the design of the power electronics, (2) Modeling, energy management, and power control in the HPC station, and (3) Next Generation Profiles for high power charging characterization.

ADVANCED PROPULSION SYSTEMS↗

dCache: Inter-disciplinary storage system

The dCache project provides open-source software deployed internationally to satisfy ever more demanding storage requirements. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters and long term data persistence on a tertiary storage. Though it was originally developed for the HEP experiments, today it is used by various scientific communities, including astrophysics, biomed, life science, which have their specific requirements. In this paper we describe some of the new requirements as well as demonstrate how dCache developers are addressing them.

Mkrtchyan, Tigran↗

Advances in GDSA Framework Development and Process Model Integration

The Spent Fuel and Waste Science and Technology (SFWST) Campaign of the U.S. Department of Energy (DOE) Office of Nuclear Energy (NE), Office of Spent Fuel & Waste Disposition (SFWD) is conducting research and development (R&D) on geologic disposal of spent nuclear fuel (SNF) and highlevel nuclear waste (HLW). A high priority for SFWST disposal R&D is to develop a disposal system modeling and analysis capability for evaluating disposal system performance for nuclear waste in geologic media. This report describes fiscal year (FY) 2020 advances of the Geologic Disposal Safety Assessment (GDSA) Framework and PFLOTRAN development groups of the SFWST Campaign. The common mission of these groups is to develop a geologic disposal system modeling capability for nuclear waste that can be used to probabilistically assess the performance of disposal options and generic sites. The capability is a framework called GDSA Framework that employs high-performance computing (HPC) capable codes PFLOTRAN and Dakota.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Carnac for Emulytics (HPC Annual Report V.1.0)

Carnac, located at Sandia's California site, is an institutional cluster for Emulytics that provides security researchers with resources to model enterprise computer networks and evaluate how resilient they are from attacks. While multiple Emulytics cluster computers have been built at Sandia, Carnac is the first system that was developed as an institutional resource that can be shared among different groups with disparate requirements.

97 MATHEMATICS AND COMPUTING↗

Machine Learning for First Principles Calculations of Material Properties for Ferromagnetic Materials

The investigation of finite temperature properties using Monte-Carlo (MC) methods requires a large number of evaluations of the system’s Hamiltonian to sample the phase space needed to obtain physical observables as function of temperature. DFT calculations can provide accurate evaluations of the energies, but they are too computationally expensive for routine simulations. To circumvent this problem, machine-learning (ML) based surrogate models have been developed and implemented on high-performance computing (HPC) architectures. In this paper, we describe two ML methods (linear mixing model and HydraGNN) as surrogates for first principles density functional theory (DFT) calculations with classical MC simulations. These two surrogate models are used to learn the dependence of target physical properties from complex compositions and interactions of their constituents. We present the predictive performance of these two surrogate models with respect to their complexity while avoiding the danger of overfitting the model. An important aspect of our approach is the periodic retraining with newly generated first principles data based on the progressive exploration of the system’s phase space by the MC simulation. The numerical results show that HydraGNN model attains superior predictive performance compared to the linear mixing model for magnetic alloy materials.

Eisenbach, Markus↗