Parallel Programming Models and Systems for High Performance Computing
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Abstract not provided.
ANSYS Inc. develops and markets engineering simulation software and services used in the aerospace, automotive, manufacturing, electronics, biomedical, energy, defense, and many other industries. ANSYS is dedicated to engineering simulation and is the world’s leading software provider. ANSYS was founded in 1970 and is headquartered in Canonsburg, Pennsylvania. ANSYS provides an engineering analysis tool combining structural, thermal, computational fluid dynamics, acoustic, and electromagnetic simulation capabilities. ANSYS has two main programs, which use the same solvers: (1) Mechanical APDL (ANSYS Design Parametric Language), a Fortran-based coding platform, and (2) ANSYS Workbench, which uses a graphical user interface to aid in finite element analysis implementation. This plan covers both APDL and Workbench. The ANSYS computer program is a large-scale, multipurpose finite element program that can be used to solve several classes of engineering analyses. The analysis capabilities of ANSYS include the ability to solve static and dynamic structural analyses, steady-state and transient heat transfer problems, mode-frequency and buckling eigenvalue problems, static or time-varying magnetic analyses, and various types of field and coupled-field applications. The program contains many special features that allow nonlinearities or secondary effects such as plasticity, large strain, hyperelasticity, creep, swelling, large deflections, contact, stress stiffening, temperature dependency, material anisotropy, and radiation to be included in the solution. As ANSYS has been developed, other special capabilities such as substructuring, submodeling, random vibration, kinetostatics, kinetodynamics, free convection fluid analysis, acoustics, magnetics, piezoelectrics, coupled-field analysis, and design optimization have been added to the program. These capabilities contribute further to making ANSYS a multipurpose analysis tool for varied engineering disciplines. The ANSYS program has been in commercial use for over 50 years and has been used extensively in the aerospace, automotive, construction, electronic, energy services, manufacturing, nuclear, plastics, oil, and steel industries. Additionally, many consulting firms and hundreds of universities have used ANSYS for analysis, research, and educational purposes. ANSYS is recognized worldwide as one of the most widely used and capable programs of its type. Ansys design analysis software is the first created within a quality system with ISO 9001 certification, the internationally accepted quality standard. Product development, testing, maintenance and support processes also meet the United States Nuclear Regulatory Commission's quality requirements, as they have for nearly four decades. The Quality Assurance Service Agreement is suitable for the customers working in the nuclear industry who need to meet specific federal regulations including 10CRF50 Appendix B and provisions of 10CFR21. ANSYS has retained its original International Organization for Standardization (ISO) 9001 accreditation certificate since1995-05-04, It’s current certificate is valid until 2027-05-29.
Developing quantum chemistry programs on the coming generation of exascale computers will be a difficult task. The programs will need to be fault-tolerant and minimize the use of global operations. This work explores the use a task-based model that uses a data-centric approach to allocate work to different processes as it applies to quantum chemistry. After introducing the key problems that appear when trying to parallelize a complicated quantum chemistry method such as coupled-cluster theory, we discuss the implications of that model as it pertains to the computational kernel of a coupled-cluster program - matrix multiplication. Also, we discuss the extensions that would required to build a full coupled-cluster program using the task-based model. Current programming models for high-performance computing are fault-intolerant and use global operations. Those properties are unsustainable as computers scale to millions of CPUs; instead one must recognize that these systems will be hierarchical in structure, prone to constant faults, and global operations will be infeasible. The FAST-OS HARE project is introducing a scale-free computing model to address these issues. This model is hierarchical and fault-tolerant by design, allows for the clean overlap of computation and communication, reducing the network load, does not require checkpointing, and avoids the complexity of many HPC runtimes. Development of an algorithm within this model requires a change in focus from imperative programming to a data-centric approach. Quantum chemistry (QC) algorithms, in particular electronic structure methods, are an ideal test bed for this computing model. These methods describe the distribution of electrons in a molecule, which determine the properties of the molecule. The computational cost of these methods is high, scaling quartically or higher in the size of the molecule, which is why QC applications are major users of HPC resources. The complexity of these algorithms means that MPI alone is insufficient to achieve parallel scaling; QC developers have been forced to use alternative approaches to achieve scalability and would be receptive to radical shifts in the programming paradigm. Initial work in adapting the simplest QC method, Hartree-Fock, to this the new programming model indicates that the approach is beneficial for QC applications. However, the advantages to being able to scale to exascale computers are greatest for the computationally most expensive algorithms; within QC these are the high-accuracy coupled-cluster (CC) methods. Parallel coupledcluster programs are available, however they are based on the conventional MPI paradigm. Much of the effort is spent handling the complicated data dependencies between the various processors, especially as the size of the problem becomes large. The current paradigm will not survive the move to exascale computers. Here we discuss the initial steps toward designing and implementing a CC method within this model. First, we introduce the general concepts behind a CC method, focusing on the aspects that make these methods difficult to parallelize with conventional techniques. Then we outline what is the computational core of the CC method - a matrix multiply - within the task-based approach that the FAST-OS project is designed to take advantage of. Finally we outline the general setup to implement the simplest CC method in this model, linearized CC doubles (LinCC).
This document describes the theoretical foundation of thermal analysis in Sierra Mechanics. The SIERRA Multimechanics Module: Aria, henceforth referred to as Aria for brevity, was developed at Sandia National Laboratories under the ASC program, and approximates linear and nonlinear continuum models of heat transfer. Aria uses the SIERRA Framework, which provides data management services commonly required by computational mechanics software, and facilitates the development of coupled, multi-mechanics applications for massively parallel computers. The mathematical models in Aria are based heavily on those of COYOTE, a well-established thermal analysis program that was also developed at Sandia and its ASC code predecessor, Calore. Aria, Calore and COYOTE share a significant body of numerical methods, which are described in detail by Reddy and Gartling. Throughout this document, the terms software and implementation are synonymous with the Aria thermal-fluid analysis computer program.
The Energy Sciences Network (ESnet) is the Office of Science’s high-performance network user facility, delivering highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the U.S. Department of Energy (DOE) science mission by connecting each and every DOE lab and its user facilities. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) Program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet connects DOE national laboratories, user facilities, and major experiments so scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large data sets, and access distributed data repositories. While ESnet provides network connectivity, it cannot be characterized as an internet service provider as it is specifically built to provide a range of network services that are tailored to meet the unique requirements of DOE’s data-intensive science.
HPC systems now exploit GPUs within their compute nodes to accelerate program performance. As a result, high-end application development has become extremely complex at the node level. In addition to restructuring the node code to exploit the cores and specialized devices, the programmer may need to choose a programming model such as OpenMP or CPU threads in conjunction with an accelerator programming model to share and manage the difference node resources. This comes at a time when programmer productivity and the ability to produce portable code has been recognized as a major concern. In order to offset the high development cost of creating CUDA or OpenCL kernels, directives have been proposed for programming accelerator devices, but their implications are not well known. In this paper, we evaluate the state of the art accelerator directives to program several applications kernels, explore transformations to achieve good performance, and examine the expressiveness and performance penalty of using high-level directives versus CUDA. We also compare our results to OpenMP implementations to understand the benefits of running the kernels in the accelerator versus CPU cores.
The Molecular Sciences Software Institute's (MolSSI) Quantum Chemistry Archive (QCArchive) project is an umbrella name that covers both a central server hosted by MolSSI for community data and the Python-based software infrastructure that powers automated computation and storage of quantum chemistry (QC) results. The MolSSI-hosted central server provides the computational molecular sciences community a location to freely access tens of millions of QC computations for machine learning, methodology assessment, force-field fitting, and more through a Python interface. Facile, user-friendly mining of the centrally archived quantum chemical data also can be achieved through web applications found at the website. The software infrastructure can be used as a standalone platform to compute, structure, and distribute hundreds of millions of QC computations for individuals or groups of researchers at any scale. The QCArchiveInfrastructure is open-source (BSD-3C), code repositories can be found at github, and releases can be downloaded via PyPI and Conda. This article is categorized under: Electronic Structure Theory > Ab Initio Electronic Structure Methods Software > Quantum Chemistry Data Science > Computer Algorithms and Programming
Inverse problems, which aim to infer unknown properties of a system using experimental and observational data, are central to addressing many of the U.S. Department of Energy’s (DOE) most critical scientific and engineering challenges. Accurate, computationally efficient, and data-efficient solutions to inverse problems are essential for advancing DOE mission-critical science drivers, including analyzing data from large-scale experimental facilities, optimizing fusion reactor performance, accelerating materials discovery, enhancing geophysical imaging, improving wildfire predictions, and enabling autonomous systems and digital twins. However, these problems are becoming increasingly complex, often involving nonlinear, highdimensional, and interconnected systems and models that span multiple physics and scales, while relying on data with varying quantity, quality, and information content. Compounding these challenges is the uncertainty inherent in DOE-relevant systems, where errors in inputs, noise in data, incompleteness of data, and discrepancies between models and reality constrain the accuracy and precision of solutions. At the same time, the convergence of recent scientific computing trends—scientific machine learning, artificial intelligence, and computing advances such as exascale computing—is creating unprecedented opportunities for tackling these challenges. The cross-cutting nature of inverse problems, combined with their growing complexity and rapidly evolving data and algorithmic demands, strongly motivates the formulation of a prioritized research agenda to maximize their capabilities and impact. In response to this need, DOE’s Advanced Scientific Computing Research (ASCR) program in the Office of Science convened the Workshop on Basic Research Needs for Inverse Problems for Complex Systems Under Uncertainty in June 2025. This workshop brought together experts across disciplines to identify grand challenges and major opportunities in the field. Through collaborative discussions, the workshop defined transformative research directions aimed at addressing the mathematical, statistical, and computational challenges posed by inverse problems under uncertainty. As a result of these efforts, four priority research directions (PRDs) were identified to guide future research and development in this area. These PRDs, summarized below, represent a roadmap for advancing the foundational science and mathematics of inverse problems, enabling robust, scalable, and uncertainty-aware solutions that are critical for DOE applications.
The probability of bit flips in hardware memory systems is projected to increase significantly as memory systems continue to scale in size and complexity. Effective hardware-based error detection and correction requires that the complete data path, involving all parts of the memory system, be protected with sufficient redundancy. First, this may be costly to employ on commodity computing platforms and second, even on high-end systems, protection against multi-bit errors may be lacking. Therefore, augmenting hardware error detection schemes with software techniques is of consider- able interest. In this paper, we consider software-level mechanisms to comprehensively detect transient memory faults. We develop novel compile-time algorithms to instrument application programs with checksum computation codes so as to detect memory errors. Unlike prior approaches that employ checksums on computational and architectural state, our scheme verifies every data access and works by tracking variables as they are produced and consumed. Experimental evaluation demonstrates that the proposed comprehensive error detection solution is viable as a completely software-only scheme. We also demonstrate that with limited hardware support, overheads of error detection can be further reduced.
This document describes the theoretical foundation of thermal analysis in Sierra Mechanics. The SIERRA Multimechanics Module: Aria, henceforth referred to as Aria for brevity, was developed at Sandia National Laboratories under the ASC program, and approximates linear and nonlinear continuum models of heat transfer. Aria uses the SIERRA Framework, which provides data management services commonly required by computational mechanics software, and facilitates the development of coupled, multi-mechanics applications for massively parallel computers. The mathematical models in Aria are based heavily on those of COYOTE, a well-established thermal analysis program that was also developed at Sandia and its ASC code predecessor, Calore. Aria, Calore and COYOTE share a significant body of numerical methods, which are described in detail by Reddy and Gartling. Throughout this document, the terms software and implementation are synonymous with the Aria thermal-fluid analysis computer program. Whether one uses Aria to perform heat transfer analysis, or in developing a new capability for the Aria application, this document provides the information needed understand the existing numerical algorithm implementations. Justification for the fundamental assumptions of heat transfer, nor derivation of the energy conservation equations are included in this document. For a more thorough theoretical background, one is referred to one of the many available textbooks, e.g. Another reference, which is freely available in downloadable electronic form, is Lienhard and Lienhard.
MPI is the dominant programming model for distributed memory parallel computers, and is often used as the intra-node programming model on multi-core compute nodes. However, application developers are increasingly turning to hybrid models that use threading within a node and MPI between nodes. In contrast to MPI, most current threaded models do not require application developers to deal explicitly with data locality. With increasing core counts and deeper NUMA hierarchies seen in the upcoming LANL/SNL 'Cielo' capability supercomputer, data distribution poses an upper boundary on intra-node scalability within threaded applications. Data locality therefore has to be identified at runtime using static memory allocation policies such as first-touch or next-touch, or specified by the application user at launch time. We evaluate several existing techniques for managing data distribution using micro-benchmarks on an AMD 'Magny-Cours' system with 24 cores among 4 NUMA domains and argue for the adoption of a dynamic runtime system implemented at the kernel level, employing a novel page table replication scheme to gather per-NUMA domain memory access traces.
Argonne Training Program in Extreme Scale Computing (ATPESC) was started by the Argonne National Laboratory with the objective of expanding the ranks of better prepared users of high performance computing (HPC) machines. One of the unique aspects of the program was inclusion of software engineering and community codes track. The inclusion was motivated by the observation that the projects with a good scientific and software process were better able to meet their scientific goals. In this paper we present our experience of running the software track from the beginning of the program until now. We discuss the motivations, the reception, and the evolution of the track over the years. We welcome discussion and input from the community to enhance the track in ATPESC, and also to facilitate inclusion of similar tracks in other HPC oriented training programs.
This document describes the theoretical foundation of thermal analysis in Sierra Mechanics. The SIERRA Multimechanics Module: Aria, henceforth referred to as Aria for brevity, was developed at Sandia National Laboratories under the ASC program, and approximates linear and nonlinear continuum models of heat transfer. Aria uses the SIERRA Framework, which provides data management services commonly required by computational mechanics software, and facilitates the development of coupled, multi-mechanics applications for massively parallel computers. The mathematical models in Aria are based heavily on those of COYOTE, a well-established thermal analysis program that was also developed at Sandia and its ASC code predecessor, Calore. Aria, Calore and COYOTE share a significant body of numerical methods, which are described in detail by Reddy and Gartling. Throughout this document, the terms software and implementation are syn onymous with the Aria thermal-fluid analysis computer program. Whether one uses Aria to perform heat transfer analysis, or in developing a new capability for the Aria application, this document provides the information needed understand the existing numerical algorithm implementations. Justification for the fundamental assumptions of heat transfer, nor derivation of the energy conservation equations are included in this document. For a more thorough theoretical background, one is referred to one of the many available textbooks, e.g. Another reference, which is freely available in downloadable electronic form, is Lienhard and Lienhard.
We report that Q-POP-Thermo is a program designed to compute thermodynamic monodomain equilibrium states and their properties for ferroelectric single crystals and thin films based on the Landau-Ginzburg-Devonshire (LGD) Theory. Utilizing symbolic manipulation with the SymPy Library, the governing equations along with appropriate boundary conditions are solved for speedy minimization of the free energy of a crystal. Utilizing the popular Differential Evolution algorithm, with appropriate hybridization, multiple phase diagrams, such as the pressure-temperature phase diagram for bulk single crystals and the common strain-temperature phase diagram for monodomain thin-film systems can be readily generated. Furthermore, a variety of material properties of stable ferroelectric phases, including dielectric, piezoelectric, and electrocaloric properties, can simultaneously be calculated. Validation studies are presented for both thin-film and single crystal systems to test the effectiveness and capability of the open-source program.
In the 1990s, the U.S. nuclear weapons program shifted emphasis from developing new designs to dismantling thousands of existing weapons and maintaining a much smaller enduring stockpile. The United States ceased underground nuclear testing, and the Department of Energy created the Stockpile Stewardship Program to maintain the safety, security, and reliability of the U.S. nuclear deterrent without full-scale testing. This video gives a behind the scenes look at a set of unique capabilities at Lawrence Livermore that are indispensable to the Stockpile Stewardship Program: high performance computing, the Superblock category II nuclear facility, the JASPER a two stage gas gun, the High Explosive Applications Facility (HEAF), the National Ignition Facility (NIF), and the Site 300 contained firing facility.
We present the first computation in a program of lattice-QCD baryon physics using staggered fermions for sea and valence quarks. For this initial study, we present a calculation of the nucleon mass, obtaining 964 ±16 MeV with all sources of statistical and systematic errors controlled and accounted for. This result is the most precise determination to date of the nucleon mass from first principles. We use the highly improved staggered quark action, which is computationally efficient. Three gluon ensembles are employed, which have approximate lattice spacings 𝑎 ≈ 0.09, 0.12, and 0.15 fm, each with equal-mass 𝑢/𝑑, 𝑠, and 𝑐 quarks in the sea. Further, all ensembles have the light valence and sea 𝑢/𝑑 quarks tuned to reproduce the physical pion mass, avoiding complications from chiral extrapolations. Our work opens a new avenue for precise calculations of baryon properties, which are both feasible and relevant to experiments in particle and nuclear physics.
Linac dynamics simulation capabilities of the PyORBIT code are discussed. PyORBIT is an open source code and a further development of the original ORBIT code that was created and used for design, studies, and commissioning of the SNS ring. The PyORBIT code, like the original one, has a two-layer structure. C++ is used to perform time-consuming computations, and the program flow is controlled from a Python language shell. The flexible structure makes it possible to use PyORBIT also for linac dynamics simulations. A benchmark of PyORBIT with Parmila and the XAL Online model is presented.