Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

Experience and Practice of Batch Scheduling on Leadership Supercomputers at Argonne

The mission of the DOE Argonne Leadership Computing Facility (ALCF) is to accelerate major scientific discoveries and engineering breakthroughs for humanity by designing and providing world-leading computing facilities in partnership with the computational science community. The ALCF operates supercomputers that are generally amongst the Top 5 fastest machines in the world. Specifically, ALCF is looking for the science that is either too big to run anywhere else, or it would take so long as to be impractical (i.e., “capability jobs”). At ALCF, batch scheduling plays a critical role for achieving a set of site goals within a set of constraints. While system utilization is an important goal at ALCF, its largest mission constraint is to enable extreme scale parallel jobs to take precedence. In this paper, we will describe the specific scheduling goals and constraints, analyze the workload traces collected in 2013–2017 from the 48-rack petascale supercomputer Mira, and discuss the upcoming scheduling challenges at ALCF.

Allcock, William↗

Aces4: A Platform for Computational Chemistry Calculations with Extremely Large Block-Sparse Arrays

Aces4 is a parallel programming platform comprising a DSL for Computational Chemistry and its runtime system. It offers a convenient way to express parallelism together with extensive support for extremely large, possibly sparse, distributed arrays. It aids scientists in the creation of performant, scalable, massively parallel programs that can effectively take advantage of leadership class computing systems to address important scientific questions. Aces4 has enabled the development and implementation of new methods in electronic structure theory which are breaking new ground in their ability to perform highly accurate calculations on ever larger molecular systems. In this paper the design of Aces4, which is based on the the Super Instruction Architecture approach, is described. Experimental scaling results for Molecular Cluster Perturbation Theory, a new method enabled by Aces4, and CCSD, a widely used computational chemistry method are given.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Programming Abstractions for Managing Workflows on Tiered Storage Systems

Scientific workflows in High Performance Computing (HPC) environments are processing large amounts of data. The storage hierarchy on HPC systems is getting deeper, driven by new technologies (NVRAMs, SSDs, etc.) There is a need for new programming abstractions that allow users to seamlessly manage data at the workflow level on multi-tiered storage systems, and provide optimal workflow performance and use of storage resources. In previous work, we introduced a software architecture Managing Data on Tiered Storage for Scientific Workflows (MaDaTS) that used a Virtual Data Space (VDS) abstraction to hide the complexities of the underlying storage system while allowing users to control data management strategies. In this article, we detail the data-centric programming abstractions that allow users to manage a workflow around its data on the storage layer. The programming abstractions simplify data management for scientific workflows on multi-tiered storage systems, without affecting workflow performance or storage capacity. We measure the overheads and effectiveness introduced by the programming abstractions of MaDaTS. Our results show that these abstractions can optimally use the storage capacity in lesser capacity storage tiers, and simplify data management without adding any performance overheads.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters

Big data is prevalent in HPC computing. Many HPC projects rely on complex workflows to analyze terabytes or petabytes of data. These workflows often require running over thousands of CPU cores and performing simultaneous data accesses, data movements, and computation. It is challenging to analyze the performance involving terabytes or petabytes of workflow data or measurement data of the executions, from complex workflows over a large number of nodes and multiple parallel task executions. To help identify performance bottlenecks or debug the performance issues in large-scale scientific applications and scientific clusters, we have developed a performance analysis framework, using state-ofthe- art open-source big data processing tools. Our tool can ingest system logs and application performance measurements to extract key performance features, and apply the most sophisticated statistical tools and data mining methods on the performance data. It utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of the big data analysis framework, we conduct case studies on the workflows from an astronomy project known as the Palomar Transient Factory (PTF) and the job logs from the genome analysis scientific cluster. Our study processed many terabytes of system logs and application performance measurements collected on the HPC systems at NERSC. The implementation of our tool is generic enough to be used for analyzing the performance of other HPC systems and Big Data workows.

97 MATHEMATICS AND COMPUTING↗

Domain-specific compilers for dynamic simulations of quantum materials on quantum computers

Abstract Simulation of the dynamics of quantum materials is emerging as a promising scientific application for noisy intermediate-scale quantum (NISQ) computers. Due to their high gate-error rates and short decoherence times, however, NISQ computers can only produce high-fidelity results for those quantum circuits smaller than some given circuit size. Dynamic simulations, therefore, pose a challenge as current algorithms produce circuits that grow in size with each subsequent time-step of the simulation. This underscores the crucial role of quantum circuit compilers to produce executable quantum circuits of minimal size, thereby maximizing the range of physical phenomena that can be studied within the NISQ fidelity budget. Here, we present two domain-specific (DS) quantum circuit compilers for the Rigetti and IBM quantum computers, specifically designed to compile circuits simulating dynamics under a special class of time-dependent Hamiltonians. The compilers outperform state-of-the-art general-purpose compilers in terms of circuit size reduction by around 25%–30% as well as wall-clock compilation time by around 40% (dependent on system size and simulation time-step). Drawing on heuristic techniques commonly used in artificial intelligence, both compilers scale well with simulation time-step and system size. Code for both compilers is open-source and packaged into a full-stack quantum simulation software with tutorials included for ease of use for future researchers wishing to perform dynamic simulations of quantum materials on quantum computers. As our DS compilers provide significant improvements in both compilation time and simulation fidelity, they provide a building block for accelerating progress toward physical quantum supremacy.

Physics↗

Computational thermodynamics and its applications

Thermodynamics is a science concerning the state of a system, whether it is stable, metastable or un- stable, when interacting with the surroundings. In this overview, fundamentals of thermodynamics are briefly reviewed through the combination of first and second laws of thermodynamics for open and nonequilibrium systems to demonstrate that the reversible equilibrium and irreversible nonequilibrium thermodynamics can be integrated to enhance the power and utilities of thermodynamics. The recent progresses in computational thermodynamics, the remaining challenges, and potential impacts in broad scientific fields are discussed here. It is shown that computational thermodynamics enables the modeling of thermodynamics of a state as a function of both external and internal variables and quantitative calculations of a broad range of properties of a multicomponent system in terms of first and second derivatives of energy, including not only equilibrium states when there are no driving forces for any internal processes and but also non-equilibrium states with driving forces for internal processes. Consequently, external constraints such as fixed strain and internal degree of freedoms such as ordering and defects can be described in a coherent framework and applied to materials design. Furthermore, two important but largely overlooked aspects in thermodynamics will be discussed, i.e. the rigorous application of statistical thermodynamics with the probability of configurations and their contributions to system properties, and the applications of second derivatives of energy with respect to either two extensive variables or two potentials or a mixture of them in terms of understanding and predicting emergent behaviors, critical phenomena, kinetic coefficients, and mechanical properties.

36 MATERIALS SCIENCE↗

A Visual Comparison of Silent Error Propagation

High-performance computing (HPC) systems play a critical role in facilitating scientific discoveries. Their scale and complexity (e.g., the number of computational units and software stack) continue to grow as new systems are expected to process increasingly more data and reduce computing time. However, with more processing elements, the probability that these systems will experience a random bit-flip error that corrupts a program's output also increases, which is often recognized as silent data corruption. Analyzing the resiliency of HPC applications in extreme-scale computing to silent data corruption is crucial but difficult. An HPC application often contains a large number of computation units that need to be tested, and error propagation caused by error corruption is complex and difficult to interpret. Here, to accommodate this challenge, we propose an interactive visualization system that helps HPC researchers understand the resiliency of HPC applications and compare their error propagation. Our system models an application's error propagation to study a program's resiliency by constructing and visualizing its fault tolerance boundary. Coordinating with multiple interactive designs, our system enables domain experts to efficiently explore the complicated spatial and temporal correlation between error propagations. At the end, the system integrated a nonmonotonic error propagation analysis with an adjustable graph propagation visualization to help domain experts examine the details of error propagation and answer such questions as why an error is mitigated or amplified by program execution.

97 MATHEMATICS AND COMPUTING↗

Domaine-Specific Runtime to Orchestrate Computation on Heterogeneous Platforms

Task-based runtime systems in the past have sought to exploit inherent asynchronicity in the application execution to reduce overall runtime. In the last decade focus shifted to supporting the heterogeneity that is increasingly prevalent in high-performance computing systems. However, existing task-based runtime systems, being general, come with a challenging set of issues such as the complexity of abstractions and overheads. And they still leave much of the burden of exposing the parallelism on the application developers who also have to fit their applications to the runtime systems’ interfaces. In this paper we take a different approach for applications to target heterogeneous systems through domain-specific run-times. Our design aspires to leverage the domain-specific knowledge of a focused class of scientific simulations to pragmatically orchestrate computations in the simulations.

97 MATHEMATICS AND COMPUTING↗

Accelerating Scientific Workflows on HPC Platforms with In Situ Processing

Scientific workflows drive most modern large-scale science breakthroughs by allowing scientists to define their computations as a set of jobs executed in a given order based on their data dependencies. Workflow management systems (WMSs) have become key to automating scientific workflows-executing computational jobs and orchestrating data transfers between those jobs running on complex high-performance computing (HPC) platforms. Traditionally, WMSs use files to communicate between jobs: a job writes out files that are read by other jobs. However, HPC machines face a growing gap between their storage and compute capabilities. To address that concern, the scientific community has adopted a new approach called in situ, which bypasses costly parallel filesystem I/O operations with faster in-memory or in-network communications. When using in situ approaches, communication and computations can be interleaved. In this work, we leverage the Decaf in situ dataflow framework to accelerate task-based scientific workflows managed by the Pegasus WMS, by replacing file communications with faster MPI messaging. We propose a new execution engine that uses Decaf to manage communications within a sub-workflow (i.e., set of jobs) to optimize inter-job communications. We consider two workflows in this study: (i) a synthetic workflow that benchmarks and compares file- and MPI-based communication; and (ii) a realistic bioinformatics workflow that computes mu-tational overlaps in the human genome. Experiments show that in situ communication can improve the bioinformatics workflow execution time by 22% to 30% compared with file communication. Our results motivate further opportunities and challenges for bridging traditional WMSs with in situ frameworks.

Decaf↗

Numerical Investigation of Fluid Flow and Space Charge in Liquid Argon Time Projection Chamber (LArTPC) Detectors

Overview This project focused on developing a high-fidelity numerical framework to simulate the multiphysics environment within Liquid Argon Time Projection Chamber (LArTPC) detectors. The primary objective was to characterize the complex interplay between ion transport, background fluid dynamics, and electric field distortions—a critical factor for the calibration and sensitivity of next-generation High Energy Physics experiments, such as DUNE. Technical Achievements The research successfully yielded a hybrid numerical space-charge solver utilizing a Cell-Centered Finite Volume Method (FVM) for ion transport coupled with a Finite Element Method (FEM) for electric potential. Key accomplishments include: • Verification & Validation: The 3-D solver was rigorously verified against 1-D analytical solutions, demonstrating high numerical accuracy in predicting space-charge-induced field deviations. • Field Distortion Analysis: 3D simulations revealed that space charge effects introduce significant non-uniformities in the electric field. Critically, the research identified that background LAr flow velocities, when comparable to ion drift velocities, markedly exacerbate these distortions. • Technology Transfer: The resulting source code and comprehensive user manuals were successfully transferred to collaborators at Fermilab, providing a portable computational tool for the broader scientific community. Challenges and Future Directions While the space-charge solver achieved all performance metrics, the integrated fluid dynamics modeling encountered convergence challenges stemming from the extreme 200-fold disparity in length scales between the detector's 37 mm inlet pipes and the 8-meter global domain. To address this, the project has identified a clear technical pivot toward Hierarchical Geometric Adaptive Mesh Refinement (HG-AMR). By implementing an h-type refinement strategy with hanging nodes, future iterations of this solver will be capable of resolving localized high-gradient inlet flows without the prohibitive computational costs of regular grids. This advancement, combined with data-driven uncertainty quantification based on MicroBooNE-style calibration, will enable the precise modeling of detector responses in large-scale cryogenic environments where direct measurement remains difficult. Impact The computational tools developed under this award provide a foundation for enhancing the energy resolution and spatial reconstruction of noble liquid detectors. By bridging the gap between theoretical fluid dynamics and experimental field calibration, this work supports the DOE’s mission to advance the frontiers of neutrino physics and dark matter detection.

42 ENGINEERING↗

ytopt: Autotuning Scientific Applications for Energy Efficiency at Large Scales

As we enter the exascale computing era, efficiently utilizing power and optimizing the performance of scientific applications under power and energy constraints has become critical and challenging. We propose a low-overhead autotuning framework to autotune performance and energy for various hybrid MPI/OpenMP scientific applications at large scales and to explore the tradeoffs between application runtime and power/energy for energy efficient application execution, then use this framework to autotune four ECP proxy applications—XSBench, AMG, SWFFT, and SW4lite. Our approach uses Bayesian optimization with a Random Forest surrogate model to effectively search parameter spaces with up to 6 million different configurations on two large-scale HPC production systems, Theta at Argonne National Laboratory and Summit at Oak Ridge National Laboratory. The experimental results show that our autotuning framework at large scales has low overhead and achieves good scalability. Using the proposed autotuning framework to identify the best configurations, we achieve up to 91.59% performance improvement, up to 21.2% energy savings, and up to 37.84% EDP (energy delay product) improvement on up to 4096 nodes.

Autotuning↗

A survey of software implementations used by application codes in the Exascale Computing Project

The US Department of Energy Office of Science and the National Nuclear Security Administration initiated the Exascale Computing Project (ECP) in 2016 to prepare mission-relevant applications and scientific software for the delivery of the exascale computers starting in 2023. The ECP currently supports 24 efforts directed at specific applications and six supporting co-design projects. These 24 application projects contain 62 application codes that are implemented in three high-level languages—C, C++, and Fortran—and use 22 combinations of graphical processing unit programming models. The most common implementation language is C++, which is used in 53 different application codes. The most common programming models across ECP applications are CUDA and Kokkos, which are employed in 15 and 14 applications, respectively. This article provides a survey of the programming languages and models used in the ECP applications codebase that will be used to achieve performance on the future exascale hardware platforms.

97 MATHEMATICS AND COMPUTING↗

Aurora: Argonne’s Next-Generation Exascale Supercomputer

Argonne National Laboratory’s next-generation supercomputer, Aurora, will be one of the world’s first exascale systems when it arrives in 2021. The exascale supercomputer, developed in partnership with Intel and Cray, will dramatically accelerate scientific discovery and usher in new technological capabilities, furthering the United States’ scientific leadership position globally. Aurora will be housed at the Argonne Leadership Computing Facility, a U.S. Department of Energy Office of Science User Facility and a premier source of world-class computing resources for open science research since 2006.

97 MATHEMATICS AND COMPUTING↗

Data-intensive management and analysis for scientifc simulations.

Scientific simulations can produce enormous amounts of data, making the analysis of results and management of files a difficult task for scientists. The simulation management and analysis system (Smaash) described here is designed to allow scientists to easily capture, store, organize, monitor, and analyze simulation results. The system is automatic, standardized, and secure. Smaash was built using open-source tools and modularized to be independent of the scientific simulation. The web-based front-end allows the scientist to easily interact with the data, and has proved its usefulness in improving the efficiency of a scientific team's workflow. High performance parallel computing allows scientists to solve complex physical problems through computer simulation. However, the massive amounts of data generated and the complex computing environment can create additional complications. A recent review by Ludaescher et al.(2009) describes how scientific workflows can assist scientists in extracting knowledge from these data-intensive operations by automating components within pipelines. Within the fusion community, Klasky et al.(2008) and colleagues have developed a system that handles the storage management, data movement, metadata generation and management, and a means to analyze the results. In response to scientists needs, a simulation management and analysis system (Smaash) was developed at the University of Chicago and Argonne National Laboratory (USA). Smaash provides an integrated way to monitor simulations and analyze computational results; catalog, store, and retrieve simulations; and prepare output for publications. The system is independent of the particular simulation code, accessible from many HPC and browser-based platforms, and built around open-source software tools. Data security and provenance is considered throughout. The analysis components are hidden behind a web-based front end, enabling scientists to focus on their results and not get bogged down by information overload.

99 GENERAL AND MISCELLANEOUS↗

Uniform-in-phase-space data selection with iterative normalizing flows

Improvements in computational and experimental capabilities are rapidly increasing the amount of scientific data that are routinely generated. In applications that are constrained by memory and computational intensity, excessively large datasets may hinder scientific discovery, making data reduction a critical component of data-driven methods. Datasets are growing in two directions: the number of data points and their dimensionality. Whereas dimension reduction typically aims at describing each data sample on lower-dimensional space, the focus here is on reducing the number of data points. A strategy is proposed to select data points such that they uniformly span the phase-space of the data. The algorithm proposed relies on estimating the probability map of the data and using it to construct an acceptance probability. An iterative method is used to accurately estimate the probability of the rare data points when only a small subset of the dataset is used to construct the probability map. Instead of binning the phase-space to estimate the probability map, its functional form is approximated with a normalizing flow. Therefore, the method naturally extends to high-dimensional datasets. The proposed framework is demonstrated as a viable pathway to enable data-efficient machine learning when abundant data are available.

97 MATHEMATICS AND COMPUTING↗

Completing the Link between Exposure Science and Toxicology for Improved Environmental Health Decision Making: The Aggregate Exposure Pathway Framework

Driven by major scientific advances in analytical methods, biomonitoring, and computational exposure assessment, and a newly articulated vision for a greater impact in public health, the field of exposure science is undergoing a rapid transition from a field of observation to a field of prediction. Deployment of an organizational and predictive framework for exposure science analogous to the computationally enabled “systems approaches” used in the biological sciences is a necessary step in this evolution. Here we propose the aggregate exposure pathway (AEP) concept as the natural and complementary companion in the exposure sciences to the adverse outcome pathway (AOP) concept in the toxicological sciences. The AEP framework offers an intuitive approach to successful organization of exposure science data within individual units of prediction common to the field, setting the stage for exposure forecasting. Looking farther ahead, we envision direct linkages between aggregate exposure pathway and adverse outcome pathways, completing the source to outcome continuum and setting the stage for more efficient integration of exposure science and toxicity testing information. Together these frameworks form and inform a decision making framework with the flexibility for risk-based, hazard-based or exposure-based decisions.

Teeguarden, Justin G.↗

Collaborative Exploration of Scientific Datasets Using Immersive and Statistical Visualization: Preprint

We discuss the value of collaborative, immersive visualization for the exploration of scientific datasets and review techniques and tools that have been developed and deployed at the National Renewable Energy Laboratory (NREL). We believe that collaborative visualizations linking statistical interfaces and graphics on laptops and high-performance computing (HPC) with 3D visualizations on immersive displays (head-mounted displays and large-scale immersive environments) enable scientific workflows that further rapid exploration of large, high-dimensional datasets by teams of analysts. We present a framework, PlottyVR, that blends statistical tools, general-purpose programming environments, and simulation with 3D visualizations. To contextualize this framework, we propose a categorization and loose taxonomy of collaborative visualization and analysis techniques. Finally, we describe how scientists and engineers have adopted this framework to investigate large, complex datasets.

collaborative visualization↗