Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “High-End Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Towards Native Execution of Deep Learning on a Leadership-Class HPC System

Large parallel machines generally offer the best parallel performance with "native execution" that is achieved using codes developed with the optimized compilers, communication libraries, and runtimes offered on the machines. In this paper, we report and analyze performance results from native execution of deep learning on a leadership-class high-performance computing (HPC) system. Using our new code called DeepEx, we present a study of the parallel speed up and convergence rates of learning achieved with native parallel execution. In the trade-off between computational parallelism and synchronized convergence, we first focus on maximizing parallelism while still obtaining convergence. Scaling results are reported from execution on up to 15,000 GPUs using two scientific data sets from atom microscopy and protein folding applications, and also using the popular ImageNet data set. In terms of the traditional measure of parallel speed up, excellent scaling is observed up to 12,000 GPUs. Additionally, accounting for convergence rates of deep learning accuracy or error, a deep learning-specific metric called "learning speed up" is also tracked. The performance results indicate the need to evaluate parallel deep learning execution in terms of learning speed up, and point to additional directions for improved exploitation of high-end HPC systems.

Yoginath, Srikanth↗

CMIP6-based Multi-model Hydroclimate Projection over the Conterminous US, Version 1.1

This dataset presents a suite of high-resolution downscaled hydro-climate projections over the conterminous United States (CONUS) based on multiple selected Global Climate Models (GCMs) from the Coupled Models Intercomparison Project phase 6 (CMIP6). The CMIP6 GCMs are downscaled using either statistical (DBCCA) or dynamical (RegCM) downscaling approaches based on two meteorological reference datasets (Daymet and Livneh). Subsequently, the downscaled precipitation, temperature, and wind speed are employed to drive two calibrated hydrologic models (VIC and PRMS), enabling the simulation of projected future hydrologic responses across the CONUS. Each ensemble member covers the 1980-2019 baseline and 2020-2059 near-future periods under the high-end (SSP585) emission scenario. Moreover, utilizing only DBCCA and Daymet, the projections are further extended to the 2060-2099 far-future period and across three additional emission scenarios (SSP370, SSP245, and SSP126). This dataset is formulated to support the SECURE Water Act Section 9505 Assessment for the US Department of Energy (DOE) Water Power Technologies Office (WPTO). For further details on this dataset, please refer to Kao et al. (2022) and Rastogi et al. (2022).

13 HYDRO ENERGY↗

DeStager: feature guided in-situ data management in distributed deep memory hierarchies

In-situ analytics have been increasingly adopted by leadership scientific applications to gain fast insights into massive output data of simulations. With the current practice, systems buffer the output data in DRAM for analytics processing, constraining it to DRAM capacity un-used by the simulation. The rapid growth of data size requires alternative approaches to accommodating data-rich analytics, such as using solid-state disks to increase effective memory capacity. For this purpose, this paper explores software solutions for exploring the deep memory hierarchies expected on future high-end machines. Leveraging the fact that many analytics are sensitive to data features (regions-of-interest) hidden in the data being processed, the approach incorporates the knowledge of the data features into in-situ data management. It uses adaptive index creation/refinement to reduce the overhead of index management. In addition, it uses data features to predict data skew and improve load balance through controlling data distribution and placement on distributed staging servers. The experimental results show that such feature-guided optimizations achieve substantial improvements over state-of-the-art approaches for managing output data in-situ.

Computer Science↗

Land Surface Modeling 2.0 for agricultural climate change impact assessments

Focus Area: This white paper addresses DOE AI4ESP focus area #2 by providing a sketch blueprint for a next generation, hybrid AI/process-based global Land Surface Modeling (LSM) framework to improve projections of climate change impacts on the land surface system including agriculture and the hydrological cycle. Science Challenge: Climate change impacts on agriculture are highly uncertain: -50% to +150% global production changes for major grains under high-end climate change, for example. The core challenge in land surface system predictability is the large gap between the scale at which the relevant biological processes act and the scale at which the risks need to be assessed. Conventional empirical and process-based modeling approaches are insufficient. A new multi-scale modeling paradigm which employs AI/ML to learn from new streams of remote sensing data and targeted ’gene-to-global’ simulations is needed.

54 ENVIRONMENTAL SCIENCES↗

Scalable Transparent Checkpoint-Restart of Global Address Space Applications on Virtual Machines over Infiniband

Checkpoint-Restart is one of the most used software approaches to achieve fault-tolerance in high-end clusters. While standard techniques typically focus on user-level solutions, the advent of virtualization software has enabled efficient and transparent system-level approaches. In this paper, we present a scalable transparent system-level solution to address fault-tolerance for applications based on global address space (GAS) programming models on Infiniband clusters. In addition to handling communication, the solution addresses transparent checkpoint of user-generated files. We exploit the support for the Infiniband network in the Xen virtual machine environment. We have developed a version of the Aggregate Remote Memory Copy Interface (ARMCI) one-sided communication library capable of suspending and resuming applications. We present efficient and scalable mechanisms to distribute checkpoint requests and to backup virtual machines memory images and file systems. We tested our approach in the context of NWChem, a popular computational chemistry suite. We demonstrated that NWChem can be executed, without any modification to the source code, on a virtualized 8-node cluster with very little overhead (below 3%). We observe that the total checkpoint time is limited by disk I/O. Finally, we measured system-size depended components of the checkpoint time on up to 1024 cores (128 nodes), demonstrating the scalability of our approach in medium/large-scale systems.

97 MATHEMATICS AND COMPUTING↗

Economic and embodied energy analysis of integrated circuit manufacturing processes

Integrated circuits (ICs) are the fundamental microelectronic building blocks of typical information and communication technology (ICT) devices such as laptops and smartphone. With the emergence of faster and cheaper low-power ICs for the growing ICT market, there is a limited information on their manufacturing energy and economic impacts owing to the lack of recurrent upgradation of proprietary IC manufacturing process data. In this work, economic and life cycle energy assessments (LCAs) of IC manufacturing and assembly processes were conducted using the latest bill of materials (BOM) manufacturing data in a state-of-the-art IC manufacturing cost software tool developed by IC Knowledge, LLC. IC manufacturing cost was estimated to be $\$$1.00-$\$$5.00/cm 2 ; the high-end cost represents the most advanced 3D NAND IC technology with the Wafer Level Chip Scale Package cost of $4/cm 2 . Total cumulative energy demand (CED) estimates of IC manufacturing using the OpenLCA software ranged from 9 to 38 MJ/cm 2 or 56–235 BJ/month with the largest share (~ 75%) from on-site energy. The IC manufacturing CED is projected to increase with the progress in IC technology node, which has increased by more than two times for technology node evolution from 110 to 14 nm. Furthermore, the representative detailed BOM data were considered to reduce the CED uncertainties of the estimated standard deviation of 35%- 42%.

97 MATHEMATICS AND COMPUTING↗

Landrush: Rethinking In-Situ Analysis for GPGPU Workflows

In-situ analysis on the output data of scientific simulations has been made necessary by ever-growing output data volumes and increasing costs of data movement as supercomputing is moving towards exascale. With hardware accelerators like GPUs becoming increasingly common in high end machines, new opportunities arise to co-locate scientific simulations and online analysis performed on the scientific data generated by the simulations. However, the asynchronous nature of GPGPU programming models and the limited context-switching capabilities on the GPU pose challenges to co-locating the scientific simulation and analysis on the same GPU. This paper dives deeper into these challenges to understand how best to co-locate analysis with scientific simulations on the GPUs in HPC clusters. Specifically, our 'Landrush' approach to GPU sharing proposes a solution that utilizes idle cycles on the GPU to provide an improved time-to-answer, that is, the total time to run the scientific simulation and analysis of the generated data. Landrush is demonstrated with experimental results obtained from leadership high-end applications on ORNL's Titan supercomputer, which show that (i) GPU-based scientific simulations have varying degrees of idle cycles to afford useful analysis task co-location, and (ii) the inability to context switch on the GPU at instruction granularity can be overcome by careful control of the analysis kernel launches and software-controlled early completion of analysis kernel executions. Results show that Landrush is superior in terms of time-to-answer compared to serially running simulations followed by analysis or by relying on the GPU driver and hardwired thread dispatcher to run analysis concurrently on a single GPU.

Computer Science↗

Matrix Algebra for GPU and Multicore Architectures (MAGMA) for Large Petascale Systems

The goal of the MAGMA project is to create a new generation of linear algebra libraries that achieve the fastest possible time to an accurate solution on hybrid Multicore+GPU-based systems, using all the processing power that future high-end systems can make available within given energy constraints. Our efforts at the University of Tennessee achieved the goals set in all of the five areas identified in the proposal: 1. Communication optimal algorithms; 2. Autotuning for GPU and hybrid processors; 3. Scheduling and memory management techniques for heterogeneity and scale; 4. Fault tolerance and robustness for large scale systems; 5. Building energy efficiency into software foundations. The University of Tennessee’s main contributions, as proposed, were the research and software development of new algorithms for hybrid multi/many-core CPUs and GPUs, as related to two-sided factorizations and complete eigenproblem solvers, hybrid BLAS, and energy efficiency for dense, as well as sparse, operations. Furthermore, as proposed, we investigated and experimented with various techniques targeting the five main areas outlined.

97 MATHEMATICS AND COMPUTING↗

TiltEM User Manual

The ability to automate the scanning transmission electron microscope (S/TEM) is tantamount to addressing next generation artificial intelligence, machine learning, and materials modeling capabilities. Constant utilization of these high-end capital equipment purchases also serves the requirements of the Department of Energy (DOE) to be fiscally responsible to the public. The development of an automated multi-modal tilt series algorithm for dark field/bright field S/TEM imaging plus chemical identification with energy dispersive x-ray spectroscopy has been achieved at PNNL. A tilt series workflow has now been generated which allows for autonomous collection of imaging and compositional information simultaneously. This advancement is due in large part to a transition from the rigid and microscope specific JEOL hardware environment, to the more adaptable GMS environment. This has thereby led to a greater degree of flexibility in the application of this method across platforms in addition to substantial time savings to the user.

47 OTHER INSTRUMENTATION↗

Automatic Transformation of MPI Programs to Asynchronous, Graph-Driven Form

The goals of this project are to develop new, scalable, high-fidelity algorithms for atomic-level simulations and program transformations that automatically restructure existing applications, enabling them to scale forward to Petascale systems and beyond. The techniques enable legacy MPI application code to exploit greater parallelism though increased latency hiding and improved workload assignment. The techniques were successfully demonstrated on high-end scalable systems located at DOE laboratories. Besides the automatic MPI program transformations efforts, the project also developed several new scalable algorithms for ab-initio molecular dynamics, including new massively parallel algorithms for hybrid DFT and new parallel in time algorithms for molecular dynamics and ab-initio molecular dynamics. These algorithms were shown to scale to very large number of cores, and they were designed to work in the latency hiding framework developed in this project. The effectiveness of the developments was enhanced by the direct application to real grand challenge simulation problems covering a wide range of technologically important applications, time scales and accuracies. These included the simulation of the electronic structure of mineral/fluid interfaces, the very accurate simulation of chemical reactions in microsolvated environments, and the simulation of chemical behavior in very large enzyme reactions.

97 MATHEMATICS AND COMPUTING↗

Memphis: Finding and Fixing NUMA-related Performance Problems on Multi-core Platforms

Until recently, most high-end scientific applications have been immune to performance problems caused by Non-Uniform Memory Access (NUMA). However, current trends in micro-processor design are pushing NUMA to smaller and smaller scales. This paper examines the current state of NUMA and makes several contributions. First, we summarize the performance problems that NUMA can present for multithreaded applications and describe methods of addressing them. Second, we demonstrate that NUMA can indeed be a significant problem for scientific applications, showing that it can mean the difference between an application scaling perfectly and failing to scale at all. Third, we describe, in increasing order of usefulness, three methods of using hardware performance counters to aid in finding NUMA-related problems. Finally, we introduce Memphis, a data-centric toolset that uses Instruction Based Sampling to help pinpoint problematic memory accesses, and demonstrate how we used it to improve the performance of several production-level codes - HYCOM, XGC1 and CAM - by 13%, 23% and 24% respectively.

97 MATHEMATICS AND COMPUTING↗

Scalable Data-Intensive Geocomputation: A Design for Real-Time Continental Flood Inundation Mapping

The convergence of data-intensive and extreme-scale computing enables an integrated software and data ecosystem for scientific discovery. Developments in this realm will fuel transformative research in data-driven interdisciplinary domains. Geocomputation provides computing paradigms in Geographic Information Systems (GIS) for interactive computing of geographic data, processes, models, and maps. Because GIS is data-driven, the computational scalability of a geocomputation workflow is directly related to the scale of the GIS data layers, their resolution and extent, as well as the velocity of the geo-located data streams to be processed. Unique in high user interactivity and low end-to-end latency requirements, geocomputation applications will dramatically benefit from the convergence of high-end data analytics (HDA) and high-performance computing (HPC). The application level challenge, however, is to identify and eliminate computational bottlenecks that arise along a geocomputation workflow. Indeed, poor scalability at any of the workflow components is detrimental to the entire end-to-end pipeline. Here, we study a large geocomputation use case in flood inundation mapping that handles multiple national-scale geospatial datasets and targets low end-to-end latency. We discuss benefits and challenges for harnessing both HDA and HPC for data-intensive geospatial data processing and intensive numerical modeling of geographic processes. We propose an HDA+HPC geocomputation architecture design that couples HDA (e.g., Spark)-based spatial data handling and HPC-based parallel data modeling. Key techniques for coupling HDA and HPC to bridge the two different software stacks are reviewed and discussed.

Liu, Yan↗

Unraveling the multi-step crystallization mechanism of polytetrafluoroethylene, modified polytetrafluoroethylene, and their nanocomposites with boron nitride nanobarbs: Experimental insights and theoretical analysis

The non-isothermal crystallization behavior and kinetics of polytetrafluoroethylene (PTFE) composites with boron nitride nanobarb (BNNB), a new generation nanostructure with unique surface morphology and mechanical “barbs” have been analyzed, understanding these properties is essential for their high-end applications as thermal interface materials (TIM) for microwave, 5G and microelectronic devices. The analysis of the crystallization parameters includes crystallization onset, peak and end temperatures, crystallization half-life and overall crystallinity of PTFE, modified PTFE and their BNNB composites. The results were further analyzed using theoretical models such as the combined Avrami-Ozawa model. It was found that BNNB supports crystallization in the modified PTFE but shows minimal effect on the crystallization of PTFE. Due to the limitation of the classical theoretical models used above in fully characterizing the multi-step crystallization process of PTFE, an in-depth analysis using the model-free advanced isoconversional computation was used to characterize the PTFE crystallization based on the evolution of activation energy with fractional crystallinity and for the first time with temperature. Three kinetic regions were identified in the crystallization mechanism. Here, this study investigated the molecular organization and microstructural evolution of PTFE, modified PTFE and their composites during non-isothermal crystallization using advanced X-ray scattering measurements. An insight into the changes undergone by the material's microstructural units including crystallite size and morphology, lamellar thickness and lamellar interfacial layer thickness, and crystallographic phase dynamics during non-isothermal cooling from the melt, was provided here in this work. The effect of copolymer modification of PTFE and the inclusion of pristine and functionalized BNNB (a thermally conductive and electrically insulating ceramic) are both new investigations that provide valuable knowledge for the development of materials with strong matrix-nanofiller interaction and guidance for optimizing sintering and cooling cycles, two key steps in PTFE processing that largely affect the material microstructural features. Overall, the result of the three-part investigation demonstrates that BNNB supports crystallization in the modified PTFE up to 20 wt% concentration and at low and high cooling rates typically used in the industrial processing of PTFE.

36 MATERIALS SCIENCE↗

FAST CALCULATION OF THE LOMB-SCARGLE PERIODOGRAM USING GRAPHICS PROCESSING UNITS

I introduce a new code for fast calculation of the Lomb-Scargle periodogram that leverages the computing power of graphics processing units (GPUs). After establishing a background to the newly emergent field of GPU computing, I discuss the code design and narrate key parts of its source. Benchmarking calculations indicate no significant differences in accuracy compared to an equivalent CPU-based code. However, the differences in performance are pronounced; running on a low-end GPU, the code can match eight CPU cores, and on a high-end GPU it is faster by a factor approaching 30. Applications of the code include analysis of long photometric time series obtained by ongoing satellite missions and upcoming ground-based monitoring facilities, and Monte Carlo simulation of periodogram statistical properties.

79 ASTRONOMY AND ASTROPHYSICS↗

The Medical Science DMZ

Objective We describe use cases and an institutional reference architecture for maintaining high-capacity, data-intensive network flows (e.g., 10, 40, 100 Gbpsþ) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet filter firewalls, network intrusion detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive data sets between research institutions over national research networks. Discussion The exponentially increasing amounts of “omics” data, the rapid increase of high-quality imaging, and other rapidly growing clinical data sets have resulted in the rise of biomedical research “big data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large data sets. Maintaining data-intensive flows that comply with HIPAA and other regulations presents a new challenge for biomedical research. Recognizing this, we describe a strategy that marries performance and security by borrowing from and redefining the concept of a “Science DMZ”—a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

97 MATHEMATICS AND COMPUTING↗

The medical science DMZ: a network design pattern for data-intensive medical science

Abstract Objective We describe a detailed solution for maintaining high-capacity, data-intensive network flows (eg, 10, 40, 100 Gbps+) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet-filter firewalls, network intrusion-detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive datasets between research institutions over national research networks, and give 3 detailed descriptions of implemented Medical Science DMZs. Discussion The exponentially increasing amounts of “omics” data, high-quality imaging, and other rapidly growing clinical datasets have resulted in the rise of biomedical research “Big Data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large datasets. Maintaining data-intensive flows that comply with the Health Insurance Portability and Accountability Act (HIPAA) and other regulations presents a new challenge for biomedical research. We describe a strategy that marries performance and security by borrowing from and redefining the concept of a Science DMZ, a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

Peisert, Sean↗

Understanding checkpointing overheads on massive-scale systems : analysis of the IBM Blue Gene/P system.

Providing fault tolerance in high-end petascale systems, consisting of millions of hardware components and complex software stacks, is becoming an increasingly challenging task. Checkpointing continues to be the most prevalent technique for providing fault tolerance in such high-end systems. Considerable research has focussed on optimizing checkpointing; however, in practice, checkpointing still involves a high-cost overhead for users. In this paper, we study the checkpointing overhead seen by various applications running on leadership-class machines like the IBM Blue Gene/P at Argonne National Laboratory. In addition to studying popular applications, we design a methodology to help users understand and intelligently choose an optimal checkpointing frequency to reduce the overall checkpointing overhead incurred. In particular, we study the Grid-Based Projector-Augmented Wave application, the Carr-Parrinello Molecular Dynamics application, the Nek5000 computational fluid dynamics application and the Parallel Ocean Program application-and analyze their memory usage and possible checkpointing trends on 65,536 processors of the Blue Gene/P system.

99 GENERAL AND MISCELLANEOUS↗

Performance Engineering Research Institute SciDAC-2 Enabling Technologies Institute Final Report

Enhancing the performance of SciDAC applications on petascale systems has high priority within DOE SC. As we look to the future, achieving expected levels of performance on high-end com-puting (HEC) systems is growing ever more challenging due to enormous scale, increasing archi-tectural complexity, and increasing application complexity. To address these challenges, PERI has implemented a unified, tripartite research plan encompassing: (1) performance modeling and prediction; (2) automatic performance tuning; and (3) performance engineering of high profile applications. The PERI performance modeling and prediction activity is developing and refining performance models, significantly reducing the cost of collecting the data upon which the models are based, and increasing model fidelity, speed and generality. Our primary research activity is automatic tuning (autotuning) of scientific software. This activity is spurred by the strong user preference for automatic tools and is based on previous successful activities such as ATLAS, which has automatically tuned components of the LAPACK linear algebra library, and other re-cent work on autotuning domain-specific libraries. Our third major component is application en-gagement, to which we are devoting approximately 30% of our effort to work directly with Sci-DAC-2 applications. This last activity not only helps DOE scientists meet their near-term per-formance goals, but also helps keep PERI research focused on the real challenges facing DOE computational scientists as they enter the Petascale Era.

97 MATHEMATICS AND COMPUTING↗