Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Advanced Supercomputing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Preparing MPICH for exascale

The advent of exascale supercomputers heralds a new era of scientific discovery, yet it introduces significant architectural challenges that must be overcome for MPI applications to fully exploit its potential. Among these challenges is the adoption of heterogeneous architectures, particularly the integration of GPUs to accelerate computation. Additionally, the complexity of multithreaded programming models has also become a critical factor in achieving performance at scale. The efficient utilization of hardware acceleration for communication, provided by modern NICs, is also essential for achieving low latency and high throughput communication in such complex systems. In response to these challenges, the MPICH library, a high-performance and widely used Message Passing Interface (MPI) implementation, has undergone significant enhancements. Here, this paper presents four major contributions that prepare MPICH for the exascale transition. First, we describe a lightweight communication stack that leverages the advanced features of modern NICs to maximize hardware acceleration. Second, our work showcases a highly scalable multithreaded communication model that addresses the complexities of concurrent environments. Third, we introduce GPU-aware communication capabilities that optimize data movement in GPU-integrated systems. Finally, we present a new datatype engine aimed at accelerating the use of MPI derived datatypes on GPUs. These improvements in the MPICH library not only address the immediate needs of exascale computing architectures but also set a foundation for exploiting future innovations in high-performance computing. By embracing these new designs and approaches, MPICH-derived libraries from HPE Cray and Intel were able to achieve real exascale performance on OLCF Frontier and ALCF Aurora respectively.

Guo, Yanfei [Argonne National Laboratory (ANL), Ar↗

The Science Behind the NASA/NOAA Electronic Theater 2002

Details of the science stories and scientific results behind the Etheater Earth Science Visualizations from the major remote sensing institutions around the country will be explained. The NASA Electronic Theater presents Earth science observations and visualizations in a historical perspective. Fly in from outer space to Temple Square and the University of Utah Campus. Go back to the early weather satellite images from the 1960s see them contrasted with the latest US/Europe/Japan global weather data. See the latest images and image sequences from NASA & NOAA missions like Terra, GOES, NOAA, TRMM, SeaWiFS, Landsat 7 visualized with state-of-the art tools. A similar retrospective of numerical weather models from the 1960s will be compared with the latest "year 2002" high-resolution models. See the inner workings of a powerful hurricane as it is sliced and dissected using the University of Wisconsin Vis-5D interactive visualization system. The largest super computers are now capable of realistic modeling of the global oceans. See ocean vortexes and currents that bring up the nutrients to feed phitoplankton and zooplankton as well as draw the crill fish, whales and fisherman. See the how the ocean blooms in response to these currents and El Nino/La Nina climate regimes. The Internet and networks have appeared while computers and visualizations have vastly improved over the last 40 years. These advances make it possible to present the broad scope and detailed structure of the huge new observed and simulated datasets in a compelling and instructive manner. New visualization tools allow us to interactively roam & zoom through massive global images larger than 40,000 x 20,000 pixels. Powerful movie players allow us to interactively roam, zoom & loop through 4000 x 4000 pixel bigger than HDTV movies of up to 5000 frames. New 3D tools allow highly interactive manipulation of detailed perspective views of many changing model quantities. See the 1m resolution before and after shots of lower Manhattan and the Pentagon after the September 11 disaster as well as shots of Afghanistan from the Space Imaging IKONOS as well as debris plume images from Terra MODIS and SPOT Image. Shown by the SGI-Octane Graphics-Supercomputer are visualizations of hurricanes Michelle 2001, Floyd, Mitch, Fran and Linda. Our visualizations of these storms have been featured on the covers of the National Geographic, Time, Newsweek and Popular Science. Highlights will be shown from the NASA's large collection of High Definition TV (HDTV) visualizations clips New visualizations of a Los Alamos global ocean model, and high-resolution results of a NASA/JPL Atlantic ocean basin model showing currents, and salinity features will be shown. El Nino/La Nina effects on sea surface temperature and sea surface height of the Pacific Ocean will also be shown. The SST simulations will be compared with GOES Gulf Stream animations and ocean productivity observations. Tours will be given of the entire Earth's land surface at 500 m resolution from recently composited Terra MODIS data, Visualizations will be shown from the Earth Science Etheater 2001 recently presented over the last years in New Zealand, Johannesburg, Tokyo, Paris, Munich, Sydney, Melbourne, Honolulu, Washington, New York City, Pasadena, UCAR/Boulder, and Penn State University. The presentation will use a 2-CPU SGI/CRAY Octane Super Graphics workstation with 4 GB RAM and terabyte disk array at 2048 x 768 resolution plus multimedia laptop with three high resolution projectors. Visualizations will also be featured from museum exhibits and presentations including: the Smithsonian Air & Space Museum in Washington, IMAX theater at the Maryland Science Center in Baltimore, the James Lovell Discovery World Science museum in Milwaukee, the American Museum of Natural History (NYC) Hayden Planetarium IMAX theater, etc. The Etheater is sponsored by NASA, NOAA and the American Meteorological Society. This presentation is brought to you by the University of Utah College of Mines and Earth Sciences and, the Utah Museum of Natural History.

Hasler, A. Fritz↗

Enhancing Network Visibility and Security through Tensor Analysis

The increasing size, variety, rate of growth and change, and complexity of network data has warranted advanced network analysis and services. Tools that provide automated analysis through traditional or advanced signature-based systems or machine learning classifiers suffer from practical difficulties. These tools fail to provide comprehensive and contextual insights into the network when put to practical use in operational cyber security. In this paper, we present an effective tool for network security and traffic analysis that uses high-performance data analytics based on a class of unsupervised learning algorithms called tensor decompositions. The tool aims to provide a scalable analysis of the network traffic data and also reduce the cognitive load of network analysts and be network-expert-friendly by presenting clear and actionable insights into the network. With this being said, In this paper, we demonstrate the successful use of the tool in two completely diverse operational cyber security environments, namely, (1) security operations center (SOC) for the SCinet network at the SuperComputing (SC) Conference in 2016 and 2017 and (2) Reservoir Labs’ Local Area Network (LAN). In each of these environments, we produce actionable results for cyber security specialists including (but not limited to) (1) finding malicious network traffic involving internal and external attackers using port scans, SSH brute forcing, and NTP amplification attacks, (2) uncovering obfuscated network threats such as data exfiltration using DNS port and using ICMP traffic, and (3) finding network misconfiguration and performance degradation patterns.

97 MATHEMATICS AND COMPUTING↗

Enabling Low-Temperature (LTP) Ignition Technologies for Multi-Mode Engines through the Development of a Validated High-Fidelity LTP Model for Predicative Simulations Tools

The goal of multi-mode engine architectures is to extend current lean-burn dilution limits with renewable fuels, which requires spark plugs to deposit high energies (hundreds of mJ) in order to initiate ignition and complete combustion. At elevated energy deposition rates, spark plugs experience increased electrode erosion and thermal losses, which ultimately shortens the spark-plug lifetime and lowers ignition efficiency. As such, in order to safeguard the efficiency gains of multi-mode concepts, new and improved ignition technologies are required. Recently, non-equilibrium low-temperature plasmas (LTP) have been shown to promote energy-efficient ignition via quenching and transport of electronically excited atoms and molecules, selective radical production and fast heating of hydrocarbon/air mixtures [1-2]. Thus, LTP is seen as a technology that can potentially improve the energy extraction efficiency of fuels, while enabling kinetically controlled combustion modes towards fuel leaner conditions to realize current DOE VTO goals of improving the sustainability of future mobility [3]. Although many previous studies have demonstrated the efficacy of plasma-assisted ignition to enhance combustion, the detailed enhancement mechanisms remain largely unknown, especially for oxygenated fuels and at elevated pressures that are most relevant to practical engine conditions. These barriers hinder the development of accurate and comprehensive numerical models that seek to describe LTP-based ignition in existing engine design software tools and methods. Current state-of-the-art simulation capabilities for LTP ignition systems are in need of improvements since they deliver qualitative results only due to important limitations of existing approaches. Firstly, validated kinetic models with elementary steps for plasma discharges in oxygenated fuel/air mixtures of relevance to the transportation sector are required. Such kinetic models do not exist at present and will be developed and validated within this project. Secondly, plasma discharges and reactive mixture ignition are multi-scale, unsteady processes requiring high-performance numerical methods and software that execute efficiently on DOE supercomputers. Such software does not exist at present and will be developed and applied to practical LTP ignition scenarios as part of this project. Thirdly, experimental databases that are tailored to serve as benchmark in support of the development of predictive computational models of LTP ignition do not exist and will be part of this project.

33 ADVANCED PROPULSION SYSTEMS↗

Cooperative Research and Development Agreement With Georgetown University Report: National Institutes of Health, National Center for Advancing Translational Sciences Clinical and Translational Science Award

Oak Ridge National Laboratory (ORNL) is participating with Georgetown University (GU) as a subrecipient in response to the National Institutes of Health (NIH), National Center for Advancing Translational Sciences (NCATS) Clinical and Translational Science Award (CTSA) (U54 Clinical Trial Optional) funding opportunity announcement. This Cooperative Research and Development Agreement is put in to place to facilitate the development and implementation of clinical interventions that demonstrably improve human health is currently a complex, recursive, and inefficient process that leads to delays of years or decades before discoveries in biomedical research result in health benefits for patients and communities. NCATS conducts and supports research in the science of translation, to discover the mechanistic and operational principles of the intervention development and dissemination process, thereby providing the scientific foundation for improvements in translational efficiency that will accelerate the realization of interventions that improve human health. Under NCATS’ leadership, the CTSA Program supports a national network of medical research institutions called hubs. GU is the lead institution in one of the NIH hubs that was created as a result of a previous NIH CTSA. The missions of the GU have historically included the advancement of health through research in the clinical and biomedical sciences, the education of future leaders in medical and nursing practice and academia, and the provision of compassionate and scientifically competent patient care and service to the Washington, DC community and the nation. GU is the lead institution for the Georgetown-Howard Universities Center for Clinical and Translational Science (GHUCCTS), a multi-institutional partnership of medical research institutions forged from a desire to promote clinical research and translational science. Through multiple collaborations among these institutions, GHUCCTS is transforming clinical research and translational science in order to bring new scientific advances to health care. Oak Ridge National Laboratory is the Department of Energy's (DOE) largest science and energy laboratory. Managed since April 2000 by a partnership of the University of Tennessee and Battelle, ORNL was established in 1943 as a part of the secret Manhattan Project to pioneer a method for producing and separating plutonium. During the 1950s and 1960s, ORNL became an international center for the study of nuclear energy and related research in the physical and life sciences. With the creation of DOE in the 1970s, ORNL's mission broadened to include a variety of energy technologies and strategies. Today the laboratory supports the nation with a peacetime science and technology mission that is just as important as, but very different from, its role during the Manhattan Project. ORNL is home to the world's premier center for high performance supercomputing to enable scientific discovery. ORNL has extensive expertise in various areas of computer science that are uniquely situated to support GU. Additionally, ORNL’s leading computational user facilities present a unique opportunity to leverage the largest scale machines for open science in support of the stated mission of the NCATS CTSA. ORNL's partnership with GU will offer unparalleled opportunity in data analytics, deep-learning, artificial intelligence, and urban dynamics.

59 BASIC BIOLOGICAL SCIENCES↗

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Abstract Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing (i) reduction of data loading time up to 4.2 times compared with a conventional method and (ii) linear scaling performance for training up to 1024 GPUs on both Summit and Perlmutter.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Scale-up Unlearnable Examples Learning with High-performance Computing

Recent advancements in AI models, like ChatGPT, are structured to retain user interactions, which could inadvertently include sensitive healthcare data. In the healthcare field, particularly when radiologists use AI-driven diagnostic tools hosted on online platforms, there is a risk that medical imaging data may be repurposed for future AI training without explicit consent, spotlighting critical privacy and intellectual property concerns around healthcare data usage. Addressing these privacy challenges, a novel approach known as Unlearnable Examples (UEs) has been introduced, aiming to make data unlearnable to deep learning models. A prominent method within this area, called Unlearnable Clustering (UC), has shown improved UE performance with larger batch sizes but was previously limited by computational resources (e.g., a single workstation). To push the boundaries of UE performance with theoretically unlimited resources, we scaled up UC learning across various datasets using Distributed Data Parallel (DDP) training on the Summit supercomputer. Our goal was to examine UE efficacy at high-performance computing (HPC) levels to prevent unauthorized learning and enhance data security, particularly exploring the impact of batch size on UE’s unlearnability. Utilizing the robust computational capabilities of the Summit, extensive experiments were conducted on diverse datasets such as Pets, MedMNist, Flowers, and Flowers102. Our findings reveal that both overly large and overly small batch sizes can lead to performance instability and affect accuracy. However, the relationship between batch size and unlearnability varied across datasets, highlighting the necessity for tailored batch size strategies to achieve optimal data protection. The use of Summit’s high-performance GPUs, along with the efficiency of the DDP framework, facilitated rapid updates of model parameters and consistent training across nodes. Our results underscore the critical role of selecting appropriate batch sizes based on the specific characteristics of each dataset to prevent learning and ensure data security in deep learning applications. The source code is publicly available at https: // github. com/ hrlblab/ UE_ HPC .

Zhu, Yanfan [Vanderbilt University, Nashville, TN,↗

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multi-threaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0x and 6.8x on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3x over the multi-thread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multithreaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0× and 6.8× on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3× over the multithread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗

NASA Next-Generation Sensing Technologies for Earth & Planetary Science

Dr. Ved Chirayath's seminar will highlight new sensing technologies and airborne platforms he is developing for Earth & Space Science as director of the NASA Silicon Valley Laboratory for Advanced Sensing. His talk will feature recent work on Fluid Lensing, the first remote sensing technology capable of imaging through ocean waves in 3D at sub-cm resolutions, MiDAR, a next-generation active hyperspectral remote sensing and optical communications instrument, airborne gas sensing of multipollutant combustion sources, and a plasma actuated unmanned aerial vehicle (UAV) that utilized high-voltage dielectric discharge devices to achieve the first plasma controlled flight in history.Fluid Lensing and the NASA FluidCam instrument have been used extensively to provide distortion-free 3D multispectral imagery from UAVs of shallow marine systems around the world. MiDAR is being deployed on aircraft and underwater remotely operated vehicles (ROVs) as a new method to remotely sense living and nonliving structures in extreme environments as an analog for future Ocean Worlds robotic exploration missions. Finally, Chirayath will present preliminary results from NeMO-Net, a supercomputer-based neural network that uses a citizen science video game for global multimodal coral reef benthic habitat mapping, fusing remote sensing data from Fluid Lensing, MiDAR, NASA's Earth Observing System, and commercial satellites, to better understand the present and past dynamics of shallow marine systems. Together, these maturing technologies present promising new ways in which to explore terrestrial, marine, and aerial systems on Earth, and, ultimately, aid in the search for extraterrestrial life within our solar system and beyond.

Chirayath, Ved↗

NASA Earth Exchange – Current overview of climate and wildfire-oriented works

NASA Earth Exchange (NEX) combines state-of-the-art supercomputing, Earth system modeling, and NASA remote sensing data feeds to deliver a work environment for exploring and analyzing petabyte-scale datasets covering large regions, continents, or the globe. As an accessible platform, NEX can accelerate fundamental research, develop new applications, and reduce overall project costs by providing the research community with data, software, and high-end computing power. Two research thrusts for the NEX community are developing and distributing the NEX-GDDP-CMIP6 downscaled climate dataset and the community-driven wildfire research. NEX-GDDP-CMIP6 dataset is comprised of global downscaled climate scenarios derived from the General Circulation Model (GCM) runs conducted under the Coupled Model Intercomparison Project Phase 6 (CMIP6) and across two of the four “Tier 1” greenhouse gas emissions scenarios known as Shared Socioeconomic Pathways (SSPs). The wildfire research thrust leverages geostationary and low earth orbit remote sensing platforms and advanced modeling (WRF). We present a high-level overview of the NEX community, discuss relevant datasets and show several visualization techniques of data used in the wildfire research activity.

NEX↗

ExM:System Support for Extreme-Scale, Many-Task Applications

The ever-increasing power of supercomputer systems is both driving and enabling the emergence of new problem-solving methods that require the effi cient execution of many concurrent and interacting tasks. Methodologies such as rational design (e.g., in materials science), uncertainty quanti fication (e.g., in engineering), parameter estimation (e.g., for chemical and nuclear potential functions, and in economic energy systems modeling), massive dynamic graph pruning (e.g., in phylogenetic searches), Monte-Carlo- based iterative fi xing (e.g., in protein structure prediction), and inverse modeling (e.g., in reservoir simulation) all have these requirements. These many-task applications frequently have aggregate computing needs that demand the fastest computers. For example, proposed next-generation climate model ensemble studies will involve 1,000 or more runs, each requiring 10,000 cores for a week, to characterize model sensitivity to initial condition and parameter uncertainty. The goal of the ExM project is to achieve the technical advances required to execute such many-task applications efficiently, reliably, and easily on petascale and exascale computers. In this way, we will open up extreme-scale computing to new problem solving methods and application classes. In this document, we report on combined technical progress of the collaborative ExM project, and the institutional financial status of the portion of the project at University of Chicago, over the rst 8 months (through April 30, 2011)

97 MATHEMATICS AND COMPUTING↗

Roadrunner Supercomputer Breaks the Petaflop Barrier

At 3:30 a.m. on May 26, 2008, Memorial Day, the "Roadrunner" supercomputer exceeded a sustained speed of 1 petaflop/s, or 1 million billion calculations per second. The sustained performance makes Roadrunner more than twice as fast as the current number 1 system on the TOP500 list. The best sustained performance to date is 74.5% efficiency, 1.026 petaflop/s. "Petaflop/s" is computer jargon—peta signifying the number 1 followed by 15 zeros (sometimes called a quadrillion) and flop/s meaning "double-precision floating point operations per second." Los Alamos held the fastest supercomputer title in 1993 with the Thinking Machines CM-5, and inaugurated the supercomputer era, assisting in the development of the Cray-1 in 1976. The Laboratory and IBM go all the way back to the first card-programmable calculators, used at Los Alamos in 1949. Los Alamos also housed serial number 1 of the IBM 704 in 1956. The Roadrunner supercomputer, developed by IBM in partnership with the Laboratory and the National Nuclear Security Administration, uses commercially available hardware, including aspects of commercial game console and graphics technologies. Because of its off-the-shelf components, the computer costs significantly less than a one-of-a-kind machine. It also uses a Linux operating system. The secret to its record-breaking performance is a unique hybrid design. Each compute node in this cluster consists of two AMD OpteronTM dual-core processors plus four PowerXCell 8iTM processors used as computational accelerators. The accelerators used in Roadrunner are a special IBM-developed variant of the Cell processor used in the Sony PlayStation® 3. The node-attached Cell accelerators are what make Roadrunner different than typical clusters. The Lab and IBM have been working on Roadrunner since 2006, but collaboration on Cell dates back to 2002. The first phase of the project included delivery of an initial Opteron-only cluster that operates at a speed of 71 teraflop/s. This initial system has been in full production at Los Alamos for almost a year, and Laboratory researchers are using this machine for classified weapons applications. The full-scale Roadrunner machine operates more than 10 times faster than the current installed system. Phase 2 of the Roadrunner project was completed in October 2007. Two external assessments, one by NNSA headquarters and one by an independent team of high performance computing experts, evaluated the machine's potential use for Laboratory applications, the Laboratory's ability to successfully manage the computing system, IBM's ability to deliver the product, and whether computer programs could be adapted to the new system. Based on the positive outcome of the assessments, the Laboratory and NNSA decided to pursue the final phase of the Roadrunner project. The powerful cluster of nodes will process information enabling the Laboratory to use Roadrunner for advanced physics and predictive simulations of complex scientific processes. Weapons science applications that can be processed by Roadrunner are applicable to all three of the U.S. Department of Energy weapons laboratories. The machine will also be well equipped to tackle the intricacies of modeling processes, ranging from the biomolecular to the cosmological. In addition, Los Alamos intends to purchase additional Roadrunner resources to support open science and technology applications. The full-scale Roadrunner system, named in honor of New Mexico's speedy state bird, broke the performance record at IBM's Poughkeepsie, New York, facility. The full machine will be moved to Los Alamos beginning in July and housed in the Nicholas Metropolis Center for Modeling and Simulation. The first computing applications are expected to begin running on the machine in January 2009. The cost of all phases of the Roadrunner project is approximately $120 million. More than 200 Laboratory employees have been involved in this effort. For more information about Roadrunner at Los Alamos National Laboratory, see the Roadrunner website at http://www.lanl.gov/roadrunner.

CELL BASED PROCESSOR↗

Numerical simulation of rotorcraft

The objective of the research is to develop and validate accurate, user-oriented viscous CFD codes (with inviscid options) for three-dimensional, unsteady aerodynamic flows about arbitrary rotorcraft configurations. Unsteady, three-dimensional Euler and Navier-Stokes codes are developed, adapted, and extended to rotor-body combinations. Flow solvers are coupled with zonal grid topologies, including rotating and nonrotating blocks. Special grid clustering and wave-fitting techniques were developed to capture low-level radiating acoustic waves. Significant progress was made in computing the propagation of acoustic waves due to the interaction of a concentrated vortex and a helicopter airfoil. The need for higher-order schemes was firmly established in relatively inexpensive two-dimensional calculations. In three dimensions, the number of grid points required to capture the low-level acoustic waves becomes very large, so that large supercomputer memory becomes essential. Good agreement was obtained between the numerical results obtained with a thin-layer Navier-Stokes code and experimental data from a model rotor. In addition, several nonrotating configurations that are sometimes proposed to simulate rotor blade tips in conventional wind tunnels were examined, and the complex flow around the radical tip shape of the world's fastest helicopter is under investigation. These studies demonstrate the flexibility and power of CFD to gain physical insight, study novel ideas, and examine various possibilities that might be difficult or impossible to set up in physical experiments. As a prelude to studies of rotor-body aerodynamic interactions, a preliminary grid topology and moving-interface strategy were developed. A new Euler/Navier-Stokes code using these techniques computes the vortical wake directly, rather than modeling it, as in most previous rotorcraft studies. Several hover cases were run for conventional and advanced-geometry blades. Numerical schemes using multi-zones and/or adaptive grids appear to be necessary to simulate the complex vortical flows in rotor wakes.

Mccroskey, William J.↗

Final Progress Report: Collaborative Research: Decadal-to-Centennial Climate & Climate Change Studies with Enhanced Variable and Uniform Resolution GCMs Using Advanced Numerical Techniques

The joint U.S-Canadian project has been devoted to: (a) decadal climate studies using developed state-of-the-art GCMs (General Circulation Models) with enhanced variable and uniform resolution; (b) development and implementation of advanced numerical techniques; (c) research in parallel computing and associated numerical methods; (d) atmospheric chemistry experiments related to climate issues; (e) validation of regional climate modeling strategies for nested- and stretched-grid models. The variable-resolution stretched-grid (SG) GCMs produce accurate and cost-efficient regional climate simulations with mesoscale resolution. The advantage of the stretched grid approach is that it allows us to preserve the high quality of both global and regional circulations while providing consistent interactions between global and regional scales and phenomena. The major accomplishment for the project has been the successful international SGMIP-1 and SGMIP-2 (Stretched-Grid Model Intercomparison Project, phase-1 and phase-2) based on this research developments and activities. The SGMIP provides unique high-resolution regional and global multi-model ensembles beneficial for regional climate modeling and broader modeling community. The U.S SGMIP simulations have been produced using SciDAC ORNL supercomputers. Collaborations with other international participants M. Deque (Meteo-France) and J. McGregor (CSIRO, Australia) and their centers and groups have been beneficial for the strong joint effort, especially for the SGMIP activities. The WMO/WCRP/WGNE endorsed the SGMIP activities in 2004-2008. This project reflects a trend in the modeling and broader communities to move towards regional and sub-regional assessments and applications important for the U.S. and Canadian public, business and policy decision makers, as well as for international collaborations on regional, and especially climate related issues.

58 GEOSCIENCES↗

Final Progress Report submitted via the DOE Energy Link (E-Link) in June 2009 [Collaborative Research: Decadal-to-Centennial Climate & Climate Change Studies with Enhanced Variable and Uniform Resolution GCMs Using Advanced Numerical Techniques]

The joint U.S-Canadian project has been devoted to: (a) decadal climate studies using developed state-of-the-art GCMs (General Circulation Models) with enhanced variable and uniform resolution; (b) development and implementation of advanced numerical techniques; (c) research in parallel computing and associated numerical methods; (d) atmospheric chemistry experiments related to climate issues; (e) validation of regional climate modeling strategies for nested- and stretched-grid models. The variable-resolution stretched-grid (SG) GCMs produce accurate and cost-efficient regional climate simulations with mesoscale resolution. The advantage of the stretched grid approach is that it allows us to preserve the high quality of both global and regional circulations while providing consistent interactions between global and regional scales and phenomena. The major accomplishment for the project has been the successful international SGMIP-1 and SGMIP-2 (Stretched-Grid Model Intercomparison Project, phase-1 and phase-2) based on this research developments and activities. The SGMIP provides unique high-resolution regional and global multi-model ensembles beneficial for regional climate modeling and broader modeling community. The U.S SGMIP simulations have been produced using SciDAC ORNL supercomputers. The results of the successful SGMIP multi-model ensemble simulations of the U.S. climate are available at the SGMIP web site (http://essic.umd.edu/~foxrab/sgmip.html) and through the link to the WMO/WCRP/WGNE web site: http://collaboration.cmc.ec.gc.ca/science/wgne. Collaborations with other international participants M. Deque (Meteo-France) and J. McGregor (CSIRO, Australia) and their centers and groups have been beneficial for the strong joint effort, especially for the SGMIP activities. The WMO/WCRP/WGNE endorsed the SGMIP activities in 2004-2008. This project reflects a trend in the modeling and broader communities to move towards regional and sub-regional assessments and applications important for the U.S. and Canadian public, business and policy decision makers, as well as for international collaborations on regional, and especially climate related issues.

54 ENVIRONMENTAL SCIENCES↗

Breaking the mold: Overcoming the time constraints of molecular dynamics on general-purpose hardware

The evolution of molecular dynamics (MD) simulations has been intimately linked to that of computing hardware. For decades following the creation of MD, simulations have improved with computing power along the three principal dimensions of accuracy, atom count (spatial scale), and duration (temporal scale). Since the mid-2000s, computer platforms have, however, failed to provide strong scaling for MD, as scale-out central processing unit (CPU) and graphics processing unit (GPU) platforms that provide substantial increases to spatial scale do not lead to proportional increases in temporal scale. Important scientific problems therefore remained inaccessible to direct simulation, prompting the development of increasingly sophisticated algorithms that present significant complexity, accuracy, and efficiency challenges. While bespoke MD-only hardware solutions have provided a path to longer timescales for specific physical systems, their impact on the broader community has been mitigated by their limited adaptability to new methods and potentials. In this work, we show that a novel computing architecture, the Cerebras wafer scale engine, completely alters the scaling path by delivering unprecedentedly high simulation rates up to 1.144 M steps/s for 200 000 atoms whose interactions are described by an embedded atom method potential. This enables direct simulations of the evolution of materials using general-purpose programmable hardware over millisecond timescales, dramatically increasing the space of direct MD simulations that can be carried out. In this paper, we provide an overview of advances in MD over the last 60 years and present our recent result in the context of historical MD performance trends.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Initial Development of Fusion Magnet Simulation Capabilities for Performance and Safety Evaluation Using the MOOSE Framework

Fusion energy holds the promise of being a transformative technology as a carbon-neutral, sustainable source of energy. Whole device modeling and the development of fusion digital twins will be increasingly important for emerging fusion device concepts at both national laboratories and within the commercial fusion industry. However, meeting the challenge of whole device modeling of fusion energy devices requires robust, multiphysics, multiscale modeling and simulation technologies capable of running on large-scale supercomputers. Detailed analysis of individual systems at-scale is also required to ensure safe and efficient operation as well as provide the safety basis for future device designs and licensing activities. In a tokamak, toroidal and poloidal magnets confine and shape the fusion plasma to promote the fusion reaction. High plasma temperatures and high magnetic field requirements in modern design concepts (leading to high amounts of energy stored within each magnet) impose electrical, thermal, and mechanical loads on the magnet components, which in turn impacts the safety considerations of the magnet and their supporting systems. Idaho National Laboratory (INL) has a history of working in this space, including development and benchmarking of the Magnetic System Circuitry Analysis Program (MSCAP) and Magnet Arcing (MAGARC) codes to study magnet quench events; notably, MAGARC was used to study quenching during the ITER Engineering Design Activity. However, these legacy codes and capabilities are not parallel and scalable, and new tools are required for future advances in this area, which leads to the INL-developed Multiphysics Object-Oriented Simulation Environment (MOOSE) framework. Developed originally for fission reactor systems under United States Department of Energy, Office of Nuclear Energy modeling and simulation programs, the MOOSE framework has been well-suited to multiscale, multiphysics modeling and simulation needs for nuclear systems. The framework is open-source, well-tested, under continuous development and deployment, and developed to a Nuclear Quality Assurance, Level 1 software quality standard. MOOSE has also been used in the fusion space previously in several projects: INL’s Tritium Migration Analysis Program, Version 8 (TMAP8) for tritium migration, UK Atomic Energy Authority’s A Unified Resource for OpenMC (fusion) Reactor Applications (AURORA) code for fusion thermo-mechanical and neutronics analysis, and Argonne National Laboratory’s Cardinal for high-fidelity computational fluid dynamics and neutronics. However, to model superconducting magnets, several MOOSE enhancements are required: additions to the current MOOSE electromagnetic capabilities, new material libraries for superconductors of interest (such as YBCO), as well as fusion-specific models for thermo-mechanics. This talk will discuss initial development activities to build these capabilities in MOOSE, focusing on initial validation and benchmarking activities. Proposed coupling workflows and future work to support the simulation of fusion magnets and magnet structural assemblies for performance and safety evaluation in MOOSE will also be discussed.

70 - PLASMA PHYSICS AND FUSION TECHNOLOGY↗