Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Automated Discovery for Emulytics

Sandia has an extensive background in cybersecurity research and is currently extending its state-of-the-art modeling via emulation capability. However, a key part of Sandia's modeling methodology is the discovery and specification of the information-system under study, and the ability to recreate that specification with the highest fidelity possible in order to extrapolate meaningful results. This work details a method to conduct information system discovery and develop tools to enable the creation of high-fidelity emulation models that can be used to enable assessment of our infrastructure information system security posture and potential system impacts that could result from cyber threats. The outcome are a set of tools and techniques to go from network discovery of operational systems to emulating complex systems. As a concrete usecase, we have applied these tools and techniques at Supercomputing 2016 to model SCinet, the world's largest research network. This model includes five routers and nearly 10,000 endpoints which we have launched in our emulation platform.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Implementing Software Resiliency in HPX for Extreme Scale Computing

The DOE Office of Science Exascale Computing Project (ECP) outlines the next milestones in the supercomputing domain. The target computing systems under the project will deliver 10x performance while keeping the power budget under 30 megawatts. With such large machines, the need to make applications resilient has become paramount. The benefits of adding resiliency to mission critical and scientific applications, includes the reduced cost of restarting the failed simulation both in terms of time and power. Most of the current implementation of resiliency at the software level makes use of a Coordinated Checkpoint and Restart (C/R). This technique of resiliency generates a consistent global snapshot, also called a checkpoint. Generating snapshots involves global communication and coordination and is achieved by synchronizing all running processes. The generated checkpoint is then stored in some form of persistent storage. On failure detection, the runtime initiates a global rollback to the most recent previously saved checkpoint. This involves aborting all running processes, rolling them back to the previous state and restarting them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Compare linear-system solver and preconditioner stacks with emphasis on GPU performance and propose phase-2 NGP solver development pathway

The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. The primary physics codes in the ExaWind project are Nalu-Wind, which is an unstructured-grid solver for the acoustically incompressible Navier-Stokes equations, and OpenFAST, which is a whole-turbine simulation code. The Nalu-Wind model consists of the mass-continuity Poisson-type equation for pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem, the moving meshes greatly affect overall solver costs as reinitialization of matrices and recomputation of preconditioners is required at every time step. In this report we evaluated GPU-performance baselines for the linear solvers in the Trilinos and hypre solver stacks using two representative Nalu-Wind simulations: an atmospheric boundary layer precursor simulation on a structured mesh, and a fixed-wing simulation using unstructured overset meshes. Both strong-scaling and weak-scaling experiments were conducted on the OLCF supercomputer Summit and similar proxy clusters. We focused on the performance of multi-threaded Gauss-Seidel and two-stage Gauss-Seidel that are extensions of classical Gauss-Seidel; of one-reduce GMRES, a communication-reducing variant of the Krylov GMRES; and algebraic multigrid methods that incorporate the afore-mentioned methods. The team has established that AMG methods are capable of solving linear systems arising from the fixed-wing overset meshes on CPU, a critical intermediate result for ExaWind FY20 Q3 and Q4 milestones. For the fixed-wing strong-scaling study (model with 3M grid-points), the team identified that Nalu-Wind simulations with the new Trilinos and hypre solvers scale to modest GPU counts, maintaining above 70% efficiency up to 6 GPUs. However, there still remain significant bottlenecks to performance: matrix assembly (hypre), AMG setup (hypre and Trilinos) In the weak-scaling experiments (going from 0.4M to 211M gridpoints), it's shown that the solver apply phases are faster on GPUs, but that Nalu-Wind simulation times grow, primarily due to the multigrid-setup process. Finally, based on the report outcomes, we propose a linear solver path-forward for the remainder of the ExaWind project. Near term, the NREL team will continue their work on GPU-based linear-system assembly. They will also investigate how the use of alternatives to the NVIDIA UVM (unified virtual memory) paradigm affects performance. Longer term, the NREL team will evaluate algorithmic performance on other types of accelerators and merge their improvements back to the main hypre repository branch. Near term, the Trilinos team will address performance bottlenecks identified in this milestone, such as implementing a GPU-based segregated momentum solve and reusing matrix graphs across linear-system assembly phases. Longer term, the Trilinos team will do detailed analysis and optimization of multigrid setup.

17 WIND ENERGY↗
DOE OSTI2020

Simulations of Laser Experiments to Study the Origin of Cosmic Magnetic Fields

Magnetic fields are ubiquitous in the universe; however, their origin is not fully understood. Cosmologists and astrophysicists have proposed a variety of ways in which small seed magnetic fields could be created. It is widely thought that the much larger values of the cosmic magnetic fields that we observe are a result of the amplification of these seed fields by the nonlinear turbulent dynamo mechanism. Such a mechanism had not yet been demonstrated in a controlled laboratory environment. We conceived experiments designed to demonstrate and study the nonlinear turbulent dynamo mechanism in the laboratory. These experiments characterize the distribution of turbulent energy among the velocity, magnetic field, and density fluctuations, providing a comprehensive picture of the energy cascade in a magnetized, turbulent plasma. The experiments build on the pathfinder experiments we have conducted on the Vulcan laser at the Rutherford-Appleton Laboratory in the UK and the OMEGA laser at Laboratory for Laser Energetics at Rochester. They utilize the high-intensity lasers at the Omega Laser Facility, the National Ignition Facility (NIF) at LLNL and the Laser Megajoule (LMJ) Facility in France – the largest laser facilities in the world. The goal of this project was to design and model the highly demanding experiments through simulation campaigns using FLASH, a highly capable radiation-MHD code we have developed, and large-scale 3D simulations on the Mira supercomputer at ANL. The simulations were vital to ensuring the experiments achieve the strong turbulence and large magnetic Reynolds numbers required for the nonlinear turbulent dynamo mechanism to operate. The simulations were also critical to determining when to fire the diagnostics, since the experiments last tens of nanoseconds but the signals of the strongly amplified magnetic fields last only a few nanoseconds. Finally, high-fidelity, validated FLASH simulations were crucial to interpreting the results of the experiments. By combining theory, experiments, and simulations, we were able to succeed in demonstrating and characterizing the nonlinear turbulent dynamo mechanism. The effort leveraged (1) the emergence of the Flash Center for Computational Science as a leader in numerical modeling of academic High Energy Density Physics (HEDP) experiments; (2) the FLASH code, which has been developed by the Flash Center and is a leadership-class, multi- physics, publicly available community code that has been applied successfully in many laser-driven experiments; (3) the expertise acquired through pathfinder experiments we performed on the Vulcan laser at the Rutherford-Appleton Laboratory in the UK and on the Omega laser facility; (4) the experience we have gained in designing and interpreting these experiments using validated simulations done with FLASH; and (5) the close collaborations we have established with outstanding experimental groups in the US and abroad. The broader impacts of the project are two-fold: (1) the work furthered the transformation of the academic community’s ability to design and analyze HEDP experiments at large laser facilities that is happening through the availability of FLASH and its widespread adoption by the academic community – a community that previously had had limited access to open and validated hydro/MHD simulation tools for HEDP experiments; and (2) the work trained young scientists to design and interpret HEDP experiments using validated simulations – a critical national need.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Parallel-in-Time Simulation of Lindblad's Equation

Constructing fast quantum logic gates is critical to building a scalable quantum computer. We consider a qudit, a quantum version of a bit that can take an arbitrary number of states, coupled with a cavity. In this project, we wish to force the qudit to reach the 0-state, for any possible initial state. The coupled system changes in time according to Lindblad’s equation, an ordinary differential equation on the density matrix of the quantum system. Lindblad’s equation contains some parameters that we can control, so-called control functions. We seek control functions which force the qudit to the 0-state within 2 microseconds, which is much faster than what is currently done in practice. The search method is gradient descent, a numerical optimization method that uses gradient information to iteratively improve the control parameters. My contribution to this project is an attempt to speed up the computation of the gradient. It currently takes about 40 seconds to compute the gradient which involves solving a set of ODEs sequentially. Current supercomputers have thousands of cores, but sequential computations can only make use of 1 core at a time. We wish to divide up the work better, so that we can use many more cores at once. To this end, we have implemented the Multigrid Reduction in Time (MGRIT) algorithm. We perform a systematic parameter search on how to best apply this algorithm. Results indicate a 25 percent speed up for solving Lindblad’s equation and determining how close the final state is the 0-state.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Current Needs in Radiochemistry

There are a number of current needs in the development of the Pu and Am separations that underpin larger interest areas including nuclear waste issues, nuclear weapons, nuclear forensics, nuclear fuels and similarly related fields. In the large areas of interest, there are a number of sub-fields that include corrosion chemistry, pyrochemistry, solution-phase chemistry, rapid separations, environmental speciation, and other similar efforts. However, in all of the these fields it is important to realize that the underlying chemistry that underpins these efforts is similar in that Pu and Am are bound to ligands that can be in the solid, aqueous, or gas phase and separations occurs due to the preferential binding of the ligand to Pu or Am based on a number of factors that include pH, concentration, oxidation state, etc. In the past, most of the Pu and Am chemistry has occurred in limited scope within universities where micro-scale and bench-top scale chemistry are the dominate investigative theme. At US national laboratory and their international counterparts, larger scale reactions have been attempted such that industrial scale chemistry has been completed. Some of this technology has been harnessed in radiopharmaceuticals and nuclear fuel reprocessing. However, such bench-top research is expensive to due to the safety and security that is concerned when dealing with Pu, and scale-up of reactions is even more resource intensive. In an effort to offset this expensive, it would be of interest to explore the ability to predict selectivity based on the computational modeling of actinide chemistry. This has historically been limited by the abilities of large supercomputers, but with the emphasis on developing exascale computers within the US and similar efforts by other countries this predictive modeling has potential to produce significant results that would yield improved understanding of actinide chemistry.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2020

US Department of Energy, Office of Science High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility

Oak Ridge National Laboratory's (ORNL's) Leadership Computing Facility (OLCF) continues to surpass its operational target goals: supporting users; delivering fast, reliable computational ecosystems; creating innovative solutions for high performance computing (HPC) needs; and managing risks, safety, and security associated with operating some of the most powerful computers in the world. The results can be seen in the cutting-edge science conducted by users and the praise from the research community. Calendar year (CY) 2019 was a big year as OLCF staff ran five world-class resources (the leadershipclass computers Titan and Summit, the large analysis cluster called Eos, and the massive parallel filesystems called Atlas and Alpine)) and also began power and cooling upgrades for a 2021 exascale system called Frontier. While continuing exceptional operation of Titan, Eos, and Rhea, the OLCF released the Summit supercomputer for production on January 1, 2019. Summit debuted as the most capable and efficient system in its class and has been recognized as the most powerful system in the world for its performance on both the high performance linpack (HPL) and conjugate gradient (HPCG) benchmark applications since June 2018 according to TOP500. Summit represents the culmination of a multiyear effort between the OLCF, IBM, NVIDIA, and Mellanox to deliver a system that is unmatched for modeling, simulation, data analysis, and learning. To hit the ground running with science-ready applications on day one, application teams worked closely with the OLCF through the Center for Accelerated Application Readiness (CAAR) program for years in advance of the Summit deployment. CY 2019 was filled with outstanding results and accomplishments: a very high rating from users on overall satisfaction for the sixth year in a row; a tremendous amount of core-hours delivered to researchers from two leadership-class systems; and success in delivering on the allocation split of roughly 60%, 30%, and 10% of core-hours offered for the Innovative and Novel Computational Impact on Theory and Experiment (INCITE), Advanced Scientific Computing Research Leadership Computing Challenge (ALCC), and Director's Discretionary (DD) programs, respectively (see Operational Performance section). These accomplishments, coupled with the high utilization rates (overall and capability usage), represent the fulfillment of the promise of both leadership-class machines: efficient facilitation of leadership-class computational applications. Table ES.1 presents a summary of the 2019 OLCF metric targets and the associated results. More information can be found in the Operational Performance section for each OLCF resource. The scientific accomplishments of OLCF users are a strong indication of long-term operational success, with publications this year in such notable journals and publications as Nature, Nature Physics, Nature Plants, Physical Review X, Journal of the American Physical Society, Cell, Nano Letters, and Trends in Biotechnology. Crucial domain-specific discoveries facilitated by resources at the OLCF are described in the High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility (OAR) Strategic Results section. For example, researchers used Summit to pinpoint and understand the production of proteins from genetic information, including mutations and the functional expression of disease (Section 8.2).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Natural Resources Assessment for the SNS Second Target Station, Oak Ridge, Tennessee

The US Department of Energy’s Oak Ridge National Laboratory (ORNL) is a leading institution in advanced materials, supercomputing, neutrons, and nuclear science. As a research laboratory managed by UT-Battelle, LLC for DOE, ORNL has national priorities in energy, security, and scientific discovery that necessitate facility improvements and expansions. DOE is also committed to environmental stewardship. The laboratory is located on the ~32,000-acre Oak Ridge Reservation (ORR), much of which is categorized as a National Environmental Research Park and a state Wildlife Management Area. DOE works with the Tennessee Wildlife Resources Agency (TWRA), Tennessee Department of Environment and Conservation (TDEC), US Fish and Wildlife Service (USFWS), US Department of Agriculture, and other agencies to serve as an effective steward of the ORR. Accordingly, project managers must conform to environmental regulations, agreements, and policies at the federal, state, and institutional levels. Per 40 CFR (Code of Federal Regulations) 1508.14, potential effects on research and science education on the National Environmental Research Park represent potential impacts of federal actions. Moreover, federal actions that affect the quantity and quality of hunting opportunities and deer reduction harvest on the Oak Ridge Wildlife Management Area must be considered whenever other aspects of the human environment (as defined by NEPA) are affected. The Spallation Neutron Source (SNS) is a premiere facility at ORNL that provides advanced capabilities in neutron scattering to promote new discoveries and research opportunities in material sciences, physics, chemistry, biological sciences, and others. A conceptual design for a Second Target Station (STS) has been in consideration for several years. The STS is intended to complement and enhance existing ORNL capabilities, notably research and exploration of complex materials. The proposed STS will involve development of existing natural areas on the ORR, which might contain sensitive resources that require mitigation or avoidance in accordance with existing policies and regulation. This report summarizes current knowledge of natural and cultural resources within the STS project area. At the time of this report, the proposed STS project consisted of an operations area comprising 55.4 acres (22.4 ha) and a total review area for potential construction comprising ~224 acres (90.6 ha). The review area is located primarily within forested natural areas of the ORR with minor development in the form of power-line rights-of-way and secondary/graveled roads (Figure 1). The primary goal was to evaluate potential effects on sensitive resources that might result from development of the STS. In addition to onthe-ground surveys during summer 2009 and fall 2019 to summer 2020 by ORNL Natural Resources Management Program and Aquatic Ecology Group staff, this report makes use of historical (pre-1995) and contemporary (1995 to present) data from additional confirmed sources (e.g., TDEC). Likewise, forest conditions were compiled primarily from a 2013 forest inventory effort for Forest Management Compartment 17 and supplemented with limited ground observations in 2019. The individuals who obtained and compiled the data that are presented here are familiar with and routinely assess sensitive resources on the ORR.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Cooperative Research and Development Agreement With Georgetown University Report: National Institutes of Health, National Center for Advancing Translational Sciences Clinical and Translational Science Award

Oak Ridge National Laboratory (ORNL) is participating with Georgetown University (GU) as a subrecipient in response to the National Institutes of Health (NIH), National Center for Advancing Translational Sciences (NCATS) Clinical and Translational Science Award (CTSA) (U54 Clinical Trial Optional) funding opportunity announcement. This Cooperative Research and Development Agreement is put in to place to facilitate the development and implementation of clinical interventions that demonstrably improve human health is currently a complex, recursive, and inefficient process that leads to delays of years or decades before discoveries in biomedical research result in health benefits for patients and communities. NCATS conducts and supports research in the science of translation, to discover the mechanistic and operational principles of the intervention development and dissemination process, thereby providing the scientific foundation for improvements in translational efficiency that will accelerate the realization of interventions that improve human health. Under NCATS’ leadership, the CTSA Program supports a national network of medical research institutions called hubs. GU is the lead institution in one of the NIH hubs that was created as a result of a previous NIH CTSA. The missions of the GU have historically included the advancement of health through research in the clinical and biomedical sciences, the education of future leaders in medical and nursing practice and academia, and the provision of compassionate and scientifically competent patient care and service to the Washington, DC community and the nation. GU is the lead institution for the Georgetown-Howard Universities Center for Clinical and Translational Science (GHUCCTS), a multi-institutional partnership of medical research institutions forged from a desire to promote clinical research and translational science. Through multiple collaborations among these institutions, GHUCCTS is transforming clinical research and translational science in order to bring new scientific advances to health care. Oak Ridge National Laboratory is the Department of Energy's (DOE) largest science and energy laboratory. Managed since April 2000 by a partnership of the University of Tennessee and Battelle, ORNL was established in 1943 as a part of the secret Manhattan Project to pioneer a method for producing and separating plutonium. During the 1950s and 1960s, ORNL became an international center for the study of nuclear energy and related research in the physical and life sciences. With the creation of DOE in the 1970s, ORNL's mission broadened to include a variety of energy technologies and strategies. Today the laboratory supports the nation with a peacetime science and technology mission that is just as important as, but very different from, its role during the Manhattan Project. ORNL is home to the world's premier center for high performance supercomputing to enable scientific discovery. ORNL has extensive expertise in various areas of computer science that are uniquely situated to support GU. Additionally, ORNL’s leading computational user facilities present a unique opportunity to leverage the largest scale machines for open science in support of the stated mission of the NCATS CTSA. ORNL's partnership with GU will offer unparalleled opportunity in data analytics, deep-learning, artificial intelligence, and urban dynamics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Sensitive Resources Assessment and Forest Analysis for the Proposed Versatile Test Reactor, Oak Ridge, Tennessee

The US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) is a leading institution in advanced materials, supercomputing, neutrons, and nuclear science. As a research laboratory managed by UT-Battelle, LLC for DOE, ORNL has national priorities in energy, security, and scientific discovery that necessitate facility improvements and expansions. DOE is also committed to environmental stewardship. The laboratory is located on the ~32,000-acre (~13,000-ha) Oak Ridge Reservation (ORR), much of which is categorized as a National Environmental Research Park (NERP) and a state Wildlife Management Area. DOE works with the Tennessee Wildlife Resources Agency (TWRA), Tennessee Department of Environment and Conservation (TDEC), US Fish and Wildlife Service (USFWS), US Department of Agriculture, and other agencies to serve as an effective steward of the ORR. Accordingly, project managers must conform to environmental regulations, agreements, and policies at the federal, state, and institutional levels. Per 40 CFR (Code of Federal Regulations) 1508.14, potential effects on research and science education also represent potential effects of federal actions on the NERP, and impacts on, e.g., deer harvest, must be considered on the Oak Ridge Wildlife Management Area when other aspects of the human environment are affected. The United States currently has no fast neutron testing capability to support advanced nuclear research and development. The proposed Versatile Test Reactor (VTR) will take advantage of current investments by the US government and private industry in nuclear reactors to expedite the design and construction process, using proven technology to create a world-class scientific infrastructure. The VTR will take advantage of fast neutrons provided by this proven technology, along with a capability to rapidly insert, conduct, and remove state-of-the-art experiments. An advantage of the VTR is that it can support future innovations in experimental capabilities without modifying the facility. The VTR will support progress in a variety of science and technology areas, including testing and qualification of advanced reactor fuels; testing and qualification of innovative structural materials; testing of innovative components and instruments; validation of advanced modeling and simulation tools; and versatility for future technical missions. Through proven technology, the VTR can take advantage of existing reactor designs and operating experience to reduce the risk, cost, and time for design and construction. The top available resources of DOE laboratories, industry, and universities will be used to expedite reactor design and construction toward developing the scientific infrastructure that affords a strong testing capability that can be sustained over many years. This report summarizes current knowledge of natural and cultural resources primarily within the VTR construction area. At the time of this report, the proposed VTR site design includes a construction area of ~150.4 acres (~69.9 ha), which contains an ~51.3-acre (~20.8-ha) operations area, located within forested natural areas of the ORR. The primary goal of the work presented here was to evaluate potential effects on sensitive resources that might result from development and construction activities associated with VTR. In addition to on-the-ground surveys during spring and summer 2020 by the ORNL Natural Resources Management Program and Aquatic Ecology Group staff, this report makes use of historical (pre-1995) and contemporary (1995 to present) data from additional confirmed sources (e.g., TDEC). Likewise, forest conditions were compiled from a 2011 forest inventory and supplemented with limited ground observations in 2020. The individuals who obtained and compiled the data presented here are familiar with and routinely assess sensitive resources on the ORR. Anyone who references this report must consider that the timing of surveys did not permit a complete delineation of the resources that will be affected. If the VTR project proceeds, additional surveys will be required to account for the seasonal patterns of various threatened and endangered species. Data deficiencies and potential resources that likely went undetected are indicated where possible. Accordingly, this report should facilitate more environmentally sound decisions during planning and 2 development of the VTR site, provide a foundation for further assessment of sensitive and cultural resources, and help project managers better address regulatory guidance and DOE policies on sustainable development in compliance with, for example, the US Endangered Species Act (ESA), Migratory Bird Treaty Act (MBTA), Tennessee Rare Plant Protection and Conservation Act of 1985, Tennessee Nongame and Endangered or Threatened Wildlife Species Conservation Act of 1974, several federal and state regulations regarding aquatic resource protection, and site-specific policies as outlined in various ORR management plans developed by ORNL and TWRA for DOE.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Two new SciDAC institutes promote mathematical tools and software technology for high-performance computing

Bigger is often said to be better, and the newest extreme-scale computers certainly are bigger, with millions of processing units. Moreover, the breadth of science performed on the U.S. Department of Energy (DOE) computing facilities is expanding, with new technology such as artificial intelligence emerging. These advances are exciting, creating new opportunities for scientific discovery; however, they also raise new questions for scientists who want to exploit these advances for tackling more complex problems. Will my simulation code be able to utilize the accelerators in extreme-scale computing systems? Can I take advantage of the deepening memory hierarchy in heterogeneous processors? Is there a way around bottlenecks caused by the widening ratio of peak floating-point operations per second to I/0 bandwidth? How can I manage my huge amounts of data effectively? Can I analyze data in situ, or must I transfer it to offline storage for later analysis? To address such questions, DOE announced that it is providing $57.5 million over the next five years for two multidisciplinary teams — FASTMath and RAPIDS2 — to develop new tools and techniques to harness supercomputers for scientific discovery. The teams, called SciDAC Institutes, are part of the Scientific Discovery through Advanced Computing program.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The Hardware of Smaller Clusters (V.3.0)

Chris Saunders and three technologists are in high demand from Sandia’s deep learning teams, and they’re kept busy by building new clusters of computer nodes for researchers who need the power of supercomputing on a smaller scale. Sandia researchers working on Laboratory Directed Research & Development (LDRD) projects, or innovative ideas for solutions on short timeframes, formulate new ideas on old themes and frequently rely on smaller cluster machines to help solve problems before introducing their code to larger HPC resources. These research teams need an agile hardware and software environment where nascent ideas can be tested and cultivated on a smaller scale.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Initial full core SMR simulations with NekRS

This document describes the completion of a recent milestone of the ExaSMR program concerning "Full core simulations". The ExaSMR project is developing tools for coupling Monte Carlo (MC) radiation transport solvers to a computational fluid dynamics (CFD) solver. Work is based on the Shift MC, OpenMC MC, and Nek5000/NekRS CFD codes. As part of this milestone, a novel set of pin-resolved CFD full-core simulations have been performed for the first time. These simulations represent a significant increase in capability in what is now possible with CFD on pre-Exascale systems. The simulations have been performed with the spectral element solvers NekRS on the supercomputer Summit. Multiple simulations campaigns have been conducted: (1) LES simulations in bare bundle, (2) RANS simulations in bare bundles, (3) RANS simulations with momentum sources and (4) Conjugate heat transfer calculations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

SST-GPU: A Scalable SST GPU Component for Performance Modeling and Profiling

Programmable accelerators have become commonplace in modern computing systems. Advances in programming models and the availability of unprecedented amounts of data have created a space for massively parallel accelerators capable of maintaining context for thousands of concurrent threads resident on-chip. These threads are grouped and interleaved on a cycle-by-cycle basis among several massively parallel computing cores. One path for the design of future supercomputers relies on an ability to model the performance of these massively parallel cores at scale. The SST framework has been proven to scale up to run simulations containing tens of thousands of nodes. A previous report described the initial integration of the open-source, execution-driven GPU simulator, GPGPU-Sim, into the SST framework. This report discusses the results of the integration and how to use the new GPU component in SST. It also provides examples of what it can be used to analyze and a correlation study showing how closely the execution matches that of a Nvidia V100 GPU when running kernels and mini-apps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

IC reports (2021)

We requested HPC support to continue research on the seismic waves generated by impacts. We used the following codes: (1) the Hybrid Optimization Software Suite (HOSS), developed at LANL. HOSS is based on a combined Finite and Discrete Element Method (FDEM). New material models are developed for the sedimentary rocks. HOSS has been recently benchmarked to iSale and FLAG codes. SPECFEM3D is an open-source code developed since the last 90s. It won the Gordon Bell award for best performance in 2003, was finalist again in 2008 for a run at 0.16 petaflops on 149,784 cores on the ‘Jaguar’ Cray system at Oak Ridge National Laboratory. It also won the BULL Joseph Fourier supercomputing award in 2010.; SW4 is a 4th-order finite difference code developed at LLNL which is currently actively developed to handle complex 3D models and to be ported on future exascale platforms. We assessed our need to a total of 3.9M CPU-hrs for year 1 and and 3.1 M for year 2.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Validation of energetic particle transport models for time-dependent integrated simulations of burning plasmas (Final ST Report)

The major goal of this project is to deliver a suite of validated energetic particle (EP) modeling tools for time-dependent integrated tokamak simulations (with the code TRANSP) that can be used for modeling plasma discharges featuring a substantial EP content, such as burning plasmas. During FY20, issues were resolved with an energetic particle transport model called TGLF-EP, with improvements made to the TGLF-EP workflow, generation of input files, and interpretation of runs on the NERSC supercomputing cluster. The recipient also gave input on simulations with the code TRANSP to reproduce the reference JET discharge, and assessed the accuracy and uncertainties of the simulated B field, current evolution, thermal plasma profiles, stored energy, etc. Input was also given on interpretation of fast-ion instability calculations and recommendations for modifications to JET afterglow discharge for the upcoming DT campaign.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Milestone M6 Report: Reducing Excess Data Movement Part 1

This is the second in a sequence of three Hardware Evaluation milestones that provide insight into the following questions: What are the sources of excess data movement across all levels of the memory hierarchy, going out to the network fabric? What can be done at various levels of the hardware/software hierarchy to reduce excess data movement? How does reduced data movement track application performance? The results of this study can be used to suggest where the DOE supercomputing facilities, working with their hardware vendors, can optimize aspects of the system to reduce excess data movement. Quantitative analysis will also benefit systems software and applications to optimize caching and data layout strategies. Another potential avenue is to answer cost-benefit questions, such as those involving memory capacity versus latency and bandwidth. This milestone focuses on techniques to reduce data movement, quantitatively evaluates the efficacy of the techniques in accomplishing that goal, and measures how performance tracks data movement reduction. We study a small collection of benchmarks and proxy mini-apps that run on pre-exascale GPUs and on the Accelsim GPU simulator. Our approach has two thrusts: to measure advanced data movement reduction directives and techniques on the newest available GPUs, and to evaluate our benchmark set on simulated GPUs configured with architectural refinements to reduce data movement.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

FY 2021 Mid-Year HPC Utilization Report M3UF-21IN0701018

Idaho National Laboratory (INL), supported by the Department of Energy Office of Nuclear Energy (DOE-NE) through the Nuclear Science User Facilities, provides access to supercomputer systems and data storage along with support staff for system management, software installation, cybersecurity and user support to the broader DOE-NE user community. Users include individuals at universities, industry, and government laboratories enabling a wide range of research and development and mission-supporting activities. The availability of high-performance computing (HPC) capabilities is a key foundation of collaboration and innovation in nuclear energy systems research. HPC resources and INL directly support the mission and objectives of DOE-NE. From October 2020 through March 2021, INL HPC capabilities were utilized by a diverse set of computing and applied researchers, for a wide range of research and engineering activities. This report focuses on current INL HPC systems and utilization from October 1, 2020 through March 31, 2021.

99 GENERAL AND MISCELLANEOUS↗