SEARCH · Engineering Papers

Results for “Distributed Memory Programming Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2022

GROWdb US River Systems - Samples

GROW Overview We developed the Genome Resolved Open Watersheds database (GROWdb), which aims to increase genomic sampling and understanding of global river microbiomes. An emphasis of GROWdb is to create a publicly available and ever-expanding microbial genome database that is focused on rivers while being interoperable with databases from other ecosystems. GROWdb is based on a network-of-networks approach to move beyond a small collection of well-studied rivers, towards a spatially distributed, global network of systematic observations. GROWdb represents the first microbial, river-focused resource parsed at various scales from genes to MAGs to community level including expression and potential based measurements that will be of interest to microbiologists, ecologists, geochemists, hydrologists, and modelers. Dataset Acknowledgement GROWdb contains data from various research campaigns, please acknowledge the following data generators, as appropriate: WHONDRS derived genomes or samples - include this statement in your acknowledgements: “This study used data from the Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) under the River Corridor Science Focus Area (SFA) at the Pacific Northwest National Laboratory (PNNL) that was generated at the U.S. Department of Energy (DOE) Joint Genome Institute User Facility. PNNL is operated by Battelle Memorial Institute for the U.S. DOE under Contract No. DE-AC05-76RL01830. The SFA is supported by the U.S. DOE, Office of Biological and Environmental Research (BER), Environmental System Science (ESS) Program.” Total Samples loaded onto this Narrative: 178 Note: Not all GROW samples may be loaded into KBase Data Availability The data underlying GROWdb are accessible across various platforms to ensure all levels of data structure are widely available. First, all reads and MAGs are publicly hosted on National Center for Biotechnology (NCBI) under Bioproject PRJNA946291. Second, all data related data presented here including MAG annotations, extended data tables, phylogenetic tree files, antibiotic resistance gene database files, and MAG abundance tables are available in Zenodo (link). Beyond the flat database files listed above, our aim for GROWdb was to maximize data use by making the data available in searchable and interactive platforms including the National Microbiome Data Collaborative (NMDC) data portal, the Department of Energy’s Systems Biology Knowledgebase (KBase), and a GROW specific user interface released here, GROWdb Explorer. Each platform provides different ways to interact with GROWdb: NMDC GROWdb formed a pilot project for the NMDC. Specifically, individual GROWdb datasets (metagenomes, metatranscriptomes, etc) are easily accessible and searchable through the NMDC data portal, where they are systematically connected to each other and to a rich suite of sample information and standard analysis results, following Findable, Accessible, Interoperable, and Reusable (FAIR) data practices. KBase GROWdb is publicly available within KBase, including samples (this Narrative), MAGs, and corresponding genome scale metabolic models. Access within KBase allows for immediate access and reuse of data, including comparison to private data using KBase’s 500+ analysis tools. Other linked narratives in KBase: GROW Metagenome Assembled Genomes (MAGs) GROW Metabolic Models GROWdb Explorer GROWdb data is also explorable through a graphical user interface built through the Colorado State University Geospatial Centroid (https://geocentroid.shinyapps.io/GROWdatabase/), allowing users to search and graph microbial and spatial data simultaneously. In summary, this microbial genome resource represents the first publicly available genome collection from rivers and offers data that can be leveraged across microbiome studies. GROWdb is an expanding repository to incorporate and unify global river multi-omic data for the future.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2018

OpenSHMEM Sets and Groups: An Approach to Worksharing and Memory Management

Collective operations in the OpenSHMEM programming model are defined over an active set, which is a grouping of Processing Elements (PEs) based on a triple of information including the starting PE, a log2 stride, and the size of the active set. In addition to the active set, collectives require Users to allocate and initialize synchronization (i.e., pSync) and scratchpad (i.e., pWrk) buffers for use by the collective operations. While active sets and the user-defined buffers were previously useful based on hardware and algorithmic considerations, future systems and applications require us to re-evaluate these concepts. In this paper, we propose Sets and Groups as abstractions to create persistent, flexible groupings of PEs (i.e., Sets) and couple these groups of PEs with memory spaces (i.e., Groups), which remove the allocation and initialization burden from the User. To evaluate Sets and Groups, we perform multiple micro-benchmarks to determine the overhead of these abstractions and demonstrate their utility by implementing a distributed All-Pairs Shortest Path (APSP) application, which we evaluate using multiple synthetic and real-world graphs.

Aderholdt, William↗

DOE OSTI2016

Fault Tolerant Frequent Pattern Mining

FP-Growth algorithm is a Frequent Pattern Mining (FPM) algorithm that has been extensively used to study correlations and patterns in large scale datasets. While several researchers have designed distributed memory FP-Growth algorithms, it is pivotal to consider fault tolerant FP-Growth, which can address the increasing fault rates in large scale systems. In this work, we propose a novel parallel, algorithm-level fault-tolerant FP-Growth algorithm. We leverage algorithmic properties and MPI advanced features to guarantee an O(1) space complexity, achieved by using the dataset memory space itself for checkpointing. We also propose a recovery algorithm that can use in-memory and disk-based checkpointing, though in many cases the recovery can be completed without any disk access, and incurring no memory overhead for checkpointing. We evaluate our FT algorithm on a large scale InfiniBand cluster with several large datasets using up to 2K cores. Our evaluation demonstrates excellent efficiency for checkpointing and recovery in comparison to the disk-based approach. We have also observed 20x average speed-up in comparison to Spark, establishing that a well designed algorithm can easily outperform a solution based on a general fault-tolerant programming model.

Shohdy, Sameh↗

DOE OSTI2016

Application Characterization at Scale: Lessons learned from developing a distributed Open Community Runtime system for High Performance Computing

Since 2012, the U.S. Department of Energy’s X-Stack program has been developing solutions including runtime systems, programming models, languages, compilers, and tools for the Exascale system software to address crucial performance and power requirements. Fine grain programming models and runtime systems show a great potential to efficiently utilize the underlying hardware. Thus, they are essential to many X-Stack efforts. An abundant amount of small tasks can better utilize the vast parallelism available on current and future machines. Moreover, finer tasks can recover faster and adapt better, due to a decrease in state and control. Nevertheless, current applications have been written to exploit old paradigms (such as Communicating Sequential Processor and Bulk Synchronous Parallel processing). To fully utilize the advantages of these new systems, applications need to be adapted to these new paradigms. As part of the applications’ porting process, in-depth characterization studies, focused on both application characteristics and runtime features, need to take place to fully understand the application performance bottlenecks and how to resolve them. This paper presents a characterization study for a novel high performance runtime system, called the Open Community Runtime, using key HPC kernels as its vehicle. This study has the following contributions: one of the first high performance, fine grain, distributed memory runtime system implementing the OCR standard (version 0.99a); and a characterization study of key HPC kernels in terms of runtime primitives running on both intra and inter node environments. Running on a general purpose cluster, we have found up to 1635x relative speed-up for a parallel tiled Cholesky Kernels on 128 nodes with 16 cores each and a 1864x relative speed-up for a parallel tiled Smith-Waterman kernel on 128 nodes with 30 cores.

Application characterization↗

DOE OSTI2015

Evolution of CMS Workload Management Towards Multicore Job Support

The successful exploitation of multicore processor architectures is a key element of the LHC distributed computing system in the coming era of the LHC Run 2. High-pileup complex-collision events represent a challenge for the traditional sequential programming in terms of memory and processing time budget. The CMS data production and processing framework is introducing the parallel execution of the reconstruction and simulation algorithms to overcome these limitations. CMS plans to execute multicore jobs while still supporting singlecore processing for other tasks difficult to parallelize, such as user analysis. The CMS strategy for job management thus aims at integrating single and multicore job scheduling across the Grid. This is accomplished by employing multicore pilots with internal dynamic partitioning of the allocated resources, capable of running payloads of various core counts simultaneously. An extensive test programme has been conducted to enable multicore scheduling with the various local batch systems available at CMS sites, with the focus on the Tier-0 and Tier-1s, responsible during 2015 of the prompt data reconstruction. Scale tests have been run to analyse the performance of this scheduling strategy and ensure an efficient use of the distributed resources. This paper presents the evolution of the CMS job management and resource provisioning systems in order to support this hybrid scheduling model, as well as its deployment and performance tests, which will enable CMS to transition to a multicore production model for the second LHC run.

Perez-Calero Yzquierdo, A.↗

DOE OSTI2024

High-Fidelity Multiphysics Modeling of a Heat Pipe Microreactor Using BlueCrab

Researchers who are actively developing nuclear microreactors are planning to employ innovative designs and features using traditional commercial modeling tools that may be inadequate for their design and licensing activities. The codes developed under the U.S. Department of Energy Office of Nuclear Energy Advanced Modeling and Simulation (NEAMS) program provide flexibility in terms of geometry modeling and multiphysics coupling and are particularly well suited for modeling novel microreactor concepts. To test the maturity of these codes, this paper introduces a conceptual heat pipe microreactor (HP-MR) designed to gather various technologies of interest to microreactor developers such as control drums, heat pipes, and hydride moderators. Here, the objective of this effort is to demonstrate NEAMS tools capability to perform high-fidelity multiphysics simulations, using coupled neutronics (via the Griffin code), heat conduction (via the BISON code), heat pipe modeling (via the Sockeye code), and hydrogen redistribution in hydride metal moderator (via the SWIFT code). Codes are coupled in-memory through the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, which permits flexible multiphysics data transfer schemes. The analysis confirmed two key aspects of the HP-MR concept: (1) its ability to follow the power load requested from the heat pipe and (2) its ability to avoid heat pipe cascading failure unless designed with high power close to operating failure limits of its heat pipes. The developed computational model was distributed publicly on the Virtual Test Bed for training purposes to accelerate adoption by industry and to provide a high-fidelity multiphysics solution for benchmarking against other tools. Additional multiphysics analyses including other transients and coupled physics were identified as necessary future work, together with a focus on validating multiphysics behavior against experiments.

Microreactor↗

DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Using Parameter Sweep in WaterTAP to Analyze New Water Treatment Technologies

We describe a powerful and generalized parameter sweep tool in this report that was originally developed to analyze the performance of existing and novel water treatment models being developed in WaterTAP. Since WaterTAP is built upon IDAES and Pyomo, the parameter sweep tool can be used to systematically explore and debug the behavior of most Pyomo and IDAES numerical models. In order to enable meaningful analyses, the parameter sweep tool has been designed with the following features: 1) Model flexibility: The parameter sweep tool does not enforce any restrictions on the types of models that can be used with it. As long as a Pyomo model can be solved and the parameter is active and mutable, the tool only needs functions that describe how to run the model, the sweep parameters, and the output quantities of interest. 2) Flexible sampling: The parameter sweep tool has inbuilt functions to generate samples from a random distribution or a multidimensional Euclidean space. Furthermore, the users have to ability to supply samples generated from a tool of their choice. 3) Multiple sweep types: A user can choose from one of 3 types of parameter sweeps depending on their needs. 4) Detailed outputs: Outputs generated by the parameter sweep tool can be stored in detailed H5 file or user-friendly CSV files for post processing. 5) Parallel computing: The parameter sweep supports shared and distributed memory parallel computing to enable the use of high performance computers (HPC) for large-scale analyses. 6) Modular: The parameter sweep tool is self-contained and can easily be integrated within an outer-loop analysis or as desired by the user. 7) Ease of use: The tool is well documented and a simple sweep can be easily executed by following the online documentation in a few lines of code. We demonstrate the use of the parameter sweep tool on a simple water treatment system from the WaterTAP repository and show its parallel scaling performance on an Apple laptop and NREL's Eagle HPC. The parameter sweep tool is actively being used with models currently being developed within WaterTAP and we expect its use to grow beyond it to other IDAES and Pyomo models.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Scalable parallel distance field construction for large-scale applications

Computing distance fields is fundamental to many scientific and engineering applications. Distance fields can be used to direct analysis and reduce data. In this paper, we present a highly scalable method for computing 3D distance fields on massively parallel distributed-memory machines. Anew distributed spatial data structure, named parallel distance tree, is introduced to manage the level sets of data and facilitate surface tracking overtime, resulting in significantly reduced computation and communication costs for calculating the distance to the surface of interest from any spatial locations. Our method supports several data types and distance metrics from real-world applications. We demonstrate its efficiency and scalability on state-of-the-art supercomputers using both large-scale volume datasets and surface models. We also demonstrate in-situ distance field computation on dynamic turbulent flame surfaces for a petascale combustion simulation. In conclusion, our work greatly extends the usability of distance fields for demanding applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization

Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a new, high-performance parallel computational framework for a broad class of NMF algorithms that iteratively solves alternating non-negative least squares (NLS) subproblems for W and H. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). The framework is flexible and able to leverage a variety of NMF and NLS algorithms, including Multiplicative Update, Hierarchical Alternating Least Squares, and Block Principal Pivoting. Our implementation allows us to benchmark and compare different algorithms on massive dense and sparse data matrices of size that spans from few hundreds of millions to billions. We demonstrate the scalability of our algorithm and compare it with baseline implementations, showing significant performance improvements. The code and the datasets used for conducting the experiments are available online.

2D↗

DOE OSTI2023

Multiphysics Simulations of MSRE with NEAMS Thermal Hydraulics Tools

This report documents the benchmarks being developed and simulations performed using tools and codes developed under the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, utilizing MSRE experimental data. In FY23, three main work scopes were investigated under the NEAMS MSR work package at ANL. The first scope investigated the Griffin-SAM coupling model for simulating the pump startup transient experiment of MSRE. The analyses start with a simple model (single-channel, single-lattice), gradually adding more details (multi-channel, full-core) into the model. The results show that the reactivity loss curve is very sensitive to the axial boundary conditions and the radial core discretization. The simple model can predict a similar reactivity trend as that of the more sophisticated model, which is likely due to error cancellation. Accurately modeling the axial boundary condition may further improve the reactivity trend but would require significant efforts to generate the mesh of the MSRE inlet and upper plenum. The core channel radial discretization for the Griffin-SAM coupled model also depends on the flow distribution. Given the complex geometry in the inlet plenum, the flow distribution needed to be calculated from CFD analysis, which was performed using the NekRS code. This analysis employed a MSRE CAD model developed by Copenhagen Atomics. The CAD model was disassembled to keep the inlet plenum region only, which was subsequently cleaned and modified so that the mesh generated is under the memory limit. The results are merged to a few radial regions to show that the flow rate is highest in the central region. This would be useful for future improvement of the Griffin-SAM coupling model of the MSRE core. The last task investigated is tritium transport modeling using the standalone SAM code. This task aimed to initiate the effort to demonstrate and validate the tritium transport model implemented in SAM. The preliminary investigation employed an MSRE model consisting of the primary loop. Three tritium transport pathways were examined including the retention in the graphite, the permeation through the HX tube wall, and the removal from the off-gas system. The results compare well with the MSRE data, but improvements are still needed on the initial conditions (i.e., the present state may not have reached equilibrium), the boundary conditions, the off-gas system modeling, and a better numerical strategy to reach the equilibrium state.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Parallel sorting algorithm classification: is manual instrumentation necessary?

Understanding parallel algorithms is crucial for accelerating scientific simulations on complex, distributed memory, high-performance computers. Modern algorithm classification approaches learn semantics directly from source code to differentiate between algorithms, however, accessing source code is not always possible. We can learn about parallel algorithms from observing their performance, as programs running the same algorithms and using the same hardware should exhibit similar performance characteristics. We present an approach to learn algorithm classes from parallel performance data directly in order to classify algorithms without access to the source code. We extend previous work to enable classifying parallel sorting algorithms using automatic instrumentation instead of requiring manual region annotations in the source code. In this work, we design and demonstrate a study for classification of parallel sorting algorithms using parallel performance data collected from automatic instrumentation, and evaluate the performance of our new methodology on classification. We leverage Caliper to collect the performance data, Thicket for our exploratory data analysis (EDA), and PyTorch and Scikit-learn to evaluate the effectiveness of random forests, support vector machines (SVMs), decision trees, neural networks, and logistic regressions on parallel performance data. Additionally, we study noise in parallel performance data, whether the removal of noise and pre-processing of the data is necessary to accurately classify parallel sorting algorithms, and determine the effectiveness of features created from performance data. In conclusion, we demonstrate classification accuracy for these five different models of up to 97.7% across four different parallel algorithm classes.

Algorithm Classification↗

DOE OSTI2024

Embedding Neural Thermal Scattering (NeTS) Modules in SERPENT for Higher Fidelity Advanced Reactor Analysis

When a neutron born in fission thermalizes to the order of $k$ $B$ $T$, it’s de-Broglie wavelength and energy approach the order of inter-atomic spacing and elementary lattice oscillations, respectively. $S$($a,β,t$) or the scattering law, uuantify these temperature-dependent crystallographic contributions to total cross section (or reaction rate). In a Monte Carlo analysis, cumulative distribution functions (CDFs) of $S$($a,β,t$) are loaded to memory from “A Compact ENDF” (ACE) files for stochastically selecting thermal scattered neutron trajectories. In this work, novel neural thermal scattering (NeTS) modules for $S$($a,β,t$) CDFs are designed, trained, serialized and embedded within SERPENT using Python’s limited C-API for on-the-fly deployment of crystalline graphite $S$($a,β,t$) sampling. Torchscript tracing and Numba just-in-time (JIT) compilation streamline neural inference on NVIDIA GPUs with CUDA libraries. Demonstrations of bare sphere thermalization of fast and thermal sources show excellent agreement between embedded NeTS in SERPENT and MCNP. With an explicit model of the reactor, NeTS can predict on-the-fly changes in TREAT neutron spectra as a function of local temperature, which can serve to improve transient and accident predictions in a multiphysics analysis framework. This framework can be further extended to account on-the-fly for changes in local graphitic microstructure to scattering cross sections, and outlines a novel coupling of modern machine learning with state-of-the-art reactor physics methods.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Latent uncertainties of the precalculated track Monte Carlo method

Purpose: While significant progress has been made in speeding up Monte Carlo (MC) dose calculation methods, they remain too time-consuming for the purpose of inverse planning. To achieve clinically usable calculation speeds, a precalculated Monte Carlo (PMC) algorithm for proton and electron transport was developed to run on graphics processing units (GPUs). The algorithm utilizes pregenerated particle track data from conventional MC codes for different materials such as water, bone, and lung to produce dose distributions in voxelized phantoms. While PMC methods have been described in the past, an explicit quantification of the latent uncertainty arising from the limited number of unique tracks in the pregenerated track bank is missing from the paper. With a proper uncertainty analysis, an optimal number of tracks in the pregenerated track bank can be selected for a desired dose calculation uncertainty. Methods: Particle tracks were pregenerated for electrons and protons using EGSnrc and GEANT4 and saved in a database. The PMC algorithm for track selection, rotation, and transport was implemented on the Compute Unified Device Architecture (CUDA) 4.0 programming framework. PMC dose distributions were calculated in a variety of media and compared to benchmark dose distributions simulated from the corresponding general-purpose MC codes in the same conditions. A latent uncertainty metric was defined and analysis was performed by varying the pregenerated track bank size and the number of simulated primary particle histories and comparing dose values to a “ground truth” benchmark dose distribution calculated to 0.04% average uncertainty in voxels with dose greater than 20% of D{sub max}. Efficiency metrics were calculated against benchmark MC codes on a single CPU core with no variance reduction. Results: Dose distributions generated using PMC and benchmark MC codes were compared and found to be within 2% of each other in voxels with dose values greater than 20% of the maximum dose. In proton calculations, a small (≤1 mm) distance-to-agreement error was observed at the Bragg peak. Latent uncertainty was characterized for electrons and found to follow a Poisson distribution with the number of unique tracks per energy. A track bank of 12 energies and 60000 unique tracks per pregenerated energy in water had a size of 2.4 GB and achieved a latent uncertainty of approximately 1% at an optimal efficiency gain over DOSXYZnrc. Larger track banks produced a lower latent uncertainty at the cost of increased memory consumption. Using an NVIDIA GTX 590, efficiency analysis showed a 807 × efficiency increase over DOSXYZnrc for 16 MeV electrons in water and 508 × for 16 MeV electrons in bone. Conclusions: The PMC method can calculate dose distributions for electrons and protons to a statistical uncertainty of 1% with a large efficiency gain over conventional MC codes. Before performing clinical dose calculations, models to calculate dose contributions from uncharged particles must be implemented. Following the successful implementation of these models, the PMC method will be evaluated as a candidate for inverse planning of modulated electron radiation therapy and scanned proton beams.

07 ISOTOPE AND RADIATION SOURCES↗

DOE OSTI2023

Distributed out-of-memory NMF on CPU/GPU architectures

We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Blackcomb: Hardware-Software Co-design for Non-Volatile Memory in Exascale Systems

Summary of technical results of Blackcomb Memory Devices We explored various different memory technologies (STTRAM, PCRAM, FeRAM, and ReRAM). The progress can be classified into three categories, below. Modeling and Tool Releases Various modeling tools have been developed over the last decade to help in the design of SRAM or DRAM-based memory hierarchies. To explore new design opportunities that NVM technologies can bring to the designers, we have developed similar high-level models for NVM, including PCRAMsim [Dong 2009], NVSim [Dong 2012], and NVMain [Poremba 2012]. NVSim is a circuit-level model for NVM performance, energy, and area estimation, which supports various NVM technologies, including STT-RAM, PCRAM, ReRAM, and legacy NAND Flash. NVSim is successfully validated against industrial NVM prototypes, and it is expected to help boost architecture-level NVM-related studies. On the other side, NVMain is a cycle accurate main memory simulator designed to simulate emerging nonvolatile memories at the architectural level. We have released these models as open source tools and provided contiguous support to them. We also proposed PS3-RAM, which is a fast, portable and scalable statistical STT-RAM reliability analysis model [Wen 2012]. Design Space Exploration and Optimization With the support of these models, we explore different device/circuit optimization techniques. For example, in [Niu 2012a] we studied the power reduction technique for the application of ECC scheme in ReRAM designs and proposed to use ECC code to relax the BER (Bit Error Rate) requirement of a single memory to improve the write energy consumption and latency for both 1T1R and cross-point ReRAM designs. In [Xu 2011], we proposed a methodology to design STT-RAM for different optimization goals such as read performance, write performance and write energy by leveraging the trade-off between write current and write time of MTJ. We also studied the tradeoffs in building a reliable crosspoint ReRAM array [Niu 2012b]. We have conducted an in depth analysis of the circuit and system level design implications of multi-layer cross-point Resistive RAM (MLCReRAM) from performance, power and reliability perspectives [Xu 2013]. The objective of this study is to understand the design trade-offs of this technology with respect to the MLC Phase Change Memory (MLCPCM).Our MLC ReRAM design at the circuit and system levels indicates that different resistance allocation schemes, programming strategies, peripheral designs, and material selections profoundly affect the area, latency, power, and reliability of MLC ReRAM. Based on this analysis, we conduct two case studies: first we compare MLC ReRAM design against MLC phase-change memory (PCM) and multi-layer cross-point ReRAM design, and point out why multi-level ReRAM is appealing; second we further explore the design space for MLC ReRAM. Architecture and Application We explored hybrid checkpointing using phase-change memory for future exascale systems [Dong 2011] and showed that the use of nonvolatile memory for local checkpointing significantly increases the number of faults covered by local checkpoints and reduces the probability of a global failure in the middle of a global checkpoint to less than 1%. We also proposed a technique called i2WAP to mitigate the write variations in NVM-based last-level cache for the improvement of the NVM lifetime [Wang 2013]. Our wear leveling technique attempts to work around the limitations of write endurance by arranging data access so that write operations can be distributed evenly across all the storage cells. During our intensive research on fault-tolerant NVM design, we found that ECC cannot effectively tolerate hard errors from limited write endurance and process imperfection. Therefore, we devised a novel Point and Discard (PAD) architecture in in [ 2012] as a hard-error-tolerant architecture for ReRAM-based Last Level Caches. PAD improves the lifetime of ReRAM caches by 1.6X-440X under different process variations without performance overhead in the system's early life. We have investigated the applicability of NVM for persistent memory design [Zhao 2013]. New byte addressable NVM enables fast persistent memory that allows in-memory persistent data objects to be updated with much higher throughput. Despite the significant improvement, the performance of these designs is only 50% of the native system with no persistence support, due to the logging or copy-on-write mechanisms used to update the persistent memory. A challenge in this approach is therefore how to efficiently enable atomic, consistent, and durable updates to ensure data persistence that survives application and/or system failures. We have designed a persistent memory system, called Klin, that can provide performance as close as that of the native system. The Klin design adopts a non-volatile cache and a non-volatile main memory for constructing a multi-versioned durable memory system, enabling atomic updates without logging or copy-on-write. Our evaluation shows that the proposed Kiln mechanism can achieve up to 2X of performance improvement to NVRAM-based persistent memory employing write-ahead logging. In addition, our design has numerous practical advantages: a simple and intuitive abstract interface, microarchitecture-level optimizations, fast recovery from failures, and no redundant writes to slow non-volatile storage media. The work was published in MICRO 2013 and received Best Paper Honorable Mentioned Award.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Assessment of the Griffin Reactor Multiphysics Application Using the Empire Micro Reactor Design Concept

In late 2019, INL and ANL agreed to jointly develop the reactor physics code named Griffin based on the integration of the two code suites, MAMMOTH/Rattlesnake (INL) and MC2 - 3/PROTEUS (ANL). Griffin is being developed based on the MOOSE framework and MOOSE quality assurance procedures. This decision was made to be able to allow DOE-NE to efficiently invest funding to this area and to provide effective and timely support for existing and potential users; the latter includes industry and government organizations who are developing various types of advanced reactors in the near and long term. Since MAMMOTH/Rattlesnake has been developed based on the MOOSE framework, the INL/ANL Griffin development team agreed to build Griffin beginning with a merger of MAMMOTH and Rattlesnake into a single code and moving forward by implementing capabilities from the PROTEUS suite into Griffin. Moving forward, both ANL and INL efforts are equally invested in the Griffin project, with management support, to provide an advanced reactor multiphysics tool to assist in reactor design, optimization, and safety analysis. Much work remains in moving Griffin forward to migrate PROTEUS capabilities and to optimize performance to meet user needs. The main objective of this work is to assess the current status of Griffin capabilities in terms of performance and accuracy, to determine priorities for PROTEUS migration, and to identify capabilities and features to improve for supporting the code integration effort. For this assessment, the Empire micro reactor problem that was developed in the ARPA-E MEITNER program was selected as an advance reactor concept of interest to the technical community. The Empire reactor problem was expanded from its original incomplete specification to be a small heat-pipe-cooled micro reactor core with ~113 cm radius and 70 cm in height, composed of 18 fuel assemblies, 12 control drums, and beryllium radial and axial reflectors. In the current model, using 5 cm axial reflectors specified in the original Empire assembly model, more than 10% of neutrons leak axially and through the empty center safety hole, as well as through heat pipe channels in fuel assembly elements that extend through the top reflector region. Several calculation models of the core were defined for systematic assessment, including 2-D and 3-D fuel assemblies and whole cores with cylindrical boundaries. Cross sections were generated using Serpent 2, and meshes were produced using the Argonne mesh tool or the INL neutronics meshing tools combined with CUBIT. Cross sections and meshes were converted to the ISOXML and Exodus formats, respectively, so that Griffin and PROTEUS could use consistent data for solving the reactor problems. With the prepared cross sections and meshes, PROTEUS was run first to ensure that all input data were correctly generated and input options in terms of angle, mesh, and energy group were accurately determined. Comparisons against Serpent 2 solutions were made in terms of eigenvalue and pin power. The same calculations and comparisons were then conducted using Griffin. For the fuel assembly and whole core problems, the PROTEUS eigenvalues agreed well with reference Serpent 2 solutions within 100 and 30 pcm, respectively, and pin power differences relative to Serpent 2 were overall less than 2.2% and RMS 0.8% for the whole core models. This indicated that all input data were properly prepared. Using the same data, Griffin was run selecting the SAAF-CFEM SN solver with Legendre-Gaussian quadrature and NDA and DSA for acceleration. It was found that the SAAF-CFEM solver of Griffin required finer meshes to achieve eigenvalue and pin power solutions in good agreement with Serpent 2, consequently requiring more memory requirement and longer computation time. On the other hand, the SPH-Diffusion 2-D core calculations performed using Griffin were able to recover the exact eigenvalue from the reference Serpent 2 solutions, resulting in a pin-power distribution with an RMS of 0.6% and maximum absolute difference of less than 1.4%. The runtimes for SPH-Diffusion for the 2-D core were less than 3 minutes on 40 cores. During this evolution of this evaluation, many updates were made in Griffin by the Griffin development team of INL (focusing on software updates) and ANL (reviewing and supporting software updates) to complete this assessment. Observations from the code assessment are presented in the conclusion section of this report, followed by a discussion of recommendations for future work.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2026

DASSH-F: Subchannel Based Thermal Analysis

The DASSH thermal analysis code is designed to rapidly allow a reactor design engineer to obtain flow rates requirements that satisfy peak temperature constraints in the domain. The advantage of using DASSH over a hand calculation is that it has a more rigorous treatment of the pin power distribution and coolant heat transfer within an assembly and between assemblies. The advantage of using DASSH over a conventional 3D subchannel code or a computational fluid dynamics code (CFD) is that it can obtain the desired solution in a matter of minutes in serial with minor computer memory needs. The DASSH methodology for pin lattice models is virtually identical to SUPERENERGY-2 with additional functionalities taken from follow on work to SUPERENERGY-2 done at ANL in the 1980s. DASSH today is an integral component of the Argonne Fast Reactor analysis suite for reactor design work. DASSH obtains the power distribution from a coupled neutron-gamma heating calculation in GAMSOR (including DIF3D) at each time point of a companion fuel cycle analysis calculation with REBUS. The domain in DASSH assumes a hexagonal grid typical for fast reactors with much of the geometry information taken from the DIF3D model. DASSH assumes the assemblies that are loaded into each grid position are ducted to control the coolant flow. The user can alternatively provide their own geometry and power profile instead of inheriting it from DIF3D. Considerable detail is given on the subchannel formulation of DASSH in this document. Much of the formulation and design of the code builds upon research done by previous authors with little new investigation. Thus the decisions made in developing the subchannel model used in DASSH have their origins over 50 years ago. Much of the heat transfer methodology in DASSH is built upon correlations for both the coolant mixing and heat transfer coefficients for pins and ducts. DASSH is thus not a rigorous treatment of a given problem, but a rapid assessment of the temperature field that has known limitations with respect to an experimental measurement or CFD calculation. The DASSH input and output are detailed along with usage of the software. The DASSH output provides tables of evaluated material properties and key coolant and pin temperature results. DASSH can create Python scripts that generate domain summary pictures. DASSH can also generate assembly temperature maps and VTK output files which allow the DASSH solution to be visualized. As the primary purpose of the DASSH software is to compute the coolant and fuel pin temperature distribution for a given model of a reactor, much of the output focus is giving the user quick summary tables needed to assess the performance of a given orifice flow specification. The present version of DASSH has a crude orifice search capability and a sufficient orifice flow search capability. The flow search tries to meet user specified constraints for 1) peak 2-sigma clad temperature, 2) peak coolant temperature, and 3) desired bulk outlet temperature. This document serves as the manual for the Fortran based DASSH software that was developed to replace the Python version of DASSH developed as part of the VTR program.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗