Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel computers.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

A Programming Model for Massive Data Parallelism with Data Dependencies

Accelerating processors can often be more cost and energy effective for a wide range of data-parallel computing problems than general-purpose processors. For graphics processor units (GPUs), this is particularly the case when program development is aided by environments such as NVIDIA s Compute Unified Device Architecture (CUDA), which dramatically reduces the gap between domain-specific architectures and general purpose programming. Nonetheless, general-purpose GPU (GPGPU) programming remains subject to several restrictions. Most significantly, the separation of host (CPU) and accelerator (GPU) address spaces requires explicit management of GPU memory resources, especially for massive data parallelism that well exceeds the memory capacity of GPUs. One solution to this problem is to transfer data between the GPU and host memories frequently. In this work, we investigate another approach. We run massively data-parallel applications on GPU clusters. We further propose a programming model for massive data parallelism with data dependencies for this scenario. Experience from micro benchmarks and real-world applications shows that our model provides not only ease of programming but also significant performance gains.

43 PARTICLE ACCELERATORS↗

Improved Transport Processes for CCSM

In atmospheric modeling, global spectral methods have dominated weather and climate simulation for the past two decades. However, global methods based upon the spherical harmonic basis functions require expensive non-local communication and thus have difficulty in exploiting the full potential of current high-performance parallel computers. The primary objective of HOMME ((11)) is the development of a class of high-order scalable conservative atmospheric models for climate and general atmospheric modeling applications. The spatial discretizations are based on continuous Galerkin (spectral element method) and discontinuous Galerkin (DG). These are local methods based on high-order accurate spectral basis functions which have been shown to perform well on massively parallel supercomputers at any resolution (18). HOMME employs a cubed-sphere geometry (26) exhibiting none of the singularities present in conventional latitude-longitude spherical geometries. The element based formulation enables the use of general curvilinear geometries and adaptive conforming or non-conforming meshes.

54 ENVIRONMENTAL SCIENCES↗

Hierarchical Adaptive Solution of Radiation Transport Problems on Unstructured Grids

Computational radiation transport has steadily gained acceptance in the last decade as a viable modeling tool due to the rapid advancements in computer software and hardware technologies. It can be applied for the analysis of a wide range of problems which arise in nuclear reactor physics, medical physics, atmospheric physics, astrophysics and other areas of engineering physics. However, radiation transport is an extremely chanllenging computational problem since the governing equation is seven-deimensional (3 in space, 2 in direction, 1 in energy, and 1 in time) with a high degree of coupleing betwen these variables. If not careful, this relatively large number of independent variables when discretized can potentially lead to sets of linear equations of intractable size. Though parallel computing has allowed the solution of very large problems, avaliable computational resources will always be finite due to the fact that every more sophisticated multiphysics models are being demanded by industry. There is thus the pressing requirement to optimize the discretizations so as to minimize the effort and maximize the accuracy.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Parallel FFT-based Poisson Solver for Isolated Three-dimensional Systems

We describe an implementation to solve Poisson's equation for an isolated system on a unigrid mesh using FFTs. The method solves the equation globally on mesh blocks distributed across multiple processes on a distributed-memory parallel computer. Test results to demonstrate the convergence and scaling properties of the implementation are presented. The solver is offered to interested users as the library PSPFFT.

99 GENERAL AND MISCELLANEOUS↗

GMH: A Message Passing Toolkit for GPU Clusters

Driven by the market demand for high-definition 3D graphics, commodity graphics processing units (GPUs) have evolved into highly parallel, multi-threaded, many-core processors, which are ideal for data parallel computing. Many applications have been ported to run on a single GPU with tremendous speedups using general C-style programming languages such as CUDA. However, large applications require multiple GPUs and demand explicit message passing. This paper presents a message passing toolkit, called GMH (GPU Message Handler), on NVIDIA GPUs. This toolkit utilizes a data-parallel thread group as a way to map multiple GPUs on a single host to an MPI rank, and introduces a notion of virtual GPUs as a way to bind a thread to a GPU automatically. This toolkit provides high performance MPI style point-to-point and collective communication, but more importantly, facilitates event-driven APIs to allow an application to be managed and executed by the toolkit at runtime.

99 GENERAL AND MISCELLANEOUS↗

Accelerating science: The usage of commercial clouds in ATLAS Distributed Computing

The ATLAS experiment at CERN is one of the largest scientific machines built to date and will have ever growing computing needs as the Large Hadron Collider collects an increasingly larger volume of data over the next 20 years. ATLAS is conducting R&D projects on Amazon Web Services and Google Cloud as complementary resources for distributed computing, focusing on some of the key features of commercial clouds: lightweight operation, elasticity and availability of multiple chip architectures. The proof of concept phases have concluded with the cloud-native, vendoragnostic integration with the experiment’s data and workload management frameworks. Google Cloud has been used to evaluate elastic batch computing, ramping up ephemeral clusters of up to O(100k) cores to process tasks requiring quick turnaround. Amazon Web Services has been exploited for the successful physics validation of the Athena simulation software on ARM processors. We have also set up an interactive facility for physics analysis allowing endusers to spin up private, on-demand clusters for parallel computing with up to 4 000 cores, or run GPU enabled notebooks and jobs for machine learning applications. The success of the proof of concept phases has led to the extension of the Google Cloud project, where ATLAS will study the total cost of ownership of a production cloud site during 15 months with 10k cores on average, fully integrated with distributed grid computing resources and continue the R&D projects.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Simulation of the full-core pin-model by JMCT Monte Carlo neutron-photon transport code

Since the large numbers of cells over a million, the tallies over a hundred million and the particle histories over ten billion, the simulation of the full-core pin-by-pin model has become a real challenge for the computers and the computational methods. On the other hand, the basic memory of the model has exceeded the limit of a single CPU, so the spatial domain and data decomposition must be considered. JMCT (J Monte Carlo Transport code) has successful fulfilled the simulation of the full-core pin-by-pin model by the domain decomposition and the nested parallel computation. The k{sub eff} and flux of each cell are obtained. (authors)

97 MATHEMATICS AND COMPUTING↗

Final Scientific/Technical Report

A variety of calcifying organisms produce a transient or metastable amorphous calcium carbonate (ACC) precursor phase that is assembled and subsequently transformed into a crystalline biomineral, typically calcite or aragonite. The complex shapes, hierarchical structures, and unique physical properties of the biominerals that result from this calcification pathway have stimulated interest in adapting these concepts for the design and creation of bio-inspired functional materials in the laboratory. ACC also forms as a reactive precursor in diverse inorganic systems and is likely to play a much broader role in calcium carbonate formation. Knowledge of the structure, composition, and behavior of this metastable phase is critical for establishing a structural and mechanistic framework for calcium carbonate formation and its role in biogeochemical processes, including carbon cycling. Minor additives, such as magnesium, phosphorus, and organic macromolecules, are known to play important roles in controlling ACC stability, transformation kinetics, and selection of final crystalline polymorph. Molecular water also occurs in many types of ACC and is thought to play a structural role in its stability and transformation behavior. One of the major challenges that remain unresolved is identification of the structural basis for the role of these minor additives and molecular water. The absence of long-range order in ACC, and other amorphous phases, has posed a challenge for study by techniques commonly used for crystalline solids. Preliminary studies in our group show that the combination of two techniques, synchrotron X-ray-based pair distribution function (PDF) analysis and nuclear magnetic resonance (NMR) spectroscopy can provide entirely new insight to structural properties of synthetic ACC over length scales that are most relevant for understanding its transformation properties. Building on preliminary experiments, we propose a systematic study of synthesis, structure, and transformation behavior in abiotic systems. The work will specifically address the influence of phosphate as a minor additive. PDF analysis will utilize total X-ray scattering data collected at synchrotron facilities optimized for this method and will provide direct characterization of the short- and intermediate-range structure of ACC synthesized under controlled conditions. Parallel computational work using reverse Monte Carlo methods will allow structural models to be constructed for a more complete analysis of PDF results. NMR spectroscopy, using a variety of single- and double-resonance techniques, will provide information on H and CO3 components, including dynamical properties, and their relationship to stabilizing additives. PDF and NMR results will be complemented by parallel studies using X-ray absorption and FT-IR spectroscopy to allow direct comparison to previous studies. These techniques will be used to follow the transformation of different ACC samples, with specific additives, to crystalline phases under controlled condition (e.g., relative humidity). This work will provide the structural and mechanistic basis for understanding ACC stability, its transformation behavior, and the factors that govern polymorph selection. This new insight will directly benefit researchers in diverse fields, as well as adding to the framework of knowledge for understanding and controlling calcium carbonate formation in natural and engineered systems.

58 GEOSCIENCES↗

Long duration battery sizing, siting, and operation under wildfire risk using progressive hedging

Battery sizing and siting problems are computationally challenging due to the need to make long-term planning decisions that are cognizant of short-term operational decisions. This paper considers sizing, siting, and operating batteries in a power grid to maximize their benefits, including price arbitrage and load shed mitigation, during both normal operations and periods with high wildfire ignition risk. Here we formulate a multi-scenario optimization problem for long duration battery storage while considering the possibility of load shedding during Public Safety Power Shutoff (PSPS) events that de-energize lines to mitigate severe wildfire ignition risk. To enable a computationally scalable solution of this problem with many scenarios of wildfire risk and power injection variability, we develop a customized temporal decomposition method based on a progressive hedging framework. Extending traditional progressive hedging techniques, we consider coupling in both placement variables across all scenarios and state-of-charge variables at temporal boundaries. This enforces consistency across scenarios while enabling parallel computations despite both spatial and temporal coupling. The proposed decomposition facilitates efficient and scalable modeling of a full year of hourly operational decisions to inform the sizing and siting of batteries. With this decomposition, we model a year of hourly operational decisions to inform optimal battery placement for a 240-bus WECC model in under 70 min of wall-clock time.

25 ENERGY STORAGE↗

Cross-scale modeling of storm surge, tide, and inundation in Mid-Atlantic Bight and New York City during Hurricane Sandy, 2012

Driven by high-resolution NAM (North America Mesoscale) and ECMWF (European Centre for Medium-Range Weather Forecasts) atmospheric fields, a 3-D unstructured grid SCHISM (Semi-implicit Cross-scale Hydroscience Integrated System Model) was applied to simulate total water level in the Mid-Atlantic Bight during Hurricane Sandy in 2012. The simulated storm surge, tide, significant wave height, and wave peak period results compared favorably with NOAA observations along US East Coast, Long Island Sound, and New York Harbor. The maximum total water level at The Battery in the New York Harbor was accurately simulated with an absolute error of less than 0.08 m (out of 2.9 m peak surge) and a timing difference less than 10 min, which includes the effects on the order of 0.1–0.3 m (5–10%) from the coastal setup induced by the surface waves. The scenario comparisons of (1) “NAM” versus “ECMWF”, (2) “2-D” versus “3-D”, and (3) “with” versus “without” wind wave model were examined. The 3-D barotropic model forced by ECMWF including the effects of wind waves performs the best, attributed to wave-induced radiation stress and reduction of bottom stress when the 3-D version is used. Simultaneously, the ELCIRC-Sub model, using a 5-m topography/bathymetry sub-grid and a regular 200-m resolution finite volume computational model grid, was developed to simulate the street-level inundation in New York City. The momentum and mass fluxes calculated by the coarser base grid model were effectively coupled with the sub-grid so that running a full-blown high-resolution base model is not required. The ELCIRC-Sub model uses MPI (Message Passing Interface) parallel computing to enlarge the coverage of the land surface and an efficient non-linear solver to improve the accuracy of the wetting-and-drying processes. The temporal comparisons of modeled water level with NOAA's tidal stations and USGS' rapid-deployed gauges showed overall performance with an average error of 0.1 m. Particularly, ELCIRC-Sub captured the profile of the highest peak surge (3.9 m), which rose 2 m within 3 hour at Kings Point, NY. The spatial comparisons of the modeled peak water level at 80 surveyed locations showed an average error less than 0.13 m. The modeled maximum inundation extent also matched well with an 80% hit rate against FEMA's Hurricane Sandy maximum flooding extent.

54 ENVIRONMENTAL SCIENCES↗

Compiler Optimization for Extreme-Scale Scripting

The data-driven task parallelism execution model can support parallel programming models that are well suited for large-scale distributed-memory parallel computing, for example, simulations and analysis pipelines running on clusters and clouds. We describe a novel compiler intermediate representation and optimizations for this execution model, including adaptions of standard techniques alongside novel techniques. These techniques are applied to Swift/T, a high-level scripting language for flexible data flow composition of functions, which may be serial or use lower-level parallel programming models such as MPI and OpenMP. This paper presents preliminary results, indicating that our compiler optimizations reduce communication overhead by 70% to 93% on distributed-memory systems.

Armstrong, Timothy G.↗

A Lyapunov Function Based Remedial Action Screening Tool Using Real-Time Data

This report summarizes the outcome of a research project that comprised the development of a Lyapunov function based remedial action screening tool using real-time data (L-RAS). The L-RAS is an advanced computational tool that is intended to assist system operators in making real-time redispatch decisions to preserve power grid stability. The tool relies on screening contingencies using a homotopy method based on Lyapunov functions to avoid, to the extent possible, the use of time domain simulations. This enables transient stability evaluation at real-time speed without the use of massively parallel computational resources. The project combined the following components. 1. Development of a methodology for contingency screening using a homotopy method based on Lyapunov functions and real-time data. 2. Development of a methodology for recommending remedial actions based on the screening results. 3. Development of a visualization and operator interaction interface. 4. Testing of screening tool, validation of control actions, and demonstration of project outcomes on a representative real system simulated on a Real-Time Digital Simulator (RTDS) cluster. The project was led by Michigan State University (MSU), where the theoretical models including homotopy-based screening, trajectory correction using real-time data, and remedial action were developed and implemented in the form of research-grade software. Los Alamos National Laboratory (LANL) contributed to the development of energy margin sensitivity dynamics, which constituted a part of the remedial action portfolio. Florida State University (FSU) and Southern California Edison (SCE) developed a model of the SCE system that was implemented on FSU's RTDS cluster to simulate real-time data that was streamed over the internet to MSU where the L-RAS tool was executed and remedial actions were communicated back to FSU to execute stabilizing controls on the simulated system. LCG Consulting developed the visualization and operator interaction interface, based on specifications provided by MSU. The project was performed from October 2012 to December 2016, at the end of which the L-RAS tool, as described above, was completed and demonstrated. The project resulted in the following innovations and contributions: (a) the L-RAS software prototype, tested on a simulated system, vetted by utility personnel, and potentially ready for wider testing and commercialization; (b) an RTDS-based test bed that can be used for future research in the field; (c) a suite of breakthrough theoretical contributions to the field of power system stability and control; and (d) a new tool for visualization of power system stability margins. While detailed descriptions of the development and implementation of the various project components have been provided in the quarterly reports, this final report provides an overview of the complete project, and is demonstrated using public domain test systems commonly used in the literature. The SCE system, and demonstrations thereon, are not included in this report due to Critical Energy Infrastructure Information (CEII) restrictions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Method to Alleviate the Long History Problem Encountered in Monte Carlo Simulations via Weight Window Variance Reduction

When the weight window is implemented in large and complex models, streaming path and deep penetration could lead to a long history problem that takes a disproportionate amount of time for the accomplishment of Monte Carlo simulations and exerts a detrimental effect on the efficiency of parallel computing. An approach set the limitation of weight window splitting was proposed for the alleviation of long history problem. Tests were conducted on a streaming dog-leg geometry model and a 3D model of the Chinese Fusion Engineering Testing Reactor. The results show that a suitable parameter in the alleviation method could optimize both the performance of variance reduction and the efficiency of parallel calculation, which makes long history problem tractable.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

RELAP-7 Level 2 Milestone Report: Demonstration of a Steady State Single Phase PWR Simulation with RELAP-7

The document contains the simulation results of a steady state model PWR problem with the RELAP-7 code. The RELAP-7 code is the next generation nuclear reactor system safety analysis code being developed at Idaho National Laboratory (INL). The code is based on INL's modern scientific software development framework - MOOSE (Multi-Physics Object-Oriented Simulation Environment). This report summarizes the initial results of simulating a model steady-state single phase PWR problem using the current version of the RELAP-7 code. The major purpose of this demonstration simulation is to show that RELAP-7 code can be rapidly developed to simulate single-phase reactor problems. RELAP-7 is a new project started on October 1 st , 2011. It will become the main reactor systems simulation toolkit for RISMC (Risk Informed Safety Margin Characterization) and the next generation tool in the RELAP reactor safety/systems analysis application series (the replacement for RELAP5). The key to the success of RELAP-7 is the simultaneous advancement of physical models, numerical methods, and software design while maintaining a solid user perspective. Physical models include both PDEs (Partial Differential Equations) and ODEs (Ordinary Differential Equations) and experimental based closure models. RELAP-7 will eventually utilize well posed governing equations for multiphase flow, which can be strictly verified. Closure models used in RELAP5 and newly developed models will be reviewed and selected to reflect the progress made during the past three decades. RELAP-7 uses modern numerical methods, which allow implicit time integration, higher order schemes in both time and space, and strongly coupled multi-physics simulations. RELAP-7 is written with object oriented programming language C++. Its development follows modern software design paradigms. The code is easy to read, develop, maintain, and couple with other codes. Most importantly, the modern software design allows the RELAP-7 code to evolve with time. RELAP-7 is a MOOSE-based application. MOOSE (Multiphysics Object-Oriented Simulation Environment) is a framework for solving computational engineering problems in a well-planned, managed, and coordinated way. By leveraging millions of lines of open source software packages, such as PETSC (a nonlinear solver developed at Argonne National Laboratory) and LibMesh (a Finite Element Analysis package developed at University of Texas), MOOSE significantly reduces the expense and time required to develop new applications. Numerical integration methods and mesh management for parallel computation are provided by MOOSE. Therefore RELAP-7 code developers only need to focus on physics and user experiences. By using the MOOSE development environment, RELAP-7 code is developed by following the same modern software design paradigms used for other MOOSE development efforts. There are currently over 20 different MOOSE based applications ranging from 3-D transient neutron transport, detailed 3-D transient fuel performance analysis, to long-term material aging. Multi-physics and multiple dimensional analyses capabilities can be obtained by coupling RELAP-7 and other MOOSE based applications and by leveraging with capabilities developed by other DOE programs. This allows restricting the focus of RELAP-7 to systems analysis-type simulations and gives priority to retain and significantly extend RELAP5's capabilities.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

National Computational Infrastructure for Lattice Gauge Theory SciDAC-2 Closeout Report

As part of this project work, researchers from Vanderbilt University, Fermi National Laboratory and Illinois Institute of technology developed a real-time cluster fault-tolerant cluster monitoring framework. This framework is open source and is available for download upon request. This work has also been used at Fermi Laboratory, Vanderbilt University and Mississippi State University across projects other than LQCD. The goal for the scientific workflow project is to investigate and develop domain-specific workflow tools for LQCD to help effectively orchestrate, in parallel, computational campaigns consisting of many loosely-coupled batch processing jobs. Major requirements for an LQCD workflow system include: a system to manage input metadata, e.g. physics parameters such as masses, a system to manage and permit the reuse of templates describing workflows, a system to capture data provenance information, a systems to manage produced data, a means of monitoring workflow progress and status, a means of resuming or extending a stopped workflow, fault tolerance features to enhance the reliability of running workflows. Requirements for an LQCD workflow system are available in documentation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Progress report for FACETS (Framework Application for Core-Edge Transport Simulations): C.S. SAP

The mission of the Computer Science Scientific Application Partnership (C.S. SAP) at LLNL is to develop and apply leading-edge scientific component technology to FACETS software. Contributions from LLNL's fusion energy program staff towards the underlying physics modules are described in a separate report. FACETS uses component technology to combine selectively multiple physics and solver software modules written in different languages by different institutions together in an tightly-integrated, parallel computing framework for Tokamak reactor modeling. In the past fiscal year, the C.S. SAP has focused on two primary tasks: applying Babel to connect UEDGE into the FACETS framework through UEDGE's existing Python interface and developing a next generation componentization strategy for UEDGE which avoids the use of Python. The FACETS project uses Babel to solve its language interoperability challenges. Specific accomplishments for the year include: (1) Refined SIDL interfaces for UEDGE to meet satisfy the standard interfaces required by FACETS for all physics modules. This required consensus building between framework and UEDGE developers. (2) Wrote prototype C++ driver for UEDGE to demonstrate how UEDGE can be called from C++ using Babel. (3) Supported the FACETS project by adding new features to Babel such as release number tagging, porting to new machines, and adding new configuration options. Babel modifications were delivered to FACETS by testing and publishing development snapshots in the projects software repository. (4) Assisted Tech-X Corporation in testing and debugging of a high level build system for the complete FACETS tool chain--the complete list of third-party software libraries that FACETS depends on directly or indirectly (e.g., MPI, HDF5, PACT, etc.). (5) Designed and implemented a new approach to wrapping UEDGE as a FACETS component without requiring Python. To get simulation results as soon as possible, our initial connection from the FACETS framework to UEDGE uses a Forthon-generated Python layer that sits on top of the UEDGE Fortran code. To run on leadership class machines that do not support shared libraries such as Franklin (NERSCS) and Jaguar (ORNL), FACETS requires a more direct connection to the UEDGE Fortran that eliminates the Python layer. We designed and developed an extension for Forthon to provide a direct connection from C++ to the UEDGE Fortran using Babel. This also required added annotations to the Forthon '.v' input files to indicate how function parameters are used.

99 GENERAL AND MISCELLANEOUS↗

Calibration of elastoplastic constitutive model parameters from full-field data with automatic differentiation-based sensitivities

Here, we present a framework for calibration of parameters in elastoplastic constitutive models that is based on the use of automatic differentiation (AD). The model calibration problem is posed as a partial differential equation-constrained optimization problem where a finite element (FE) model of the coupled equilibrium equation and constitutive model evolution equations serves as the constraint. The objective function quantifies the mismatch between the displacement predicted by the FE model and full-field digital image correlation data, and the optimization problem is solved using gradient-based optimization algorithms. Forward and adjoint sensitivities are used to compute the gradient at considerably less cost than its calculation from finite difference approximations. Through the use of AD, we need only to write the constraints in terms of AD objects, where all of the derivatives required for the forward and inverse problems are obtained by appropriately seeding and evaluating these quantities. We present three numerical examples that verify the correctness of the gradient, demonstrate the AD approach's parallel computation capabilities via application to a large-scale FE model, and highlight the formulation's ease of extensibility to other classes of constitutive models.

42 ENGINEERING↗

Algorithms for Efficient Reproducible Floating Point Summation

We define “reproducibility” as getting bitwise identical results from multiple runs of the same program, perhaps with different hardware resources or other changes that should not affect the answer. Many users depend on reproducibility for debugging or correctness. However, dynamic scheduling of parallel computing resources, combined with nonassociative floating point addition, makes reproducibility challenging even for summation, or operations like the BLAS. We describe a “reproducible accumulator” data structure (the “binned number”) and associated algorithms to reproducibly sum binary floating point numbers, independent of summation order. We use a subset of the IEEE Floating Point Standard 754-2008 and bitwise operations on the standard representations in memory. Our approach requires only one read-only pass over the data, and one reduction in parallel, using a 6-word reproducible accumulator (more words can be used for higher accuracy), enabling standard tiling optimization techniques. Summing n words with a 6-word reproducible accumulator requires approximately 9 n floating point operations (arithmetic, comparison, and absolute value) and approximately 3 n bitwise operations. The final error bound with a 6-word reproducible accumulator and our default settings can be up to 2 29 times smaller than the error bound for conventional (recursive) summation on ill-conditioned double-precision inputs.

Computer Science↗