Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,603 records · Page 89

In-Situ Statistical Analysis of Autotune Simulation Data using Graphical Processing Units

Developing accurate building energy simulation models to assist energy efficiency at speed and scale is one of the research goals of the Whole-Building and Community Integration group, which is a part of Building Technologies Research and Integration Center (BTRIC) at Oak Ridge National Laboratory (ORNL). The aim of the Autotune project is to speed up the automated calibration of building energy models to match measured utility or sensor data. The workflow of this project takes input parameters and runs EnergyPlus simulations on Oak Ridge Leadership Computing Facility’s (OLCF) computing resources such as Titan, the world’s second fastest supercomputer. Multiple simulations run in parallel on nodes having 16 processors each and a Graphics Processing Unit (GPU). Each node produces a 5.7 GB output file comprising 256 files from 64 simulations. Four types of output data covering monthly, daily, hourly, and 15- minute time steps for each annual simulation is produced. A total of 270TB+ of data has been produced. In this project, the simulation data is statistically analyzed in-situ using GPUs while annual simulations are being computed on the traditional processors. Titan, with its recent addition of 18,688 Compute Unified Device Architecture (CUDA) capable NVIDIA GPUs, has greatly extended its capability for massively parallel data processing. CUDA is used along with C/MPI to calculate statistical metrics such as sum, mean, variance, and standard deviation leveraging GPU acceleration. The workflow developed in this project produces statistical summaries of the data which reduces by multiple orders of magnitude the time and amount of data that needs to be stored. These statistical capabilities are anticipated to be useful for sensitivity analysis of EnergyPlus simulations

accelerators↗

Data co-processing for extreme scale analysis level II ASC milestone (4745).

Exascale supercomputing will embody many revolutionary changes in the hardware and software of high-performance computing. A particularly pressing issue is gaining insight into the science behind the exascale computations. Power and I/O speed con- straints will fundamentally change current visualization and analysis work ows. A traditional post-processing work ow involves storing simulation results to disk and later retrieving them for visualization and data analysis. However, at exascale, scien- tists and analysts will need a range of options for moving data to persistent storage, as the current o ine or post-processing pipelines will not be able to capture the data necessary for data analysis of these extreme scale simulations. This Milestone explores two alternate work ows, characterized as in situ and in transit, and compares them. We nd each to have its own merits and faults, and we provide information to help pick the best option for a particular use.

Rogers, David↗

A comprehensive approach to decipher biological computation to achieve next generation high-performance exascale computing.

The human brain (volume=1200cm3) consumes 20W and is capable of performing > 10^16 operations/s. Current supercomputer technology has reached 1015 operations/s, yet it requires 1500m^3 and 3MW, giving the brain a 10^12 advantage in operations/s/W/cm^3. Thus, to reach exascale computation, two achievements are required: 1) improved understanding of computation in biological tissue, and 2) a paradigm shift towards neuromorphic computing where hardware circuits mimic properties of neural tissue. To address 1), we will interrogate corticostriatal networks in mouse brain tissue slices, specifically with regard to their frequency filtering capabilities as a function of input stimulus. To address 2), we will instantiate biological computing characteristics such as multi-bit storage into hardware devices with future computational and memory applications. Resistive memory devices will be modeled, designed, and fabricated in the MESA facility in consultation with our internal and external collaborators.

James, Conrad D.↗

SBIR PHASE I FINAL REPORT: Adoption of High Performance Computational (HPC) Modeling Software for Widespread Use in the Manufacture of Welded Structures

Many US manufacturing companies have moved fabrication and production facilities off shore because of cheaper labor costs. A key aspect in bringing these jobs back to the US is the use of technology to render US-made fabrications more efficient overall with higher quality. A new initiative of the current administration has the goal of enhancing competitiveness to retain manufacturing jobs in the US. One significant competitive advantage that has emerged in the US over the last two decades is the use of virtual design for fabrication of large structures in the light and heavy materials industries. Industries that have used virtual design and analysis tools have reduced material parts size, developed environmentally-friendly fabrication processes, improved product quality and performance, and reduced manufacturing costs. Indeed, Caterpillar Inc. (CAT), one of the partners in this effort, continues to have a large fabrication presence in the US because of the use of weld fabrication modeling to optimize fabrications by controlling weld residual stresses and distortions and improving fatigue, corrosion, and fracture performance. This report describes Engineering Mechanics Corporation of Columbus (Emc2's) DOE SBIR Phase I results which extended an existing, state-of-the-art software code, VFT, currently used to design and model large welded structures prior to fabrication - to a broader range of products with widespread applications for small and medium-sized enterprises (SMEs). VFT helps control distortion, can minimize and/or control residual stresses, control welding microstructure, and pre-determine welding parameters such as weld-sequencing, pre-bending, thermal-tensioning, etc. VFT uses material properties, consumable properties, etc. as inputs. Through VFT, manufacturing companies can avoid costly design changes after fabrication. This leads to the concept of joint design/fabrication where these important disciplines are intimately linked to minimize fabrication costs. VFT currently is tied to a commercial solver which makes it prohibitively expensive for use by SMEs, as there is a significant licensing cost for the solver - over and above for the relatively minimal cost for VFT. Emc2 developed this software code over a number of years in close cooperation with CAT (Peoria, IL), who currently uses this code exclusively for worldwide fabrication, product design and development activities. The use of VFT has allowed CAT to move directly from design to product fabrication and helped eliminate (to a large extent) new product prototyping and subsequent testing. Additionally, CAT has been able to eliminate/reduce costly one-of-a-kind appliances used to reduce distortion effects due to fabrication. In this context, SMEs can realize the same kind of improved product quality and reduced cost through adoption of the adapted version of VFT for design and subsequent manufacture of new products. Emc2's DOE SBIR Phase I effort successfully adapted VFT so that SMEs have access to this sophisticated and proven methodology that is quick, accurate and cost effective and available on-demand to address weld-simulation and fabrication problems prior to manufacture. The open source code, WARP3D, a high performance finite element code mainly used in fracture and damage assessment of structures, was modified so that computational weld problems can be solved efficiently on multiple processors and threads with VFT. The thermal solver for VFT, based on a series of closed form solution approximations, was enhanced for solution on multiple processors greatly increasing overall speed. In addition, the graphical user interface (GUI) has been tailored to integrate these solutions with WARP3D. The GUI is used to define all the weld pass descriptions, number of passes, material properties, consumable properties, weld speed, etc. for the structure to be modeled. The GUI was improved to make it user-friendly for engineers that are not experts in finite element modeling. Finally, a plan for porting VFT onto the Ohio Supercomputer Center (OSC) through its hosted Manufacturing and Polymer Portal has been developed. This access route will permit SMEs to perform weld modeling to improve their competitiveness at a reasonable cost. All of these improvements are detailed in this repo

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

SISGR -- Domain Microstructures and Mechanisms for Large, Reversible and Anhysteretic Strain Behaviors in Phase Transforming Ferroelectric Materials

This four-year project (including one-year no-cost extension) aimed to advance fundamental understanding of field-induced strain behaviors of phase transforming ferroelectrics. We performed meso-scale phase field modeling and computer simulation to study domain evolutions, mechanisms and engineering techniques, and developed computational techniques for nanodomain diffraction analysis; to further support above originally planned tasks, we also carried out preliminary first-principles density functional theory calculations of point defects and domain walls to complement meso-scale computations as well as performed in-situ high-energy synchrotron X-ray single crystal diffraction experiments to guide theoretical development (both without extra cost to the project thanks to XSEDE supercomputers and DOE user facility Advanced Photon Source).

36 MATERIALS SCIENCE↗

A Model-Based Case for Redundant Computation

Despite its seemingly nonsensical cost, we show through modeling and simulation that redundant computation merits full consideration as a resilience strategy for next-generation systems. Without revolutionary breakthroughs in failure rates, part counts, or stable-storage bandwidths, it has been shown that the utility of Exascale systems will be crushed by the overheads of traditional checkpoint/restart mechanisms. Alternate resilience strategies must be considered, and redundancy is a proven unrivaled approach in many domains. We develop a distribution-independent model for job interrupts on systems of arbitrary redundancy, adapt Daly’s model for total application runtime, and find that his estimate for optimal checkpoint interval remains valid for redundant systems. We then identify conditions where redundancy is more cost effective than non-redundancy. These are done in the context of the number one supercomputers of the last decade, showing that thorough consideration of redundant computation is timely - if not overdue.

97 MATHEMATICS AND COMPUTING↗

Final Report for Project "Framework Application for Core-Edge Transport Simulations (FACETS)"

This is the final report for the Colorado State University Component of the FACETS Project. FACETS was focused on the development of a multiphysics, parallel framework application that could provide the capability to enable whole-device fusion reactor modeling and, in the process, the development of the modeling infrastructure and computational understanding needed for ITER. It was intended that FACETS be highly flexible, through the use of modern computational methods, including component technology and object oriented design, to facilitate switching from one model to another for a given aspect of the physics, and making it possible to use simplified models for rapid turnaround or high-fidelity models that will take advantage of the largest supercomputer hardware. FACETS was designed in a heterogeneous parallel context, where different parts of the application can take advantage through parallelism based on task farming, domain decomposition, and/or pipelining as needed and applicable. As with all fusion simulations, an integral part of the FACETS project was treatment of the coupling of different physical processes at different scales interacting closely. A primary example for the FACETS project is the coupling of existing core and edge simulations, with the transport and wall interactions described by reduced models. However, core and edge simulations themselves involve significant coupling of different processes with large scale differences. Numerical treatment of coupling is impacted by a number of factors including, scale differences, form of information transferred between processes, implementation of solvers for different codes, and high performance computing concerns. Operator decomposition involving the computation of the individual processes individually using appropriate simulation codes and then linking/synchronizing the component simulations at regular points in space and time, is the defacto approach to high performance simulation of multiphysics, multiscale systems. Various forms of operator decomposition are used in nearly all fusion simulations. However, operator decomposition generally has a complex effect on accuracy and stability of numerical simulations. Yet, these effects can be difficult to detect. The Colorado State University component of the FACETS team led by P.I.D. Estep was focused on analyzing the effects of operator decomposition on fusion simulations. The approach was based on a posteriori error analysis employing adjoint problems, computable residuals, and variational analysis to produce accurate computational error estimates for quantities of interest. Computable residuals are used to quantify the effects of various discretization choices. The generalized Greens function satisfying the adjoint problem quantities the effects of stability. Technical issues to be addressed included: (1) defining appropriate adjoint operators for operator decomposition discretizations; (2) determining the appropriate residuals for the multifaceted aspects involved with multiphysics discretizations; (3) producing the estimates within the computational framework of existing fusion codes; (4) carrying out the analysis for discretizations used in fusion simulations; and (5) devising efficient approaches to mitigating the effects of discretization. This report provides a summary of the accomplished research and a detailed description of personnel, activities, outcomes and achievements.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Final Project Report "Advanced Concept Exploration For Fast Ignition Science Program"

The Fast Ignition (FI) Concept for Inertial Confinement Fusion has the potential to provide a significant advance in the technical attractiveness of Inertial Fusion Energy (IFE) reactors. FI differs from conventional “central hot spot” (CHS) target ignition by decoupling compression from heating: using the laser (or heavy ion beam or Z pinch) drive pulse (10’s of ns) to create a dense fuel and a second, much shorter (~10 ps) high intensity pulse to ignite a small region of it. There are two major physics issues concerning this concept; controlling the laser-induced generation of large electron currents and their propagation through high density plasmas. This project has addressed these two significant scientific issues in Relativistic High Energy Density (RHED) physics. Learning to control relativistic laser matter interaction (and the limits and potential thereof) will enable a wide range of applications. While these physics issues are of specific interest to inertial fusion energy science, they are also important for a wide range of other HED phenomena, including high energy ion beam generation, isochoric heating of materials, and the development of high brightness x-ray sources. Generating, controlling, and understanding the extreme conditions needed to advance this science has proved to be challenging: Our studies have pushed the boundaries of physics understanding and are at the very limits of experimental, diagnostic, and simulation capabilities in high energy density laboratory physics (HEDLP). Our research strategy has been based on pursuing the fundamental physics underlying the Fast Ignition (FI) concept. We have performed comprehensive study of electron generation and transport in fast-ignition targets with experiments, theory, and numerical modeling. A major issue is that the electrons produced in these experiments cannot be measured directly—only effects due to their transport. We focused mainly on x-ray continuum photons from bremsstrahlung and x-ray line radiation from K-shell fluorescence. Integrated experiments, which combine target compression with short-pulse laser heating, yield additional information on target heating efficiency. This indirect way of studying the underlying behavior of the electrons must be validated with computational modeling to understand the physics and improve the design. This program execution required a large, well-organized team and it was managed by a joint Collaboration between General Atomics (GA), Lawrence Livermore National Laboratory (LLNL), and the Laboratory for Laser Energetics (LLE). The Collaboration was formed 8 years ago to understand the physics issues of the Fast Ignition concept, building on the strengths of each partner. GA fulfills its responsibilities jointly with the University of California, San Diego (UCSD), The Ohio State University (OSU) and the University of Nevada at Reno (UNR). Since RHED physics is pursued vigorously in many countries, international researchers have been an important part of our efforts to make progress. The division of responsibility was as follows: (1) LLE had primary leadership for channeling studies and the integrated energy transfer, (2) LLNL led the development of measurement methods, analysis, and deployment of diagnostics, and (3) GA together with UCSD, OSU and UNR studied the detailed energy-transfer physics. The experimental program was carried out using the Titan laser at the Jupiter Laser Facility at LLNL, the OMEGA and OMEGA EP lasers at LLE and the Texas Petawatt laser (TPW) at UT Austin. Modeling has been pursued on large computing facilities at LLNL, OSU, and UCSD using codes developed (by us and others) within the HEDLP program, commercial codes, and by leveraging existing supercomputer codes developed by the NNSA ICF program. This Consortium brought together all the components—resources, facilities, and personnel—necessary to accomplish its aggressive goals. The ACE Program has been strongly collaborative, taking advantage of the expertise of the participating institutions to provide a research effort that is far greater than the sum of its parts. The results of this work have firmly strengthened the scientific foundation from which the viability of FI and other applications can be evaluated. Program execution has also led to improved diagnostics for probing dense, hot plasmas, detailed understanding of high-current, relativistic electron energy generation and transport across boundaries and into dense plasmas, and greatly enhanced predictive modeling capabilities. One important aspect of this program was the involvement and training of young scientists including postdoctoral fellows and graduate students. During the entire 8 years of FI and ACE project period since 2005, more than fifteen graduate students completed their doctoral dissertations including three from OSU and two from UCSD in last three years. This project generated an impressive forty articles in high quality journals including nine (including two under review) in Physical Review Letters during the last funding period since 2011.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Construction of Blaze at the University of Illinois at Chicago: A Shared, High-Performance, Visual Computer for Next-Generation Cyberinfrastructure-Accelerated Scientific, Engineering, Medical and Public Policy Research

The Blaze high-performance visual computing system serves the high-performance computing research and education needs of University of Illinois at Chicago (UIC). Blaze consists of a state-of-the-art, networked, computer cluster and ultra-high-resolution visualization system called CAVE2(TM) that is currently not available anywhere in Illinois. This system is connected via a high-speed 100-Gigabit network to the State of Illinois' I-WIRE optical network, as well as to national and international high speed networks, such as the Internet2, and the Global Lambda Integrated Facility. This enables Blaze to serve as an on-ramp to national cyberinfrastructure, such as the National Science Foundation’s Blue Waters petascale computer at the National Center for Supercomputing Applications at the University of Illinois at Chicago and the Department of Energy’s Argonne Leadership Computing Facility (ALCF) at Argonne National Laboratory. DOE award # DE-SC005067, leveraged with NSF award #CNS-0959053 for “Development of the Next-Generation CAVE Virtual Environment (NG-CAVE),” enabled us to create a first-of-its-kind high-performance visual computing system. The UIC Electronic Visualization Laboratory (EVL) worked with two U.S. companies to advance their commercial products and maintain U.S. leadership in the global information technology economy. New applications are being enabled with the CAVE2/Blaze visual computing system that is advancing scientific research and education in the U.S. and globally, and help train the next-generation workforce.

47 OTHER INSTRUMENTATION↗

Heterogeneous scalable framework for multiphase flows

Two categories of challenges confront the developer of computational spray models: those related to the computation and those related to the physics. Regarding the computation, the trend towards heterogeneous, multi- and many-core platforms will require considerable re-engineering of codes written for the current supercomputing platforms. Regarding the physics, accurate methods for transferring mass, momentum and energy from the dispersed phase onto the carrier fluid grid have so far eluded modelers. Significant challenges also lie at the intersection between these two categories. To be competitive, any physics model must be expressible in a parallel algorithm that performs well on evolving computer platforms. This work created an application based on a software architecture where the physics and software concerns are separated in a way that adds flexibility to both. The develop spray-tracking package includes an application programming interface (API) that abstracts away the platform-dependent parallelization concerns, enabling the scientific programmer to write serial code that the API resolves into parallel processes and threads of execution. The project also developed the infrastructure required to provide similar API s to other application. The API allow object-oriented Fortran applications direct interaction with Trilinos to support memory management of distributed objects in central processing units (CPU) and graphic processing units (GPU) nodes for applications using C++.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

High performance graphics processor based computed tomography reconstruction algorithms for nuclear and other large scale applications.

The goal of this work is to develop a fast computed tomography (CT) reconstruction algorithm based on graphics processing units (GPU) that achieves significant improvement over traditional central processing unit (CPU) based implementations. The main challenge in developing a CT algorithm that is capable of handling very large datasets is parallelizing the algorithm in such a way that data transfer does not hinder performance of the reconstruction algorithm. General Purpose Graphics Processing (GPGPU) is a new technology that the Science and Technology (S&T) community is starting to adopt in many fields where CPU-based computing is the norm. GPGPU programming requires a new approach to algorithm development that utilizes massively multi-threaded environments. Multi-threaded algorithms in general are difficult to optimize since performance bottlenecks occur that are non-existent in single-threaded algorithms such as memory latencies. If an efficient GPU-based CT reconstruction algorithm can be developed; computational times could be improved by a factor of 20. Additionally, cost benefits will be realized as commodity graphics hardware could potentially replace expensive supercomputers and high-end workstations. This project will take advantage of the CUDA programming environment and attempt to parallelize the task in such a way that multiple slices of the reconstruction volume are computed simultaneously. This work will also take advantage of the GPU memory by utilizing asynchronous memory transfers, GPU texture memory, and (when possible) pinned host memory so that the memory transfer bottleneck inherent to GPGPU is amortized. Additionally, this work will take advantage of GPU-specific hardware (i.e. fast texture memory, pixel-pipelines, hardware interpolators, and varying memory hierarchy) that will allow for additional performance improvements.

97 MATHEMATICS AND COMPUTING↗

Louisiana Immersive Technologies Enterprise (LITE): Enabling Coastal Stewardship and Reliable Energy

This project will focus on research and application development in the area of large scale data computing and visualization that are critical to the DoE BER program areas of climate change research, environmental sciences, life sciences, and medical sciences. Most commercial applications available today are unable to handle the sizes of data in these domains, usually in the order of terabytes and petabytes. This project will address this problem, by taking advantage of the advanced scientific computing infrastructure of the LITE and LONI to conduct research and develop software applications that run natively on these 64-bit supercomputing platforms. The project is consistent with the goals of Scientific Discovery through Advanced Computing (SciDAC), a DoE initiative.

54 ENVIRONMENTAL SCIENCES↗

Final Report for Project DE-FC02-06ER25755 [Pmodels2]

In this report, we describe the research accomplished by the OSU team under the Pmodels2 project. The team has worked on various angles: designing high performance MPI implementations on modern networking technologies (Mellanox InfiniBand (including the new ConnectX2 architecture and Quad Data Rate), QLogic InfiniPath, the emerging 10GigE/iWARP and RDMA over Converged Enhanced Ethernet (RoCE) and Obsidian IB-WAN), studying MPI scalability issues for multi-thousand node clusters using XRC transport, scalable job start-up, dynamic process management support, efficient one-sided communication, protocol offloading and designing scalable collective communication libraries for emerging multi-core architectures. New designs conforming to the Argonne’s Nemesis interface have also been carried out. All of these above solutions have been integrated into the open-source MVAPICH/MVAPICH2 software. This software is currently being used by more than 2,100 organizations worldwide (in 71 countries). As of January ’14, more than 200,000 downloads have taken place from the OSU Web site. In addition, many InfiniBand vendors, server vendors, system integrators and Linux distributors have been incorporating MVAPICH/MVAPICH2 into their software stacks and distributing it. Several InfiniBand systems using MVAPICH/MVAPICH2 have obtained positions in the TOP500 ranking of supercomputers in the world. The latest November ’13 ranking include the following systems: 7th ranked Stampede system at TACC with 462,462 cores; 11th ranked Tsubame 2.5 system at Tokyo Institute of Technology with 74,358 cores; 16th ranked Pleiades system at NASA with 81,920 cores; Work on PGAS models has proceeded on multiple directions. The Scioto framework, which supports task-parallelism in one-sided and global-view parallel programming, has been extended to allow multi-processor tasks that are executed by processor groups. A quantum Monte Carlo application is being ported onto the extended Scioto framework. A public release of Global Trees (GT) has been made, along with the Global Chunks (GC) framework on which GT is built. The Global Chunks (GC) layer is also being used as the basis for the development of a higher level Global Graphs (GG) layer. The Global Graphs (GG) system will provide a global address space view of distributed graph data structures on distributed memory systems.

97 MATHEMATICS AND COMPUTING↗

Proceedings of RIKEN BNL Research Center Workshop: The Physics of p ↑ +A Collisions at RHIC

The RIKEN BNL Research Center (RBRC) was established in April 1997 at Brookhaven National Laboratory. It is funded by the "Rikagaku Kenkyusho" (RIKEN, The Institute of Physical and Chemical Research) of Japan. The Memorandum of Understanding between RIKEN and BNL, initiated in 1997, has been renewed in 2002, 2007 and again in 2012. The Center is dedicated to the study of strong interactions, including spin physics, lattice QCD, and RHIC physics through the nurturing of a new generation of young physicists. The RBRC has both a theory and experimental component. The RBRC Theory Group and the RBRC Experimental Group consists of a total of 25-30 researchers. Positions include the following: full time RBRC Fellow, half-time RHIC Physics Fellow, and full-time postdoctoral Research Associate. The RHIC Physics Fellows hold joint appointments with RBRC and other institutions and have tenure track positions at their respective universities or BNL. To date, RBRC has over 95 graduates (Fellows and Post- docs) of which approximately 40 theorists and 20 experimenters have already attained tenure positions at major institutions worldwide. Beginning in 2001 a new RIKEN Spin Program (RSP) category was implemented at RBRC. These appointments are joint positions of RBRC and RIKEN and include the following positions in theory and experiment: RSP Researchers, RSP Research Associates, and Young Researchers, who are mentored by senior RBRC Scientists. A number of RIKEN Jr. Research Associates and Visiting Scientists also contribute to the physics program at the Center. RBRC has an active workshop program on strong interaction physics with each workshop focused on a specific physics problem. In most cases all the talks are made available on the RBRC website. In addition, highlights to each speaker’s presentation are collected to form proceedings which can therefore be made available within a short time after the workshop. To date there are over one hundred proceeding volumes available. A 10 teraflops RBRC QCDOC computer funded by RIKEN, Japan, was unveiled at a dedication ceremony at BNL on May 26, 2005. This supercomputer was designed and built by individuals from Columbia University, IBM, BNL, RBRC, and the University of Edinburgh, with the U.S. D.O.E. Office of Science providing infrastructure support at BNL. Physics results were reported at the RBRC QCDOC Symposium following the dedication. QCDSP, a 0.6 teraflops parallel processor, dedicated to lattice QCD, was begun at the Center on February 19, 1998, was completed on August 28, 1998, and was decommissioned in 2006. It was awarded the Gordon Bell Prize for price performance in 1998. QCDOC was decommissioned in May 2012. The next generation computer in this sequence, QCDCQ (600 Teraflops), is currently operational and is expected to produce many more interesting discoveries in the future.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Discovering The Folding Rules That Proteins Obey FY08 LDRD Final Report

This project’s objective was to study protein folding using a combination of experimental, theoretical and simulation methods. Our study was enabled by a robust microfluidic mixing devices and by long time simulations on supercomputers. The project partially funded research for 7 published articles and we anticipate one more publication for which some of the data was taken using LDRD funds. The publications describe development of new instrumentation for studies of protein folding (2, 3, 4, 5, 6) and new measurements of protein folding kinetics (1, 4, 5, 8) that contribute to furthering of our understanding of the complex folding phenomena. The LDRD project also supported Dr. Bakajin’s work on a review article (7).

59 BASIC BIOLOGICAL SCIENCES↗

Performance Refactoring of Instrumentation, Measurement, and Analysis Technologies for Petascale Computing: the PRIMA Project

The growing number of cores provided by today’s high-end computing systems present substantial challenges to application developers in their pursuit of parallel efficiency. To find the most effective optimization strategy, application developers need insight into the runtime behavior of their code. The University of Oregon (UO) and the Julich Supercomputing Centre of Forschungszentrum Julich (FZJ) developed the performance analysis tools TAU and Scalasca, respectively, which allow high-performance computing (HPC) users to collect and analyze relevant performance data – even at very large scales. TAU and Scalasca are considered among the most advanced parallel performance systems available, and are used extensively across HPC centers in the U.S., Germany, and around the world.

97 MATHEMATICS AND COMPUTING↗

Pervasive Restart In MOOSE-based Applications

Multiphysics applications are inherently complicated. Solving for multiple, interacting physical phenomena involves the solution of multiple equations, and each equation has its own data dependencies. Feeding the correct data to these equations at exactly the right time requires extensive effort in software design. In an ideal world, multiphysics applications always run to completion and produce correct answers. Unfortunately, in reality, there can be many reasons why a simulation might fail: power outage, system failure, exceeding a runtime allotment on a supercomputer, failure of the solver to converge, etc. A failure after many hours spent computing can be a significant setback for a project. Therefore, the ability to “continue” a solve from the point of failure, rather than starting again from scratch, is an essential component of any high-quality simulation tool. This process of “continuation” is commonly termed “restart” in the computational community. While the concept of restarting an application sounds ideal, the aforementioned complexities and data dependencies present in multiphysics applications make its implementation decidedly non-trivial. A running multiphysics calculation accumulates an enormous amount of “state”: current time, solution history, material properties, status of mechanical contact, etc. This “state” data comes in many different forms, including scalar, tensor, vector, and arbitrary, application-specific data types. To be able to restart an application, you must be able to both store and retrieve this data, effectively recreating the state of the application before the failure. When utilizing the Multiphysics Object Oriented Simulation Environment (MOOSE) framework developed at Idaho National Laboratory, this state data is stored both internally within the framework itself (such as solution vectors and the current time) and within the applications that use the framework. In order to implement restart in MOOSE-based applications, the total state of the system (both within the framework and without) must be stored and retrieved. To this end, the MOOSE team has implemented a “pervasive” restart capability which allows any object within MOOSE (or within a MOOSE-based application) to be declared as “state” data, and handles the storage and retrieval of said data.

97 MATHEMATICS AND COMPUTING↗

High Energy Physics and Nuclear Physics Network Requirements

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the U.S. Department of Energy (DOE) Office of Science (SC), the single largest supporter of basic research in the physical sciences in the United States. In support of SC programs, ESnet regularly updates and refreshes its understanding of the networking requirements needed by instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 25 years. In August 2013, ESnet and the DOE SC Offices of High Energy Physics (HEP) and Nuclear Physics (NP) organized a review to characterize the networking requirements of the programs funded by the HEP and NP program offices. Several key findings resulted from the review. Among them: 1. The Large Hadron Collider's ATLAS (A Toroidal LHC Apparatus) and CMS (Compact Muon Solenoid) experiments are adopting remote input/output (I/O) as a core component of their data analysis infrastructure. This will significantly increase their demands on the network from both a reliability perspective and a performance perspective. 2. The Large Hadron Collider (LHC) experiments (particularly ATLAS and CMS) are working to integrate network awareness into the workflow systems that manage the large number of daily analysis jobs (1 million analysis jobs per day for ATLAS), which are an integral part of the experiments. Collaboration with networking organizations such as ESnet, and the consumption of performance data (e.g., from perfSONAR [PERformance Service Oriented Network monitoring Architecture]) are critical to the success of these efforts. 3. The international aspects of HEP and NP collaborations continue to expand. This includes the LHC experiments, the Relativistic Heavy Ion Collider (RHIC) experiments, the Belle II Collaboration, the Large Synoptic Survey Telescope (LSST), and others. The international nature of these collaborations makes them heavily reliant on transoceanic connectivity, which is subject to longer term service disruptions than terrestrial connectivity. The network engineering aspects of undersea connectivity will continue to be a significant part of the planning, deployment, and operation of the data analysis infrastructure for HEP and NP experiments for the foreseeable future. Given their critical dependency on networking services, the experiments have expressed the need for tight integration (both technically and operationally) of the domestic and the transoceanic parts of the network infrastructure that supports the experiments. 4. The datasets associated with simulations continue to increase in size, and the need to move these datasets between analysis centers is placing ever-increasing demands on networks and on data management systems at the supercomputing centers. In addition, there is a need to harmonize cybersecurity practice with the data transfer performance requirements of the science. This report expands on these points, and addresses others as well. The report contains a findings section in addition to the text of the case studies discussed during the review.

43 PARTICLE ACCELERATORS↗