Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “execution model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

CIGS Technology Advancement via Fundamental Modeling of Defect/Impurity Interactions (Final Technical Report)

The primary goals of the proposed work were to provide modeling tools (and the associated insight which comes along with model development) for design and optimization of CuIn x Ga 1-x Se 2 (CIGS) and CdSeTe (CST) solar cell manufacturing processes and to establish the foundation for comprehensive end-to-end predictive modeling tools to enable optimization of thin film photovoltaic technology for performance, cost, yield, and reliability. The initial focus of efforts within this project was to develop coupled process/optical/device models for CIGS PV technology and to work with Siva Power to apply that TCAD (technology computer-aided design) system to improve the efficiency and reduce manufacturing costs for CIGS solar cells. Our approach to that end was to generate an extensive database of DFT calculations and to use those calculations via statistical thermodynamics methods and Monte Carlo simulation to develop and characterize models for the behavior of native defects as well as intentional and unintentional impurities, including the redistribution of the primary components of CIGS films. Increased effort went toward coupling those models for defect behavior and composition evolution to the performance of multicrystalline CIGS solar cells via prediction of doping level and recombination lifetime as function of manufacturing process. In the second budget period, the project pivoted to developing a similar system for the CdSeTe system, focused especially on understanding the role of Se/Te alloy concentration. Execution of the project resulted in the successful development of TCAD systems for both CIGS and CdSeTe thin film PV within the Synopsys Sentaurus framework by utilizing the Alagator interface. In the first budget period of the project, we developed quantitative models for the major components of CIGS PV and implemented them within a framework that couples process, optical, and device simulation. From the insights we have gained, we identified novel opportunities for enhancing CIGS solar cell performance and have laid the groundwork to further optimize the layer structure, composition profile, and thermal cycles for substantially improved efficiency and lower manufacturing costs. For the CIGS system, process changes to achieve greater than 1% absolute enhancement in efficiency were identified, but testing of those approaches was stymied by lack of a domestic CIGS manufacturing partner after the closure of Siva Power as well as Miasole. For CdSeTe, a fully capable TCAD system only became ready to apply near the end of the project period, so substantial opportunities remain to apply those models to enhance the leading thin film PV technology.

14 SOLAR ENERGY↗

Thermal hydraulic model of the molten salt reactor experiment with the NEAMS system analysis module

System analysis codes have a long history of providing best-estimate and conservative safety analysis for both light water and advanced reactor technologies, including molten salt reactors. As interest continues to expand with advanced reactor concepts, system analysis codes will need revisions to accommodate the behavior of these technologies. Legacy system analysis codes will need to be updated to the latest numerical techniques to shorten execution time and increase the accuracy of results. One example of a modern system analysis code that already encompasses these characteristics is the System Analysis Module (SAM). One key objective of this paper was to review available information for system code modeling of the Molten Salt Reactor Experiment (MSRE) from sources in the open literature and collect the information from these open sources in one place for the first time. This supports the potential objective of developing an open specification for system code analysis for MSRE steady state and transients with and without reactor kinetics. Data from actual MSRE tests will serve as the basis for code-to-code comparison exercises, including the MSRE zero power physics tests, the fuel pump start-up and coast down tests, and the natural circulation transient. The objective is to produce a code-to-code benchmark with a standardized set of comparison problems, recognizing the limitations of the original data. To demonstrate an initial application of this objective and the usefulness of compiling this open data, two Molten Salt Reactor Experiment (MSRE)-related models were developed to evaluate SAM for liquid fueled molten salt reactors. One model was the SAM MSRE hydraulic mockup, which provided experimental data for pressure drop measurements. The second model was the complete MSRE primary loop. The MSRE primary loop model incorporated a fluoride salt fuel/coolant with heat transfer in both the core and heat exchanger. For both the hydraulic mockup and MSRE primary loop models, a holistic 1-D system description was built using open documentation, an open description that can be readily modified and applied for any system analysis code. SAM results for the pressure drop of the hydraulic mockup model were within 6% with measurements. Coolant temperatures for the primary loop model matched the expected axial change in temperature from historical calculations. Using alternative coolant properties obtained from the literature, corresponding to salts with different actinide contents, returned similar trends in core temperature profiles. A thermal hydraulic demonstration of a loss-of-flow transient showed the importance of coupling SAM thermal hydraulic analysis to neutronics. Here, this coupling is essential for simulating MSR transients with system analysis codes.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Collaborative Project: Understanding Climate Model Biases in Tropical Atlantic and Their Impact on Simulations of Extreme Climate Events

Recent studies have revealed that among all the tropical oceans, the tropical Atlantic has experienced the most pronounced warming trend over the 20th century. Many extreme climate events affecting the U.S., such as hurricanes, severe precipitation and drought events, are influenced by conditions in the Gulf of Mexico and the Atlantic Ocean. It is therefore imperative to have accurate simulations of the climatic mean and variability in the Atlantic region to be able to make credible projections of future climate change affecting the U.S. and other countries adjoining the Atlantic Ocean. Unfortunately, almost all global climate models exhibit large biases in their simulations of tropical Atlantic climate. The atmospheric convection simulation errors in the Amazon region and the associated errors in the trade wind simulations are hypothesized to be a leading cause of the tropical Atlantic biases in climate models. As global climate models have resolutions that are too coarse to resolve some of the atmospheric and oceanic processes responsible for the model biases, we propose to use a high-resolution coupled regional climate model (CRCM) framework to address the tropical bias issue. We propose to combine the expertise in tropical coupled atmosphere-ocean modeling at Texas A&M University (TAMU) and the coupled land-atmosphere modeling expertise at Pacific Northwest National Laboratory (PNNL) to develop a comprehensive CRCM for the Atlantic sector within a general and flexible modeling framework. The atmospheric component of the CRCM will be the NCAR WRF model and the oceanic component will be the Rutgers/UCLA ROMS. For the land component, we will use CLM modified at PNNL to include more detailed representations of vegetation and soil hydrology processes. The combined TAMU-PNNL CRCM model will be used to simulate the Atlantic climate, and the associated land-atmosphere-ocean interactions at a horizontal resolution of 9 km or finer. A particular focus of the model development effort will be to optimize the performance of WRF and ROMS over several thousand of cores by focusing on both the parallel communication libraries and the I/O interfaces, in order to achieve the sustained throughput needed to perform simulations on such fine resolution grids. The CRCM model will be developed within the framework of the Coupler (CPL7) software that is part of the NCAR Community Earth System Model (CESM). Through efforts at PNNL and within the community, WRF and CLM have already been coupled via CPL7. Using the flux coupler approach for the whole CRCM model will allow us to flexibly couple WRF, ROMS, and CLM with each model running on its own grid at different resolutions. In addition, this framework will allow us to easily port parameterizations between CESM and the CRCM, and potentially allow partial coupling between the CESM and the CRCM. TAMU and PNNL will contribute cooperatively to this research endeavor. The TAMU team led by Chang and Saravanan has considerable experience in studying atmosphere-ocean interactions within tropical Atlantic sector and will focus on modeling issues that relate to coupling WRF and ROMS. The PNNL team led by Leung has extensive expertise in atmosphere-land interaction and will be responsible for improving the land surface parameterization. Both teams will jointly work on integrating WRF-ROMS and WRF-CLM to couple WRF, ROMS, and CLM through CPL7. Montuoro of the TAMU Supercomputing Center will be responsible for improving the MPI and Parallel IO interfaces of the CRCM. Both teams will contribute to the design and execution of the proposed numerical experiments and jointly perform analysis of the numerical experiments.

54 ENVIRONMENTAL SCIENCES↗

Parallel application-level behavioral attributes for performance and energy management of high-performance computing systems

Run time variability of parallel applications continues to present significant challenges to their performance and energy efficiency in high-performance computing (HPC) systems. When run times are extended and unpredictable, application developers perceive this as a degradation of system (or subsystem) performance. Extended run times directly contribute to proportionally higher energy consumption, potentially negating efforts by applications, or the HPC system, to optimize energy consumption using low-level control techniques, such as dynamic voltage and frequency scaling (DVFS). Therefore, successful systemic management of application run time performance can result in less wasted energy, or even energy savings. We have been studying run time variability in terms of communication time, from the perspective of the application, focusing on the interconnection network. More recently, our focus has shifted to developing a more complete understanding of the effects of HPC subsystem interactions on parallel applications. In this context, the set of executing applications on the HPC system is treated as a subsystem, along with more traditional subsystems like the communication subsystem, storage subsystem, etc. To gain insight into the run time variability problem, our earlier work developed a framework to emulate parallel applications (PACE) that stresses the communication subsystem. Evaluation of run time sensitivity to network performance of real applications is performed with a tool called PARSE, which uses PACE. In this paper, we propose a model defining application-level behavioral attributes, that collectively describes how applications behave in terms of their run time performance, as functions of their process distribution on the system (spacial locality), and subsystem interactions (communication subsystem degradation). These subsystem interactions are produced when multiple applications execute concurrently on the same HPC system. We also revisit our evaluation framework and tools to demonstrate the flexibility of our application characterization techniques, and the ease with which attributes can be quantified. The validity of the model is demonstrated using our tools with several parallel benchmarks and application fragments. Results suggest that it is possible to articulate application-level behavioral attributes as a tuple of numeric values that describe course-grained performance behavior.

Computer Science↗

Full waveform inversion for time-distance helioseismology

Inferring interior properties of the Sun from photospheric measurements of the seismic wavefield constitutes the helioseismic inverse problem. Deviations in seismic measurements (such as wave travel times) from their fiducial values estimated for a given model of the solar interior imply that the model is inaccurate. Contemporary inversions in local helioseismology assume that properties of the solar interior are linearly related to measured travel-time deviations. It is widely known, however, that this assumption is invalid for sunspots and active regions and is likely for supergranular flows. Here, we introduce nonlinear optimization, executed iteratively, as a means of inverting for the subsurface structure of large-amplitude perturbations. Defining the penalty functional as the L {sub 2} norm of wave travel-time deviations, we compute the total misfit gradient of this functional with respect to the relevant model parameters at each iteration around the corresponding model. The model is successively improved using either steepest descent, conjugate gradient, or the quasi-Newton limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Performing nonlinear iterations requires privileging pixels (such as those in the near field of the scatterer), a practice that is not compliant with the standard assumption of translational invariance. Measurements for these inversions, although similar in principle to those used in time-distance helioseismology, require some retooling. For the sake of simplicity in illustrating the method, we consider a two-dimensional inverse problem with only a sound-speed perturbation.

79 ASTRONOMY AND ASTROPHYSICS↗

Predictive scale-bridging simulations through active learning

Throughout computational science, there is a growing need to utilize the continual improvements in raw computational horsepower to achieve greater physical fidelity through scale-bridging over brute-force increases in the number of mesh elements. For instance, quantitative predictions of transport in nanoporous media, critical to hydrocarbon extraction from tight shale formations, are impossible without accounting for molecular-level interactions. Similarly, inertial confinement fusion simulations rely on numerical diffusion to simulate molecular effects such as non-local transport and mixing without truly accounting for molecular interactions. With these two disparate applications in mind, we develop a novel capability which uses an active learning approach to optimize the use of local fine-scale simulations for informing coarse-scale hydrodynamics. Our approach addresses three challenges: forecasting continuum coarse-scale trajectory to speculatively execute new fine-scale molecular dynamics calculations, dynamically updating coarse-scale from fine-scale calculations, and quantifying uncertainty in neural network models.

97 MATHEMATICS AND COMPUTING↗

Summary of the Persistent Current Effect Measurements in Nb 3 Sn and NbTi Accelerator Magnets at Fermilab

In the past 10 years, Fermilab has been executing an intensive R&D program on accelerator magnets based on Nb 3 Sn superconductor technology. This R&D effort includes dipole and quadrupole models for different programs, such as LARP and 11 T dipoles for the LHC high-luminosity upgrade. Before the Nb 3 Sn R&D program, Fermilab was involved in the production of the low-beta quadrupole magnets for LHC based on the NbTi superconductor. Additionally, during the 2003-2005 campaign to optimize the operation of the Tevatron, a large number of Tevatron magnets were re-measured. As a result of this field analysis, a systematic study of the persistent current decay and snapback effect in these magnets was performed. This paper summarizes the result of this study and presents a comparison between Nb 3 Sn and NbTi dipoles and quadrupoles.

43 PARTICLE ACCELERATORS↗

Generalizable coordination of large multiscale workflows: challenges and learnings at scale

The advancement of machine learning techniques and the heterogeneous architectures of most current supercomputers are propelling the demand for large multiscale simulations that can automatically and autonomously couple diverse components and map them to relevant resources to solve complex problems at multiple scales. Nevertheless, despite the recent progress in workflow technologies, current capabilities are limited to coupling two scales. In the first-ever demonstration of using three scales of resolution, we present a scalable and generalizable framework that couples pairs of models using machine learning and in situ feedback. We expand upon the massively parallel Multiscale Machine-Learned Modeling Infrastructure (MuMMI), a recent, award-winning workflow, and generalize the framework beyond its original design. We discuss the challenges and learnings in executing a massive multiscale simulation campaign that utilized over 600,000 node hours on Summit and achieved more than 98% GPU occupancy for more than 83% of the time. We present innovations to enable several orders of magnitude scaling, including simultaneously coordinating 24,000 jobs, and managing several TBs of new data per day and over a billion files in total. Finally, we describe the generalizability of our framework and, with an upcoming open-source release, discuss how the presented framework may be used for new applications.

Bhatia, Harsh↗

TChem 2.0

The TChem toolkit is an object-oriented C++ software library that enables numerical simulations using complex chemistry and facilitates the analysis of detailed kinetic models. The toolkit provides capabilities for thermodynamic properties and gas-phase and surface chemistry. The computations rely on the Kokkos framework for parallel execution on many-core and GPU-based platforms. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525. SAND2021-2752 O

Kim, Kyungjoo↗

Partial differential equations preconditioner resilient to soft and hard faults

We present a domain-decomposition-based preconditioner for the solution of partial differential equations (PDEs) that is resilient to both soft and hard faults. The algorithm reformulates the PDE as a sampling problem, followed by a solution update through data manipulation that is resilient to both soft and hard faults. This reformulation allows us to recast the problem as a set of independent tasks, and exploit data locality to reduce global communication. We discuss two different parallel implementations: (a) a single program multiple data (SPMD) version based on a one-to-one mapping between subdomain and MPI processes responsible for both state and computation; and (b) an asynchronous server–client implementation where all state information is held by the servers and clients are designed solely as computational units. We present a scalability comparison of both implementations under nominal conditions, showing efficiency within ~80% for up to 12,000 cores. We present a resilience analysis under different fault scenarios based on the server–client implementation. This framework provides resiliency to hard faults such that if a client crashes, it stops asking for work, and the servers simply distribute the work among all of the other clients alive. Erroneous subdomain solves (e.g. due to soft faults) appear as corrupted data, which is either rejected if that causes a task to fail, or is seamlessly filtered out during the regression stage through a suitable noise model. Three different types of faults are modeled: hard faults modeling nodes (or clients) crashing; soft faults occurring during the communication of the tasks between server and clients; and soft faults occurring during task execution. We demonstrate the resiliency of the approach for a 2D elliptic PDE, and explore the effect of the faults at various failure rates.

97 MATHEMATICS AND COMPUTING↗

Framework Application for Core Edge Transport Simulations (FACETS) Performance Analysis (Final Report)

The goal of the FACETS project (Framework Application for Core-Edge Transport Simulations) was to provide a multiphysics, parallel framework application (FACETS) that will enable whole-device modeling for the U.S. fusion program, to provide the modeling infrastructure needed for ITER, the next step fusion confinement device. Through use of modern computational methods, including component technology and object oriented design, FACETS is able to switch from one model to another for a given aspect of the physics in a flexible manner. This enables use of simplified models for rapid turnaround or high-fidelity models that can take advantage of the largest supercomputer hardware. FACETS does so in a heterogeneous parallel context, where different parts of the application execute in parallel by utilizing task farming, domain decomposition, and/or pipelining as needed and applicable. ParaTools, Inc. was tasked with supporting the performance analysis and tuning of the FACETS components and framework in order to achieve the parallel scaling goals of the project. The TAU Performance System® was used for instrumentation, measurement, archiving, and profile / tracing analysis. ParaTools, Inc. also assisted in FACETS performance engineering efforts. Through the use of the TAU Performance System, ParaTools provided instrumentation, measurement, analysis and archival support for the FACETS project. Performance optimization of key components has yielded significant performance speedups. TAU was integrated into the FACETS build for both the full coupled application and the UEDGE component. The performance database provided archival storage of the performance regression testing data generated by the project, and helped to track improvements in the software development.

97 MATHEMATICS AND COMPUTING↗

Coupled Bison/Thermochimica Simulation of Hydride Formation in Zircalloy Cladding

The report describes implementation of the thermodynamics models for simulation of zirconium hydride formation in LWR fuel cladding. The current models for ZrH x formation are based on empirical rate models that must calibrated to materials and regimes of interest. The thermodynamic approach enables hydride formation calculations based on fundamental thermodynamics principles and material composition. Thermodynamic formulations and models were implemented in the Idaho National Laboratory fuel performance code Bison using Material, Kernel and AuxKernel methods. The results for the Material and Kernel implementation are shown in this report. The Material method implementation naturally fits the concept of modeling evolution of material state in the material models. Given the relatively high cost of thermodynamic computation, the advantage of the AuxKernel compared to Material implementation is the ability to control the execution and computational cost. The implemented formulations are illustrated on example problems which fit the experimental data and satisfy the computational requirements of conservation equations.

36 MATERIALS SCIENCE↗

Dakota A Multilevel Parallel Object-Oriented Framework for Design Optimization Parameter Estimation Uncertainty Quantification and Sensitivity Analysis (V.6.14) (User's Manual)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a users manual for the Dakota software and provides capability overviews and procedures for software execution, as well as a variety of example studies.

97 MATHEMATICS AND COMPUTING↗

Joint LLNL, LANL, SNL, and IRSN High Multiplication Subcritical (Multiplicity) Benchmark Experiments Execution Plan (IER-518 CED-3a)

As part of the experiment design and planning, the critical experiment design team (CEDT), as well as additional stakeholders, convened a series of meetings to discuss the goals and requirements of execution for this experiment. The slides from these meetings are attached in Appendix A. The following sections summarize the outcome of those discussions and present the planned experimental configurations and measurements. The stated goals of this experiment are as follows: 1) Measure time-tagged list-mode data for configurations exceeding neutron multiplication of 100; 2) Provide intercomparison between LLNL, LANL, and IRSN detector systems and methodologies; 3) Generate experiment execution report(s) useful to a fundamental physics benchmark for the ICSBEP; 4) Leverage existing critical experiment and detector system benchmarks to limit required modeling and uncertainty analysis.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Resonant Frequency Control For the PIP-II Injector Test RFQ: Control Framework and Initial Results

For the PIP-II Injector Test (PI-Test) at Fermilab, a four-vane radio frequency quadrupole (RFQ) is designed to accelerate a 30-keV, 1-mA to 10-mA, H- beam to 2.1 MeV under both pulsed and continuous wave (CW) RF operation. The available headroom of the RF amplifiers limits the maximum allowable detuning to 3 kHz, and the detuning is controlled entirely via thermal regulation. Fine control over the detuning, minimal manual intervention, and fast trip recovery is desired. In addition, having active control over both the walls and vanes provides a wider tuning range. For this, we intend to use model predictive control (MPC). To facilitate these objectives, we developed a dedicated control framework that handles higher-level system decisions as well as executes control calculations. It is written in Python in a modular fashion for easy adjustments, readability, and portability. Here we describe the framework and present the first control results for the PI-Test RFQ under pulsed and CW operation.

43 PARTICLE ACCELERATORS↗

Predicting the Impact of Climate Change on U.S. Power Grids and Its Wider Implications on National Security

We discuss our technosocial analytics research and devel-opment on predicting and assessing the impact of climate change on U.S. power-grids and the wider implications for national security. The ongoing efforts extend cutting-edge modeling theories derived from climate, energy, social sciences, and national security domains to form a unified system coupled with an interactive visual interface for technosocial analysis. The goal of the system is to create viable future scenarios that address both technical and social factors involved in the model domains. These scenarios enable policy makers to formulate a coherent, unified strategy towards building a safe and secure society. The paper gives an executive summary of our efforts in the past fiscal year and provides a glimpse of our work planned for the second year of the three-year project being conducted at the Pacific Northwest National Laboratory.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

PRIMA-X - Performance Retargeting of Instrumentation, Measurement, and Analysis Technologies for Exascale Computing

The PRIMA-X (Performance Retargeting of Instrumentation, Measurement, and Analysis Technologies for Exascale Computing) project is the successor of the DOE PRIMA (Performance Refactoring of Instrumentation, Measurement, and Analysis Technologies for Petascale Computing) project, which addressed the challenge of creating a core measurement infrastructure that would serve as a common platform for both integrating leading parallel performance systems (notably TAU and Scalasca) and developing next-generation scalable performance tools. The PRIMA-X project shifts the focus away from refactorization of robust performance tools towards a re-targeting of the parallel performance measurement and analysis architecture for extreme scales. The massive concurrency, asynchronous execution dynamics, hardware heterogeneity, and multi-objective prerequisites (performance, power, resilience) that identify exascale systems introduce fundamental constraints on the ability to carry forward existing performance methodologies. In particular, there must be a deemphasis of per-thread observation techniques to significantly reduce the otherwise unsustainable flood of redundant performance data. Instead, it will be necessary to assimilate multi-level resource observations into macroscopic performance views, from which resilient performance metrics can be attributed to the computational features of the application. This requires a scalable framework for node-level and system-wide monitoring and runtime analyses of dynamic performance information. Also, the interest in optimizing parallelism parameters with respect to performance and energy drives the integration of tool capabilities in the exascale environment further. Initially, PRIMA-X was a collaborative project between the University of Oregon (lead institution) and the German Research School for Simulation Sciences (GRS). Because Prof. Wolf, the PI at GRS, accepted a position as full professor at Technische Universitaet Darmstadt (TU Darmstadt) starting February 1st, 2015, the work of GRS was continued at TU Darmstadt. This report covers the work at TU Darmstadt after the transition. The first main accomplishment of TU Darmstadt is the development of different techniques to aggregate performance data on the level of threads. TU Darmstadt evaluated different schemes designed at GRS during the first phase of this project and integrated them into Score-P, a widely used performance measurement framework for HPC applications. The second main accomplishment is a substantial increase of Score-P’s scalability, achieved by improving its internal representation of the underlying system. Third, we extended our performance-modeling tool Extra-P to automatically create models with more than one input parameter (e.g., process count and input size), which was previously impossible. These models allow the performance behavior of an application to be extrapolated to larger machines. Specific applications of the new multi-parameter capability realized in this project include modeling the isoefficiency function of task-based applications and a novel co-design methodology. Further extensions of Extra-P comprise online modeling functionality, automatic refinement of the model search space, and segmented performance modeling. Further contributions include new locality metrics for both shared- and distributed-memory applications, a new way of measuring the impact of inter-application inference on the execution time of an application, and finally an automatic method that selects appropriate OpenMP constructs for the parallelization of sequential code and classifies the affected variables based on their sharing semantics.

97 MATHEMATICS AND COMPUTING↗

Characterizing and Modeling Cloud Applications/Jobs on a Google Data Center

In this paper, we characterize and model Google applications and jobs, based on a 1-month Google trace from a large-scale Google data center. We address four contributions: (1) we compute the valuable statistics about task events and resource utilization for Google applications, based on various types of resources and execution types; (2) we analyze the classification of applications via a K-means clustering algorithm with optimized number of sets, based on task events and resource usage; (3) we study the correlation of Google application properties and running features (e.g., job priority and scheduling class); (4) we finally build a model that can simulate Google jobs/tasks and dynamic events, in accordance with Google trace. Experiments show that the tasks simulated based on our model exhibit fairly analogous features with those in Google trace. 95+ % of tasks' simulation errors are 20 %, confirming a high accuracy of our simulation model.

Di, Sheng↗