High Performance Computing Data Center (Fact Sheet)
This two-page fact sheet describes the new High Performance Computing Data Center in the ESIF and talks about some of the capabilities and unique features of the center.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This two-page fact sheet describes the new High Performance Computing Data Center in the ESIF and talks about some of the capabilities and unique features of the center.
This paper presents the development of a new entropy-based feature selection method for identifying and quantifying impacts. Here, impacts are defined as statistically significant differences in spatio-temporal fields when comparing datasets with and without an external forcing in an Earth system model. Temporal feature selection is performed by first computing the cross-fuzzy entropy to quantify similarity of patterns between two datasets and then applying changepoint detection to identify regions of statistically constant entropy. The method is used to capture temperate north surface cooling from a 9-member simulation ensemble of the Mt. Pinatubo volcanic eruption, which injected 10 Tg of SO 2 into the stratosphere. The results estimate a mean difference decrease in near surface air temperature of -0.560 K with a 99% confidence interval between -0.864 K and -0.257 K between April and November of 1992, one year following the eruption. A sensitivity analysis with decreasing SO 2 injection revealed that the impact is statistically significant at 5 Tg but not at 3 Tg. Using identified features, a dependency graph model based on a 9-day lag had significantly fewer nodes than a graph based on monthly means. Furthermore, this demonstrates our method’s ability to perform dimension reduction while still uncovering source-to-impact pathways.
BigPanDA monitoring is a web application that provides various processing and representation of the Production and Distributed Analysis (PanDA) system objects states. Analysing hundreds of millions of computation entities, such as an event or a job, BigPanDA monitoring builds different scales and levels of abstraction reports in real time mode. Provided information allows users to drill down into the reason of a concrete event failure or observe the broad picture such as tracking the computation nucleus and satellites performance or the progress of a whole production campaign. PanDA system was originally developed for the ATLAS experiment. Currently, it manages execution of more than 2 million jobs distributed over 170 computing centers worldwide on daily basis. BigPanDA is its core component commissioned in the middle of 2014 and now is the primary source of information for ATLAS users about the state of their computations and the source of decision support information for shifters, operators and managers. In this work, we describe the evolution of the architecture, current status and plans for the development of the BigPanDA monitoring.
GPUs have become part of the mainstream high performance computing facilities that increasingly require more computational power to simulate physical phenomena quickly and accurately. However, GPU nodes also consume significantly more power than traditional CPU nodes, and high power consumption introduces new system operation challenges, including increased temperature, power/cooling cost, and lower system reliability. This paper explores how power consumption and temperature characteristics affect reliability, provides insights into what are the implications of such understanding, and how to exploit these insights toward predicting GPU errors using neural networks.
A method and system for administering an interactive computer network and more particularly to performing security in a network of interactive electronic components, such as a computer network using a currency-based transactional economy model where scarcity serves as a limiting factor to accessing and engaging in various activities within the system.
Ongoing design activities at Argonne National Laboratory are requiring a thorough verification of the Argonne Reactor Computation codes be performed. REBUS is central to this system. The driver for this effort requires the Triangular-Z and hexagonal-Z core geometry options of REBUS to be verified. Previous work identified the REBUS features required to be verified to support current design activities, features of which are generally applicable to hexagonal-Z fast reactor designs. The scope of this verification effort includes verifying REBUS’s ability to correctly intepret the user input model, verifying that the features identified yield the intended results, and verifying the correctness of the REBUS output tables. The REBUS software verification relies heavily upon the accuracy of the embedded DIF3D software, the verification of which was completed and documented elsewhere. Given that DIF3D produces an accurate solution, the primary focus of the verification in the REBUS software is to ensure that it properly uses the DIF3D solution and that the depletion system (Bateman equations) are correctly implemented. This manuscript reiterates the verification tasks and displays results with respect to the features needed for current design activities. Analytic solutions of the Batemen equations are displayed and the results calculated with REBUS are displayed demonstrating the accuracy. Since coupled Bateman and neutron diffusion/transport solutions are extremely difficult to obtain, much of the focus is placed on how REBUS uses a given DIF3D solution assuming the accuracy of the DIF3D solution. The verification effort identified no issues that are debilitating or otherwise impactful to the design usage of REBUS, and thus REBUS version 11.0, release 3012 is considered verified. It is important to note that several outputs of REBUS are identified to be inaccurate, such as burnup in MWD/MT. Most of the relevant ones for VTR are generally accurate with 10-20% errors which is not impactful as all regular REBUS users are aware of this issue and know how to hand calculate the results. The REBUS manual further makes it clear that these values are consistent with the methodology being used by REBUS and thus the “errors” are more of an inconsistent definition with respect to what a user would expect given a definition in literature. Other issues that were identified included unclear documentation and software bugs all of which were inconsequential to the final results.
Ongoing design activities at Argonne National Laboratory are requiring a thorough verification of the Argonne Reactor Computation codes be performed. DIF3D is central to this system. The driver for this effort requires the 3D Cartesian, triangular-Z, and hexagonal-Z core geometry options of DIF3D be verified. Previous work identified the DIF3D features required to be verified to support current design activities, features of which are generally applicable to hexagonal-Z fast reactor designs. The scope of this verification effort includes verifying DIF3D’s ability to correctly translate the user’s model in to DIF3D’s preferred format, verifying that options planned for use have the desired effect, and verifying the correctness of the eigenvalue, fixed-source, forward, and adjoint solvers in DIF3D-FD and DIF3D-VARIANT. This manuscript provides the verification tasks and their results with respect to the features needed for current design activities. Since analytic solutions of the neutron diffusion and transport equations are either limited in scope or not possible, multiple tiers of problems unique to each solver and geometry type were implemented. Each of these tiers tests features independent and complementary arguments for why the separate testing of functionalities is acceptable. Finally, this separate testing was also supplemented with a high-level integral check of each the diffusion and transport capabilities and applicable geometries. To accommodate cases which an analytic solution is not feasible, MCNP6.2 was relied upon to provide a higher-order reference solution. This therefore required that the capabilities within MCNP6.2 which were relied upon for this work are also verified in this work. No MCNP discrepancies were noted in this effort. Note that the MCNP6.2 verification included in this work does not stand as a full verification of MCNP6.2, but merely verifies the features used in verifying DIF3D. The verification effort identified no issues that are debilitating or otherwise impactful to design usage of DIF3D, and thus DIF3D version 11.0, release 3012 is considered verified. As some additional changes have been made to the ARC software since this point all versions between release 3012 and 3266 can be considered verified as version 3253 was used for all updates in this revision. The types of issues that were identified were predominantly in the areas of: unclear documentation, software bugs which were inconsequential to final results, editing options which were ignored in favor of printing more information than requested, bugs in the outputs of intermediate results, or secondary output binary file information which was not present. While not a bug, this verification report also identified that the algorithm used to evaluate the peak fast flux in a nodal transport solution can be quite unreliable due to the methodology used and the location of the peak within the mesh. The authors of the report therefore recommend the usage of the EvaluateFlux software (distributed with ARC) as a more robust alternative noting that DIF3D will properly notify the user when the peaking values it is providing are potentially incorrect.
Ongoing design activities at Argonne National Laboratory are requiring a thorough verification of the Argonne Reactor Computation codes be performed. DIF3D is central to this system. The driver for this effort requires the 3D Cartesian, triangular-Z, and hexagonal-Z core geometry options of DIF3D be verified. Previous work identified the DIF3D features required to be verified to support current design activities, features of which are generally applicable to hexagonal-Z fast reactor designs. The scope of this verification effort includes verifying DIF3D’s ability to correctly translate the user’s model in to DIF3D’s preferred format, verifying that options planned for use have the desired effect, and verifying the correctness of the eigenvalue, fixed-source, forward, and adjoint solvers in DIF3D-FD and DIF3D-VARIANT. This manuscript provides the verification tasks and their results with respect to the features needed for current design activities. Since analytic solutions of the neutron diffusion and transport equations are either limited in scope or not possible, multiple tiers of problems unique to each solver and geometry type were implemented. Each of these tiers tests features independent and complementary arguments for why the separate testing of functionalities is acceptable. Finally, this separate testing was also supplemented with a high-level integral check of each the diffusion and transport capabilities and applicable geometries. To accommodate cases which an analytic solution is not feasible, MCNP6.2 was relied upon to provide a higher-order reference solution. This therefore required that the capabilities within MCNP6.2 which were relied upon for this work are also verified in this work. No MCNP discrepancies were noted in this effort. Note that the MCNP6.2 verification included in this work does not stand as a full verification of MCNP6.2, but merely verifies the features used in verifying DIF3D. The verification effort identified no issues that are debilitating or otherwise impactful to design usage of DIF3D, and thus DIF3D version 11.0, release 3012 is considered verified. As some additional changes have been made to the ARC software since this point all versions between release 3012 and 3266 can be considered verified as version 3253 was used for all updates in this revision. The types of issues that were identified were predominantly in the areas of: unclear documentation, software bugs which were inconsequential to final results, editing options which were ignored in favor of printing more information than requested, bugs in the outputs of intermediate results, or secondary output binary file information which was not present. While not a bug, this verification report also identified that the algorithm used to evaluate the peak fast flux in a nodal transport solution can be quite unreliable due to the methodology used and the location of the peak within the mesh. The authors of the report therefore recommend the usage of the EvaluateFlux software (distributed with ARC) as a more robust alternative noting that DIF3D will properly notify the user when the peaking values it is providing are potentially incorrect.
To achieve the exaFLOPS performance within a contained power budget, next generation supercomputers will feature hundreds of millions of components operating at low- and near-threshold voltage. As the probability that at least one of these components fails during the execution of an application approaches certainty, it seems unrealistic to expect that any run of a scientific application will not experience some performance faults. We believe that there is need of a new generation of light-weight performance and debugging tools that can be used online even during production runs of parallel applications and that can identify performance anomalies during the application execution. In this work we propose the design and implementation of such a monitoring system.
A number of Department of Energy (DOE) science applications, involving exascale computing systems and large experimental facilities, are expected to generate large volumes of data, in the range of petabytes to exabytes, which will be transported over wide-area networks for the purpose of storage, visualization, and analysis. The objectives of this proposal are to (1) develop and test the component technologies and their synthesis methods to achieve source-to-sink high-performance flows, and (2) develop tools that provide these capabilities through simple interfaces to users and applications. In terms of the former, we propose to develop (1) optimization methods that align and transition multiple storage flows to multiple network flows on multicore, multibus hosts; and (2) edge and long-haul network path realization and maintenance using advanced provisioning methods including OSCARS and OpenFlow. We also propose synthesis methods that combine these individual technologies to compose high-performance flows using a collection of constituent storage-network flows, and realize them across the storage and local network connections as well as long-haul connections. We propose to develop automated user tools that profile the hosts, storage systems, and network connections; compose the source-to-sink complex flows; and set up and maintain the needed network connections.
In this work, networked infrastructures of recursively defined systems composed of discrete cyber and physical components are considered. The components of basic systems at the finest levels can be disrupted by cyber or physical means, and can be reinforced to survive at certain costs. A problem of ensuring the infrastructure performance is formulated as a game between a provider and an attacker, who probabilistically choose components to reinforce and attack, respectively. The disruptions of this infrastructure are characterized using the aggregate failure correlation function that specifies the conditional failure probability of the infrastructure given the failure of an individual system at that level. The survival probabilities of basic systems satisfy simple product-form, first-order differential equations expressed in terms of the multiplier functions. The utility functions of the provider and attacker are composed of the reward and cost terms, both expressed in terms of the component reinforcement and attack probabilities. The Nash equilibrium of this game is characterized, along with the sensitivity functions of the survival probabilities of basic systems that highlight their dependence individually on the cost-benefit terms, the correlation functions, and the multiplier functions. These results are illustrated using simplified models of a distributed cloud servers infrastructure, a 5G data network infrastructure, a high performance computing federation, and a smart energy grid infrastructure.
Georgia Tech (Prime Recipient), the University of Texas at El Paso (UTEP) and the National Energy Technology Laboratory (NETL) investigated a hybrid fuel cell/ gas turbine system concept as a combined heat and hybrid power (CHHP) system for both robust and high power-to-process heat ratio cogeneration. The novelty of the proposed system entailed the distinct, elevated electrical efficiencies it maintains while simultaneously supporting a broad span of heating needs (e.g., supply temperatures) demanded across variable heat loads. The scope included: 1) leveraging a pioneering national lab facility configured for dynamic system operability development of hybrid fuel cell/gas turbine cycles; 2) enabling technology development to adjust and modulate the quality and quantity of thermal supply to bottoming heat loads via novel extreme temperature gas bypass valves. Hybrid fuel cell/gas turbine systems have primarily been reduced-to-practice in a constrained (e.g., initial proof-of-concept) manner and have still demonstrated considerable electrical efficiencies. However, these pre-pilot systems have focused upon electrical efficiencies and electrical power generation as the exclusive energy demand. Such hybrid systems had not been extensively researched or developed for flexible and variable operation consisting of both power and heat demands; however, these variable combined power and heat demands are characteristic of many types of manufacturers such as animal/poultry processing, bakeries and milk/flour/pastry manufacturing, textile mills, and electrochemical processing. Commercially, developing the system into a working combined heat and power system benefits these types of manufacturers by allowing them to meet their power and heat demands at a lower cost, higher efficiency, and/or through onsite generation. Therefore, the technical scope of this project was largely to study and facilitate these hybrid systems as combined heat and hybrid power (CHHP) systems that include dynamic operability for variable heat and power loads and/or grid dynamics for various types of manufacturers. Simulation results were used to predict the performance of the CHHP system and conceptually develop it to achieve desired dynamic operability. Experimentally, the primary goal was to design, manufacture, and experiment upon a high-temperature bypass valve. Experimental data included air mass flow rates through the valve orifice when the valve was changed to variable extent between fully closed and fully open. The experimental data was then used to create a semi-empirical computational model of the bypass valve. Concluded simulation goals for the research included developing computational heat exchanger models for the hybrid system inclusive of the bottoming heat exchanger and the recuperative heat exchanger, and then combining the computational recuperator model with the computational valve model. Afterwards, the computational models were then integrated to predict the dynamic operation of hybrid fuel cell/gas turbine cycles throughout a design space and reporting such. The scope stated in the preceding paragraph was packaged into five specific goals: 1) enabling the simulation of dynamic combined heat and power through the creation of computational, modular heat exchanger models; 2) simulation and exploration of the CHHP system’s performance by integrating the heat exchanger models with the national lab’s pre-existing hybrid system (computational) simulation, but without the recuperator bypass valve concept in order to initially determine how the (baseline) system behaves and can be controlled in order to meet variable heat and power demands; 3) development and initial deployment of the high-temperature recuperator bypass valve technology in order to confirm and characterize the approach; 4) usage of the experimental data for the valve to create a semi-empirical computational model for the bypass valve which could then be combined with the heat exchanger computational models; 5) repeat of the second task of simulating and exploring the system’s performance, but this time including the bypass valve to resolve its efficacy. Tasks were successfully completed, and the general notion of flexibly operating, high electrical efficiency CHHP was further corroborated. Supportive details are provided in the report.
INL HPC team provided updates related to the INL HPC program and HPC systems for HPC User Group Quarterly Review - September 11, 2024
Purpose: To report the first experience on the development of a cloud-based treatment planning system and investigate the performance improvement of dose calculation and treatment plan optimization of the cloud computing platform. Methods: A cloud computing-based radiation treatment planning system (cc-TPS) was developed for clinical treatment planning. Three de-identified clinical head and neck, lung, and prostate cases were used to evaluate the cloud computing platform. The de-identified clinical data were encrypted with 256-bit Advanced Encryption Standard (AES) algorithm. VMAT and IMRT plans were generated for the three de-identified clinical cases to determine the quality of the treatment plans and computational efficiency. All plans generated from the cc-TPS were compared to those obtained with the PC-based TPS (pc-TPS). The performance evaluation of the cc-TPS was quantified as the speedup factors for Monte Carlo (MC) dose calculations and large-scale plan optimizations, as well as the performance ratios (PRs) of the amount of performance improvement compared to the pc-TPS. Results: Speedup factors were improved up to 14.0-fold dependent on the clinical cases and plan types. The computation times for VMAT and IMRT plans with the cc-TPS were reduced by 91.1% and 89.4%, respectively, on average of the clinical cases compared to those with pc-TPS. The PRs were mostly better for VMAT plans (1.0 ≤ PRs ≤ 10.6 for the head and neck case, 1.2 ≤ PRs ≤ 13.3 for lung case, and 1.0 ≤ PRs ≤ 10.3 for prostate cancer cases) than for IMRT plans. The isodose curves of plans on both cc-TPS and pc-TPS were identical for each of the clinical cases. Conclusion: A cloud-based treatment planning has been setup and our results demonstrate the computation efficiency of treatment planning with the cc-TPS can be dramatically improved while maintaining the same plan quality to that obtained with the pc-TPS. This work was supported in part by the National Cancer Institute (1R01 CA133474) and by Leading Foreign Research Institute Recruitment Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and Future Planning (MSIP) (Grant No.2009-00420)
Performance improvements in biomolecular simulations based on molecular dynamics (MD) codes are widely desired. Unfortunately, the factors, which allowed past performance improvements, particularly the microprocessor clock frequencies, are no longer increasing. Hence, novel software and hardware solutions are being explored for accelerating performance of widely used MD codes. In this paper, we describe our efforts on porting, optimizing and tuning of Large-scale Atomic/Molecular Massively Parallel Simulator, a popular MD framework, on heterogeneous architectures: multi-core processors with graphical processing unit (GPU) accelerators. Our implementation is based on accelerating the most computationally expensive non-bonded interaction terms on the GPUs and overlapping the computation on the CPU and GPUs. This functionality is built on top of message passing interface that allows multi-level parallelism to be extracted even at the workstation level with the multi-core CPUs and allows extension of the implementation on GPU-enabled clusters. We hypothesize that the optimal benefit of heterogeneous architectures for applications will come by utilizing all possible resources (for example, CPU-cores and GPU devices on GPU-enabled clusters). Benchmarks for a range of biomolecular system sizes are provided, and an analysis is performed on four generations of NVIDIA's GPU devices. On GPU-enabled Linux clusters, by overlapping and pipelining computation and communication, we observe up to 10-folds application acceleration in multi-core and multi-GPU environments illustrating significant performance improvements. Detailed analysis of the implementation is presented that allows identification of bottlenecks in algorithm, indicating that code optimization and improvements on GPUs could allow microsecond scale simulation throughput on workstations and inexpensive GPU clusters, putting widely desired biologically relevant simulation time-scales within reach of a large user community. In order to systematically optimize simulation throughput and to enable performance prediction, we have developed a parameterized performance model that will allow developers and users to explore the performance potential of future heterogeneous systems for biological simulations.
This paper investigates applying artificial intelligence (AI) algorithms to attitude control system of satellites to optimally tune the controller using high performance computing. This methodology is applied to the Virtual Telescope for X-ray Observation mission, which is a precise formation of two separate spacecraft observing multiple objects in the space in the X-ray domain. The mission is divided into phases based on the instrumentation and the mission goal. To reach an stable precise formation robust to stochastic slew and slew rate (i.e., Euler angles and angular velocities) in a minimal constrained time T , consumed energy of the attitude control system, denoted as E, and root-mean-square state error of attitude control system, denoted as e, are minimized. Monte-Carlo simulation is used for the sensitivity analysis of optimization and designing a controller. Deep neural networks (DNN), Gaussian processes (GP), and support vector regression (SVR) learn this optimization as a surrogate model, while their hyperparameters are optimized in a novel approach. THETA supercomputer at Argonne Leadership Computing Facility (ALCF) is used for optimizing the hyperparameters of DNN. The surrogate model meets the requirements of the mission, and it shows a better performance over the optimization and Monte-Carlo. The optimal DNN can satisfy the mission requirements e and T while reducing E for 90% compared to the other given methods.
Not Available
We describe the operation of a cryogenic instrumentation platform incorporating commercially available field-programmable gate arrays (FPGAs). The functionality of the FPGAs at temperatures approaching 4 K enables signal routing, multiplexing, and complex digital signal processing in close proximity to cooled devices or detectors within the cryostat. The performance of the FPGAs in a cryogenic environment is evaluated, including clock speed, error rates, and power consumption. Although constructed for the purpose of controlling and reading out quantum computing devices with low latency, the instrument is generic enough to be of broad use in a range of cryogenic applications.