Nuclear Science User Facilities High Performance Computing: Artificial Intelligence and Machine Learning Software Training for Researchers
Artificial Intelligence and Machine Learning Software Training for Researchers
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Artificial Intelligence and Machine Learning Software Training for Researchers
A presentation on serving machine learning models in a production environment
A powerpoint presentation providing information on Integrating Conversational Artificial Intelligence Systems into Science Gateways
Explore the source record for details and available documents.
NREL is tackling the most pressing data center challenges - energy consumption, resilience, resource adequacy, and energy source systems integration - through cutting-edge research and collaborative development.
The NRDS Portal is a login-based data storage solution and science data gateway for researchers to centralize and analyze data before publication. Users can upload, edit, review, and approve their own datasets within the site to eventually be published for public use on the main NRDS site. More development was needed to extend NRDS Portal with new Artificial Intelligence features.
In FY21, INL HPC capabilities were utilized by a diverse set of computing and applied researchers, for a wide range of research and engineering activities. This report focuses on current INL HPC systems and utilization from October 1, 2020 through September 30, 2021.
INL HPC team provided updates related to the INL HPC program and HPC systems for HPC User Group Quarterly Review - September 11, 2024
Containers are seeing widespread use in the world of High Performance Computing, with many HPC Centers either providing their own containerization solution or adopting existing ones like Singularity and Apptainer. The demand for containerization options come from users who want to take advantage of the portability and reproducibility containers can provide, as well as being able to build and use applications that are only distributed in container form or are otherwise unsuited to natively run in an HPC environment. The users served by the Oak Ridge Leadership Computing Facility are no exception. We go over the past and current containerization offerings at the Oak Ridge Leadership Computing Facility, mainly focusing on the Summit supercomputer. We arrive at using a combination of Podman and Singularity to allow users to build and run containers directly on Summit, without requiring external resources or hardware for any step of the process. We look at a couple of projects running on Summit that greatly benefited from being able to use containers on Summit. And we compare benchmarks running natively and in containers on Summit at different scales, observing minimal performance difference and consistent behavior across all tests.
INL HPC capabilities were utilized by individuals at universities, industry, and government laboratories for a wide range of research and engineering activities between October 1, 2020 and September 30, 2021. The HPC systems themselves experienced minimal downtime over the course of the fiscal year. Most downtime was planned to apply critical security patches to system software. The most serious of the unplanned downtime events occurred due to maintenance on the Kohler backup generator in July. Users were kept notified of all events, both planned and unplanned, via 36 total email notifications in FY-21. The INL HPC systems continue to provide high availability for collaboration and innovation in nuclear energy systems research.
High-performance computing systems rely upon scheduling algorithms to achieve high utilization. These schedulers rely upon user estimates of job resource requirements, such as runtime, to determine optimal scheduling of incoming jobs. These user estimates, however, are prone to error. To mitigate this error, significant research has been directed at providing better estimates of job runtime, usually employing machine learning techniques. These techniques are dependent upon the input features selected. Among the possible features is the primary application used by the job. In a survey of more than 20 papers directed at improving runtime prediction, only four included primary application as an input feature. We focus this investigation specifically on the value of adding primary application as an input feature, and find that it does improve model performance, especially for jobs with longer runtimes, though this improvement varies based on the application used. We recommend further research to determine the cause of this variability as well as an optimal strategy for employing a mixture of models both including and not including primary application as a feature.
High-performance computing systems rely upon scheduling algorithms to achieve high utilization. These schedulers rely upon user estimates of job resource requirements, such as runtime, to determine optimal scheduling of incoming jobs. These user estimates, however, are prone to error. To mitigate this error, significant research has been directed at providing better estimates of job runtime, usually employing machine learning techniques. These techniques are dependent upon the input features selected. Among the possible features is the primary application used by the job. In a survey of more than 20 papers directed at improving runtime prediction, only four included primary application as an input feature. We focus this investigation specifically on the value of adding primary application as an input feature, and find that it does improve model performance, especially for jobs with longer runtimes, though this improvement varies based on the application used. We recommend further research to determine the cause of this variability as well as an optimal strategy for employing a mixture of models both including and not including primary application as a feature.
Cooperative Computing implemented as Cooperative Computing System (CCS) in grid has been proved a considerably reliable technique to execute the tasks with real time constraints in a grid environment. This technique can be applied in many high performance distributed computing applications. HPC has a large number of applications in various fields of physics. One such application in radiation physics is X-ray tomography. X-Ray tomography contains numerous applications in various fields of science, technology and research. As the technology is changing from analog to digital in almost all the scenarios, this paper presents an idea towards the attachment of X-ray tomography assembly to HPC environment so as to obtain the highly reliable optimization.
Can cloud computing infrastructures provide HPC-competitive performance for scientific applications broadly? Despite prolific related literature, this question remains open. Answers are crucial for designing future systems and democratizing high-performance computing. We present a multi-level approach to investigate the performance gap between HPC and cloud computing, isolating different variables that contribute to this gap. Our experiments are divided into (i) hardware and system microbenchmarks and (ii) user application proxies. The results show that today’s high-end cloud computing can deliver HPC-competitive performance not only for computationally intensive applications, but also for memory- and communication-intensive applications – at least at modest scales – thanks to the high-speed memory systems and interconnects and dedicated batch scheduling now available on some cloud platforms.
This paper discusses the development of, approaches for, experiences with, and some results from a large-scale, high-performance-computer-based (HPC-based) co-simulation of electric power transmission and distribution systems using the Integrated Grid Modeling System (IGMS). IGMS was developed at the National Renewable Energy Laboratory (NREL) as a novel Independent System Operator (ISO)-to-appliance scale electric power system modeling platform that combines off-the-shelf tools to simultaneously model 100s to 1000s of distribution systems in co-simulation with detailed ISO markets, transmission power flows, and AGC-level reserve deployment. Lessons learned from the co-simulation architecture development are shared, along with a case study that explores the reactive power impacts of PV inverter voltage support on the bulk power system.
The Simulation EnviRonment for Geomorphology, Hydrodynamics, and Ecohydrology in Integrated form (SERGHEI) is a multi-dimensional, multi-domain, and multi-physics model framework for environmental and landscape simulation, designed with an outlook towards Earth system modelling. At the core of SERGHEI's innovation is its performance-portable high-performance parallel-computing (HPC) implementation, built from scratch on the Kokkos portability layer, allowing SERGHEI to be deployed, in a performance-portable fashion, in graphics processing unit (GPU)-based heterogeneous systems. In this work, we explore combinations of MPI and Kokkos using OpenMP and CUDA backends. In this contribution, we introduce the SERGHEI model framework and present with detail its first operational module for solving shallow-water equations (SERGHEI-SWE) and its HPC implementation. This module is designed to be applicable to hydrological and environmental problems including flooding and runoff generation, with an outlook towards Earth system modelling. Its applicability is demonstrated by testing several well-known benchmarks and large-scale problems, for which SERGHEI-SWE achieves excellent results for the different types of shallow-water problems. Finally, SERGHEI-SWE scalability and performance portability is demonstrated and evaluated on several TOP500 HPC systems, with very good scaling in the range of over 20 000 CPUs and up to 256 state-of-the art GPUs.
Explore the source record for details and available documents.