SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59

All sources NASA NTRS DOE OSTI

DOE OSTI2016

Accelerating Science Impact through Big Data Workflow Management and Supercomputing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. ATLAS, one of the largest collaborations ever assembled in the the history of science, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. To manage the workflow for all data processing on hundreds of data centers the PanDA (Production and Distributed Analysis)Workload Management System is used. An ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF), is realizing within BigPanDA and megaPanDA projects. These projects are now exploring how PanDA might be used for managing computing jobs that run on supercomputers including OLCF’s Titan and NRC-KI HPC2. The main idea is to reuse, as much as possible, existing components of the PanDA system that are already deployed on the LHC Grid for analysis of physics data. The next generation of PanDA will allow many data-intensive sciences employing a variety of computing platforms to benefit from ATLAS experience and proven tools in highly scalable processing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Statistical relationships across epigenomes using large-scale hierarchical clustering

Recent advances in genomics and sequencing platforms have revolutionized our ability to create immense data sets, particularly for studying epigenetic regulation of gene expression. However, the avalanche of epigenomic data is difficult to parse for biological interpretation given nonlinear complex patterns and relationships. This attractive challenge in epigenomic data lends itself to machine learning for discerning infectivity and susceptibility. In this study, we explore over 3000 epigenomes of uninfected individuals and provide a framework to characterize the relationships among epigenetic modifiers, their modifiers, genetic loci, and specific immune cell types across all chromosomes using hierarchical clustering. Hierarchical clustering of epigenomic data revealed consistent epigenetic patterns across chromosomes, demonstrating that variation due to epigenetic modifiers is greater than variation between cell types. Gene Ontology and KEGG pathway analyses indicated significant enrichment of genes involved in chromatin remodeling, mRNA splicing, immune responses, and the regulation of microRNAs and snoRNAs. Epigenetic modifiers frequently formed biologically relevant clusters, including the cohesin complex, RNA Polymerase II transcription factors, and PRC2 complex members. These clustering behaviors remained consistent across all chromosomes, supported by entropy analysis and high Adjusted Rand Index scores, indicating robust cross-chromosomal similarity. Co-occurrence analysis further revealed specific sets of modifiers that consistently appeared together within clusters, reflecting shared biological functions and interactions. Validation using another dataset confirmed the reproducibility of these clustering patterns and modifier co-occurrence relationships, underscoring the reliability and generalizability of the methodology.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Early Evaluation of Fugaku A64FX Architecture Using Climate Workloads

The Energy Exascale Earth System Model (E3SM) Project is an ongoing, state-of-the-science Earth system modeling, simulation, and prediction project that targets efficient utilization of U.S. Department of Energy’s (DOE) supercomputers to meet the science needs of the nation and the mission needs of DOE. This work focuses on our early evaluation of the A64FX architecture on Fugaku supercomputer using E3SM benchmarks. We will present results that track hardware trends, facilitate architecture comparison and the specific impact on our workload using an atmospheric model benchmark. We have two variants of the code written in Fortran and C++/Kokkos respectively which were used to collect data on a variety of CPU and GPU platforms. Furthermore, we have conducted a comparative evaluation of the compilers on the A64FX architecture and found GNU to be the best performer for our workload. Our experience so far indicates that Fugaku/A64FX shows promising energy efficiency (performance/Watt) with further performance gains possible through architecture-aware optimization efforts.

Sreepathi, Sarat↗

DOE OSTI2019

Graph Algorithms in PGAS: Chapel and UPC++

The Partitioned Global Address Space (PGAS) programming model can be implemented «««< HEAD either with programming language features or with runtime library APIs, each implementation favoring different aspects (e.g., productivity, abstraction, flexibility, or performance). Certain language and runtime features, such as collectives, explicit and asynchronous communication primitives, and constructs facilitating overlap of communication and computation (such as futures and conjoined futures) can enable better performance and scaling for irregular applications, in particular for distributed graph analytics. We compare graph algorithms in one of each of these environments: the Chapel PGAS programming language and the UPC++ PGAS runtime library. We implement algorithms for breadth-first search and triangle counting graph kernels in both environments. We discuss the code in each of the environments, and compile performance data on a Cray Aries and an Infiniband platform. Our results show that the library-based approach of UPC++ currently provides strong performance while Chapel provides a high-level abstraction that, harder to optimize, still provides comparable performance. We argue that various language primitives and runtime library features should be considered in conjunction with each other to design future graph analytics libraries so as to incorporate the best of both worlds.

Distributed Algorithms, Graph algorithms, HPC↗

DOE OSTI2021

Cyber-Physical Event Emulation-Based Transmission-and-Distribution Co-Simulation for Situational Awareness of Grid Anomalies (SAGA)

Energy management of transmission and distribution networks (T&D) is becoming more challenging with the accelerated adoption of distributed energy resources (DERs)-such as distributed photovoltaic generation and battery energy storage systems (BESS)-on the electric grid. To better analyze the impacts of DERs on both transmission and distribution systems, a comprehensive T&D co-simulation platform is developed. Further, with DERs more actively participating in system operation-e.g., by providing real-time grid services-their cyber vulnerability needs to be better understood to maintain system reliability. This paper discusses a cyber-physical events emulation-based T&D co-simulation platform to perform comprehensive cyber events emulations, physical simulation, and analysis of interdependent impacts. Results from the case studies-which show how cyber events on a synthetic distribution network can impact operations on the transmission and distribution network-validate that the proposed T&D cosimulation platform can perform cyber-physical events emulation and produce response in near realtime; therefore, with extensive simulation using the proposed co-simulation platform, the system operators can accumulate adequate training data for system situational awareness of grid anomalies.

anomalies detection↗

DOE OSTI2023

Spy in the GPU-box: Covert and Side Channel Attacks on Multi-GPU System

The deep learning revolution has been enabled in large part by GPUs, and more recently accelerators, which make it possible to carry out computationally demanding training and inference in acceptable times. As the size of machine learning networks and workloads continues to increase, multi-GPU machines have emerged as an important platform offered on High Performance Computing and cloud data centers. Since these machines are shared among multiple users, it becomes increasingly important to protect applications against potential attacks. In this paper, we explore the vulnerability of Nvidia's DGX multi-GPU machines to covert and side channel attacks. These machines consist of a number of discrete GPUs that are interconnected through a combination of custom interconnect (NVLink) and PCIe connections. We reverse engineer the interconnected cache hierarchy and show that it is possible for an attacker on one GPU to cause contention on the L2 cache of another GPU. We use this observation to first develop a covert channel attack across two GPUs, achieving the best bandwidth of around 4 MB/s. We also develop a prime and probe attack on a remote GPU allowing an attacker to recover the cache access pattern of another workload. This access pattern can be used in any number of side channel attacks: we demonstrate a proof of concept attack that fingerprints the application running on the remote GPU, with high accuracy. We also develop a proof of concept attack to extract hyperparameters of a machine learning workload. Our work establishes for the first time the vulnerability of these machines to microarchitectural attacks and can guide future research to improve their security.

Dutta, Sankha↗

DOE OSTI2026

NuMaCo

The Nuclear Material and Composition (NuMaCo) server is a web-based platform for storing, managing, and serving material composition data. It includes the SCALE Standard Composition Library and the PNNL Compendium, along with their associated references and uncertainty data. NuMaCo also provides tools to generate, compare, and validate material input definitions for both SCALE and MCNP.

Skutnik, Steve (000000016441135X)↗

DOE OSTI2021

Historical global Earth System Model simulations with E3SM's dynamic root module

This data package contains global-scale Earth System Model simulations run with DOE's E3SM Model. The package includes one set of simulations run in the default configuration and a second set of simulations run with the dynamic root module enabled. These two datasets can be used to assess the impact that dynamic roots have on Land Surface fluxes such as carbon and water. The outputs are provided here on a 0.5 x 0.5 global grid at monthly resolution. The simulations were run with atmospheric forcing from the Global Soil Wetness Project. We provide specifically here Gross Primary Production, Transpiration, relative root fraction per soil layer as well as Air Temperature, Precipitation and all the parameterizations used for the simulations. Our purpose for generating and analyzing these simulations was to assess the role that root dynamics and foraging for water has in the recovery timescale of ecosystems following climate perturbation. All data provided are in the "netcdf" format using standard CF-1 (Climate and Forecast) Convention. The data are broken up into 20 year chunks to facilitate easier read-in. The data are all machine readable using various software platforms including Matlab, NCO, Panoply and GrADS.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Earth System Model Simulations of Ameriflux sites using dynamic roots and dynamic leaf to root allocation

This data package contains point-scale Earth System Model simulations run with DOE's E3SM Model. The package includes four sets of simulations, one set of simulations run in the default configuration, a second set of simulations run with the dynamic root module enabled, the third set with dynamic leaf to fine root allocation, and the fourth with the combination of dynamic roots and dynamic allocation. These simulations were run in point mode a Ameriflux sites, US-MMS, US-Moz, US-Syv, and US-NR1. These datasets can be used to assess the impact that dynamic roots have on Land Surface fluxes such as carbon and water. The simulations were run with atmospheric forcing from the Global Soil Wetness Project. We provide specifically here Gross Primary Production, Transpiration, relative root fraction per soil layer as well as Air Temperature, Precipitation and all the parameterizations used for the simulations. Our purpose for generating and analyzing these simulations was to assess the role that root dynamics and foraging for water has in the recovery timescale of ecosystems following climate perturbation. All data provided are in the "netcdf" format using standard CF-1 (Climate and Forecast) Convention. The data are all machine readable using various software platforms including Matlab, NCO, Panoply and R.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

Geocube

NOTICE: Geocube has been replaced by EDX Spatial, a modern geospatial platform to map, visualize, query and explore spatial data. Online web mapping tool for visualization and simple analysis of Earth-energy data files from public and DOE related sources. Geocube allows users to upload and visualize their own datasets but also comes preloaded with individual spatial datasets as well as spatial data collections that align to topical themes.

carbon storage↗

DOE OSTI2023

Extreme waves in present and future climates using physics-based synthetic tropical cyclones in the Gulf of Mexico.

Historically, extreme waves have been the most destructive force in the Gulf of Mexico, destroying and damaging thousands of offshore structures. The American Petroleum Institute has revised its design criteria for offshore platforms several times over the decades, as new data continues to become available, showing that data for our present climate is insufficient. This problem is exacerbated in a changing climate. By using synthetic physics-based tropical cyclones to force a third-generation wave model, we can generate adequate statistics for present and future climates. These simulations have been completed using Joule Supercomputer.

Climate Change↗

DOE OSTI2026

Towards continual machine learning for particle accelerators

This talk covers our work on errant beam prognostics at the Spallation Neutron Source (SNS), focusing on the end-to-end process from data collection to the development and deployment of predictive models in specific. A short overview of AIML work done for accelerators and current trends will be presented. We will walk through key steps involved in creating robust Machine Learning (ML) models, including model training, validation, and deployment in an operational setting. In addition to presenting our technical approach, we will share valuable lessons learned, emphasizing the importance of infrastructure to support the continuous adaptation of models to evolving data and system behaviors. This talk will provide insights into the challenges and solutions involved in applying ML to real-world operational environments, with a particular focus on managing data drift and changes in accelerator setup while ensuring model resilience over time.

Accelerator Physics↗

DOE OSTI2012

The Portals 4.0 Network Programming Interface

This report presents a specification for the Portals 4.0 network programming interface. Portals 4.0 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4.0 is well suited to massively parallel processing and embedded systems. Portals 4.0 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia’s Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4.0 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities.

Barrett, Brian W.↗

DOE OSTI2013

The portals 4.0.1 network programming interface.

This report presents a specification for the Portals 4.0 network programming interface. Portals 4.0 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4.0 is well suited to massively parallel processing and embedded systems. Portals 4.0 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia s Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4.0 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities. 3

Barrett, Brian W.↗

DOE OSTI2018

The Portals 4.2 Network Programming Interface

This report presents a specification for the Portals 4 network programming interface. Portals 4 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded systems. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

NIF Monthly Highlights for July 2019

July was a productive month for scientists at the National Ignition Facility, with a significant number of target shots performed, including many developing new platforms and taking first-of-their-kind data. The HED program, the ICF program, the National Security Applications Program and the Discovery Science users each achieved significant results. The facility itself also made tremendous progress towards commissioning several new capabilities for laser diagnostics, and target-physics diagnostic systems that measure x-ray and neutron data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2014

The Portals 4.0.2 Networking Programming Interface

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2013

Veterans Commissioning Training Program for Commercial Healthcare Facilities [Final Report]

This project was designed to develop a building energy training curriculum, with hopes for creating immediate, long-term job opportunities for returning Iraq and Afghanistan veterans as energy commissioning agents within both public and private sectors. As part of this proposed effort, an online version of the curricula were developed in part using a modified version of the Association of Energy Engineers (AEE) Certified Building Commissioning Professional program. Case studies utilizing building information modeling (BIM) data from actual retrocommissioning projects and 3D gaming platform were developed as part of the curriculum. In addition to using materials from existing literature, the PIs also conducted scientific-based research on energy retrofitting related to building envelopes using numerical simulation approach. And the research findings and recommendation were incorporated in the developed curriculum. Currently, there are a lack of scientific-based guidance on envelope retrofitting practice. Many retrofitting practices were based on experiences, and are limited by the lack of scientific evidence. Existing retrofitting training material have limitations due to this missing piece. We have succeeded in producing some scientific evidence to guide and provide solutions to some building envelope design and retrofitting. The results from some of our research using computational fluid dynamics have demonstrated the feasibility of using our recommended solutions. Research results and findings were disseminated to the research and industry communities through conference papers (Appendix B) and presentations (Appendix C), and some results and findings were incorporated into the developed training curriculum (Appendix A).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗