Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data layout”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

zorder-lib: Library API for Z-Order Memory Layout

This document describes the motivation for, elements of, and use of the zorder-lib, a library API that implements organization of and access to data in memory using either a-order (also known as “row-major” order) or z-order memory layouts. The primary motivation for this work is to improve the performance of many types of data-intensive codes by increasing both spatial and temporal locality of memory accesses. The basic idea is that the cost associated with accessing a datum is less when it is nearby in either space or time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Transparent in Situ Data Transformations in ADIOS

Though an abundance of novel "data transformation" technologies have been developed (such as compression, level-of-detail, layout optimization, and indexing), there remains a notable gap in the adoption of such services by scientific applications. In response, we develop an in situ data transformation framework in the ADIOS I/O middleware with a "plug in" interface, thus greatly simplifying both the deployment and use of data transform services in scientific applications. Our approach ensures user-transparency, runtime-configurability, compatibility with existing I/O optimizations, and the potential for exploiting read-optimizing transforms (such as level-of-detail) to achieve I/O reduction. We demonstrate use of our framework with the QLG simulation at up to 8,192 cores on the leadership-class Titan supercomputer, showing negligible overhead. We also explore the read performance implications of data transforms with respect to parameters such as chunk size, access pattern, and the "opacity" of different transform methods including compression and level-of-detail.

Computer Science↗
DOE OSTI2022

Availability and redundancy design considerations for Electron-Ion Collider cooling water systems and components

The Electron-Ion Collider (EIC) is composed of several large-scale systems, which all require highly available cooling systems based on the Conceptual Design Report. Since the overall EIC availability goal is 85%, all the subsystems will contribute. Availability is defined by the time that a component or system is functional given a required or scheduled run time. The focus of the project was to determine if redundant or extra cooling water and deionized water pumps were needed in a subsystem to increase the overall EIC availability. The largest subsystem in the ring is the 10 o’clock cooling system, so this was used for redundancy analysis. Relationships between the components were defined as parallel, series, or m-out-of-n for the cooling towers, valves, pumps and heat exchangers to perform calculations. A comparison between an original system versus a redundant one was then done using component availability from industry standards. Cost analysis was also performed to see how much the availability and cost would decrease if the redundancy application was not chosen. The Collider-Accelerator Department (CAD) also has a calculation in place for the Relativistic-Heavy Ion Collider (RHIC), so cooling water failure data was collected from their operational log and analyzed for real-life component availability. A comparison of component availability from the CAD data to the industry standards was made through the 10 o’clock cooling system layout. The results showed that the subsystems need to have over a 98% availability for an 85% goal, and that the CAD data is very close to the industry component standards. The cost differential is miniscule compared to the potential availability increase. Therefore, redundant systems can increase the overall EIC availability by a sizable amount and are worth implementing.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Flow Redirection and Induction in Steady State (FLORIS) Wind Plant Power Production Data Sets

This dataset contains turbine- and plant-level power outputs for 252,500 cases of diverse wind plant layouts operating under a wide range of yawing and atmospheric conditions. The power outputs were computed using the Gaussian wake model in NREL's FLOw Redirection and Induction in Steady State (FLORIS) model, version 2.3.0. The 252,500 cases include 500 unique wind plants generated randomly by a specialized Plant Layout Generator (PLayGen) that samples randomized realizations of wind plant layouts from one of four canonical configurations: (i) cluster, (ii) single string, (iii) multiple string, (iv) parallel string. Other wind plant layout parameters were also randomly sampled, including the number of turbines (25-200) and the mean turbine spacing (3D-10D, where D denotes the turbine rotor diameter). For each layout, 500 different sets of atmospheric conditions were randomly sampled. These include wind speed in 0-25 m/s, wind direction in 0 deg.-360 deg., and turbulence intensity chosen from low (6%), medium (8%), and high (10%). For each atmospheric inflow scenario, the individual turbine yaw angles were randomly sampled from a one-sided truncated Gaussian on the interval 0 deg.-30 deg. oriented relative to wind inflow direction. This random data is supplemented with a collection of yaw-optimized samples where FLORIS was used to determine turbine yaw angles that maximize power production for the entire plant. To generate this data, a subset of cases were selected (50 atmospheric conditions from 50 layouts each for a total of additional 2,500 cases) for which FLORIS was re-run with wake steering control optimization. The IEA onshore reference turbine, which has a 130 m rotor diameter, a 110 m hub height, and a rated power capacity of 3.4 MW was used as the turbine for all simulations. The simulations were performed using NREL's Eagle high performance computing system in February 2021 as part of the Spatial Analysis for Wind Technology Development project funded by the U.S. Department of Energy Wind Energy Technologies Office. The data was collected, reformatted, and preprocessed for this OEDI submission in May 2023 under the Foundational AI for Wind Energy project funded by the U.S. Department of Energy Wind Energy Technologies Office. This dataset is intended to serve as a benchmark against which new artificial intelligence (AI) or machine learning (ML) tools may be tested. Baseline AI/ML methods for analyzing this dataset have been implemented, and a link to their repository containing those models has been provided. The .h5 data file structure can be found in the GitHub repository under explore_wind_plant_data_h5.ipynb.

AI↗
DOE OSTI2025

Data Structure Alchemy

In an increasingly more data-driven world, the project set out to uncover the first principles of data-structure design, chart the immense design space they form, and build automation that can synthesize an optimal structure, or even a whole storage engine, for any given workload, hardware platform, and cost target. Data structures are at the center of every computational system and are directly responsible for its performance. Two core technical thrusts were defined: 1) Mapping design spaces for key data-centric abstractions (filters, hash functions, storage-engine layouts, neural-network topologies, blockchain protocols, image layouts, etc.). 2) Developing search & synthesis algorithms, initially analytical cost models, later neural-guided bi-level optimisers that navigate sextillions of candidate designs in seconds and materialise the best one as ready‐to-run code. This report distills the key insights, accomplishments, and impact.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

DIRAQ: scalable in situ data- and resource-aware indexing for optimized query performance

Scientific data analytics in high-performance computing environments has been evolving along with the advancement of computing capabilities. With the onset of exascale computing, the increasing gap between compute performance and I/O bandwidth has rendered the traditional post-simulation processing a tedious process. Despite the challenges due to increased data production, there exists an opportunity to benefit from "cheap" computing power to perform query-driven exploration and visualization during simulation time. To accelerate such analyses, applications traditionally augment, post-simulation, raw data with large indexes, which are then repeatedly utilized for data exploration. However, the generation of current state-of-the-art indexes involves a compute- and memory-intensive processing, thus rendering them inapplicable in an in situ context. In this paper we propose DIRAQ, a parallel in situ, in network data encoding and reorganization technique that enables the transformation of simulation output into a query-efficient form, with negligible runtime overhead to the simulation run. DIRAQ's effective core-local, precision-based encoding approach incorporates an embedded compressed index that is 3---6 $$\times $$ ? smaller than current state-of-the-art indexing schemes. Its data-aware index adjustmentation improves performance of group-level index layout creation by up to 35 % and reduces the size of the generated index by up to 27 %. Moreover, DIRAQ's in network index merging strategy enables the creation of aggregated indexes that speed up spatial-context query responses by up to $$10\times $$ 10 ? versus alternative techniques. DIRAQ's topology-, data-, and memory-aware aggregation strategy results in efficient I/O and yields overall end-to-end encoding and I/O time that is less than that required to write the raw data with MPI collective I/O.

Computer Science↗
DOE OSTI2019

Comparison of Wind Farm Layout Optimization Results Using a Simple Wake Model and Gradient-Based Optimization to Large-Eddy Simulations: Preprint

Comparison of wind farm layout optimization results with simulations or wind tunnel tests is important because the models used during optimization use simplifying assumptions that can alter the design space. Some of the characteristics of the simple models may negatively influence the resulting layouts. In this paper we perform wind farm layout optimization and compare the resulting improvements to large eddy simulation (LES) results. We begin by describing the models used, including changes specific for use with gradient-based optimization. We then compare our version of the models with the original versions and large eddy simulation LES) data from previous publications. Using the models described, we performed gradient-based wind farm layout optimization, using exact gradients, and compare the results to LES simulations. The models predicted an improvement in annual energy production (AEP) of 7.4%, while the LES reported an AEP improvement of 9:9%. We conclude that the improvements found by optimizing with the simple models are not just an artifact of the models, but are real improvements.

17 WIND ENERGY↗
DOE OSTI2021

Cross-Scale Land-Atmosphere Experiment Field Campaign Report

The experimental setup was installed in 2017. It is located adjacent to the world’s largest and most extensive climate research facility, the Southern Great Plains (SGP) atmospheric observatory, which was established by the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) user facility. This site was selected due to the abundance of environmental data that can support the proposed project. The location and layout of the experimental setup are presented in Figure 1. The site extends through two agricultural fields separated by the Coal Road. One agricultural field (hereafter called “the homogeneous site”) contains a relatively homogeneous soil composition and a homogeneous alfalfa vegetation cover. The other agricultural field (hereafter called “the heterogeneous site”) extends through two different soil compositions and is covered by two vegetation covers, which are different in type and coverage ratio.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

PDF Entity Annotation Tool (PEAT)

While different text mining approaches – including the use of Artificial Intelligence (AI) and other machine based methods - continue to expand at a rapid pace, the tools used by researchers to create the labeled datasets required for training, modeling, and evaluation remain rudimentary. Labeled datasets contain the target attributes the machine is going to learn; for example, training an algorithm to delineate between images of a car or truck would generally require a set of images with a quantitative description of the underlying features of each vehicle type. Development of labeled textual data that can be used to build natural language machine learning models for scientific literature is not currently integrated into existing manual workflows used by domain experts. Published literature is rich with important information, such as different types of embedded text, plots, and tables that can all be used as inputs to train ML/natural language processing (NLP) models, when extracted and prepared in machine readable formats. Currently, both normalized data extraction of use to domain experts and extraction to support development of ML/NLP models are labor intensive and cumbersome manual processes. Automatic extraction of data and information from formats such as PDFs that are optimized for layout and human readability, not machine readability. The PDF (Portable Document Format) Entity Annotation Tool (PEAT) was developed with the goal of allowing users to annotate publications within their current print format, while also allowing those annotations to be captured in a machine-readable format. One of the main issues with traditional annotation tools is that they require transforming the PDF into plain text to facilitate the annotation process. While doing so lessens the technical challenges of annotating data, the user loses all structure and provenance that was inherent in the underlying PDF. Also, textual data extraction from PDFs can be an error prone process. Challenges include identifying sequential blocks of text and a multitude of document formats (multiple columns, font encodings, etc.). As a result of these challenges, using existing tools for development of NLP/ML models directly from PDFs is difficult because the generated outputs are not interoperable. We created a system that allows annotations to be completed on the original PDF document structure, with no plain text extraction. The result is an application that allows for easier and more accurate annotations. In addition, by including a feature that grants the user the ability to easily create a schema, we have developed a system that can be used to annotate text for different domain-centric schemas of relevance to subject matter experts. Different knowledge domains require distinct schemas and annotation tags to support machine learning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

EBR-II Data Digitization

1. Objectives To produce a validation database out of those recorded signals it will be necessary also to identify the documents need to reconstruct the status of reactor at the time of the beginning of the recordings. This should comprehends the core loading specification (assemblies type and location and burn-up) along with this data the assemblies drawings and the core drawings will be identified. The first task of the project will be identify the location of the sensors, with respect the reactor plant layout, and the physical quantities recorded by the Experimental Breeder Reactor-II (EBR-II) data acquisition system. This first task will allow guiding and prioritizing the selection of drawings needed to numerically reproduce those signals. 1.1 Scopes and Deliverables The deliverables of this project are the list of sensors in EBR-II system, the identification of storing location of those sensors, identification of a core isotopic composition at the moment of the start of system recording. Information of the sensors in EBR-II reactor system was summarized from the EBR-II system design descriptions listed in Section 1.2.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2016

Optimizing End-to-End Big Data Transfers over Terabits Network Infrastructure

While future terabit networks hold the promise of significantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today's 100 gigabit networks to realize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink, for instance, the data storage infrastructure at both the source and sink and its interplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this study, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environment, and we present a new bulk data movement framework for terabit networks, called LADS. LADS exploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to benefit from hardware-level zero-copy, and operating system bypass capabilities when available. It can further improve data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared storage resource, improving input/output bandwidth, and data transfer rates across the high speed networks. We also investigate the performance degradation problems of LADS due to I/O contention on the parallel file system (PFS), when multiple LADS tools share the PFS. We design and evaluate a meta-scheduler to coordinate multiple I/O streams while sharing the PFS, to minimize the I/O contention on the PFS. Finally, with our evaluations, we observe that LADS with meta-scheduling can further improve the performance by up to 14 percent relative to LADS without meta-scheduling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Regen: An object layout regenerator on large-scale production HPC systems

This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects and regenerates or preallocates new object layouts according to the identified access patterns. Finally, Regen removes or reuses the obsolete or regenerated object layouts as necessary. As a result, Regen accelerates access to objects by providing a flexible object layout. We implement Regen as a framework on top of Proactive Data Container (PDC) and evaluate it on Cori supercomputer, a production-scale HPC system, by using realistic HPC I/O benchmarks. The experimental results show that Regen improves the I/O performance by up to 16.92 × compared with an existing system.

Distributed file system↗
DOE OSTI2018

Models of exotic interferometer cross-correlations in emergent space-time

Here, a Lorentz invariant framework is developed to model the cross spectrum of two interferometers in a space-time that emerges from a Planck scale quantum system with exact causal symmetry and holographic spacelike rotational correlations. Space-time relationships between world lines are generated by entanglement of geometrical states on causal diamonds. The entanglement is tied to a unique observable signature: an exotic imaginary broad band cross spectrum, with a frequency structure determined by the layout of the interferometers. The models will be used to interpret data from the reconfigured Fermilab Holometer, and for conceptual design of future experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2018

GPU Data Access on Complex Geometries for D3Q19 Lattice Boltzmann Method

GPU performance of the lattice Boltzmann method (LBM) depends heavily on memory access patterns. When LBM is advanced with GPUs on complex computational domains, geometric data is typically accessed indirectly, and lattice data is typically accessed lexicographically in the Structure of Array (SoA) layout. Although there are a variety of existing access patterns beyond the typical choices, no study has yet examined the relative efficacy between them. Here, we compare a suite of memory access schemes via empirical testing and performance modeling. We find strong evidence that semi-direct addressing is the superior addressing scheme for the majority of cases examined: Semi-direct addressing increases computational speed and often reduces memory consumption. For lattice layout, we find that the Collected Structure of Arrays (CSoA) layout outperforms the SoA layout. When compared to state-of-the-art practices, our recommended addressing modifications lead to performance gains between 10-40% across different complex geometries, fluid volume fractions, and resolutions. The modifications also lead to a decrease in memory consumption by as much as 17%. Having discovered these improvements, we examine a highly resolved arterial geometry on a leadership class system. On this system we present the first near-optimal strong results for LBM with arterial geometries run on GPUs. We also demonstrate that the above recommendations remain valid for large scale, many device simulations, which leads to an increased computational speed and average memory usage reductions. To understand these observations, we employ performance modeling which reveals that semi-direct methods outperform indirect methods due to a reduced number of total loads/stores in memory, and that CSoA outperforms SoA and bundling due to improved caching behavior.

Herschlag, Gregory↗
DOE OSTI2016

The control room radiological habitability assessment of high temperature reactor pebble bed module (HTR-PM) - 18465

The control room radiological habitability is important for staff safety in nuclear power plant accident, which is also a licensing requirement of the High Temperature Reactor Pebble Bed Module (HTR-PM) in China. In this study, a comprehensive evaluation of the control room radiological habitability is performed for three design basis accident scenario of HTR-PM. For each scenario, three exposure pathways are taken into account, which includes the immersion external exposure, the inhalation exposure and the direct exposure from radioactive cloud in the environment. The atmospheric concentration is calculated using the NRC-recommended ARCON96 code, based on the two-year meteorological observation data at HTR-PM s site and the final building layout. The evaluation result demonstrates that the individual dose level of staffs in the control room is far below the requirement of the regulatory, which guarantees the radiological habitability of control room of HTR-PM. The contribution of each radionuclide has been analyzed, which suggests that iodine and metal radionuclides such as Cs-137 and Co-60 are primary contributor of the total dose, which are worthy of particular attention in the control room habitability design. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2018

Argonne Reactor Code Software Verification and Validation Plan for the Versatile Test Reactor

This Argonne Reactor Code (ARC) software has taken a pivotal role in the design work of the Versatile Test Reactor. The ARC software itself consists of several independent or semiindependent pieces of software that are used in concert to carry out specific analysis and design objectives. In this report, we discuss the validation that is needed for the ARC software overall and discuss available, usable experimental data and how it relates to the VTR project. We also discuss the verification that is required for the ARC software with respect to its usage in the VTR project and give some concrete examples as to what it must accomplish. A list of goals for the verification and validation work are presented as the general procedure that will be implemented for every individual piece of software in ARC. The ARC software has been continuously developed for over 50 years and many of the features used for the VTR project have thus been rigorously tested for accuracy during that timeframe. The ARC software is presently under source code control and undergoes nightly regression testing of not only the individual pieces, but the interconnected aspects that are common to most analysis projects. Its greatest weakness, from a software verification and validation perspective, is both a lack of documentation and outdated documentation. As an example, much of the previous validation work will require contemporary effort to re-evaluate given the updated base data libraries and modifications to the software that have occurred since. This requires new documentation to be produced to support the usage of the software in the VTR project. The bulk of the ARC software is focused on steady state neutronics and TH behavior of a reactor system. In this regard, we have identified the ZPPR, FFTF, and EBR-II programs as the best candidates for sources of validation data for the VTR project. The ZPPR-15 series in the ZPPR program was chosen as it is the best representation of the VTR fuel form and reactor core layout of all of the ZPPR measurements. Additional support for it comes from the fact that most of the data has already been processed into usable ARC models for VTR validation. The FFTF and EBR-II reactors were both functional reactors that provide reactor scale measurements. FFTF is clearly a closer match to VTR and its startup measurements would be a very valuable validation basis. EBR-II has invaluable burnup related measurements that will play a key role in the ARC software validation work. Each piece of ARC software will require a report detailing the software verification. The validation work will be logically split into the results of modeling each identified experimental resource (ZPPR, EBR-II, FFTF). The intended timeline for the VTR project is to have the ARC software verified and validated by the end of fiscal year 2020. This is incumbent on sufficient funding allocated to V&V activities and no change in programmatic priorities.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

Characterizing in-stream turbulent flow for tidal energy converter siting in Cook Inlet, Alaska

Cook Inlet in Alaska is the most promising location for tidal energy development in the U.S. due to its significant tidal range of approximately 10 meters and high volume flux. The inlet's unique geometry and flow characteristics make it the most energetic tidal stream in the nation, with GW-scale potential energy capacity. With the growing interest in tidal energy converter (TEC) deployment in this area, we implemented a regional-scale, 3D hydrodynamic modeling framework to predict tidal current and turbulence characteristics that can assist TEC designers and project managers. We validated the model results extensively using various datasets collected with bottom-mounted acoustic Doppler current profilers and velocimeters. The comparison between the model outputs and observational data highlighted the effectiveness of the 3D FVCOM model and the Mellor-Yamada Level 2.5 Turbulence Model in accurately assessing macro-scale kinetic energy, turbulence intensity, and the production and dissipation rates at a prospective TEC site. Using two months of model simulation data, we examined the channel cross-section for TEC deployment, focusing on undisturbed power density and macro-scale turbulent properties. Further, our findings indicate that understanding the turbulence characteristics and flow properties can enhance Stage I/II resource characterization by identifying optimal locations for TECs and their layouts within the channel. Furthermore, we demonstrated that TEC designers can utilize macro-scale turbulence data from 3D coastal models as boundary conditions for other turbulence models, allowing for a more detailed resolution of the turbulence structure at TEC siting locations. Ultimately, this work emphasizes the importance of estimating flow and turbulence conditions in energetic systems to understand turbulent sites better and improve resource characterization.

16 TIDAL AND WAVE POWER↗