Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

An open source fast fluid dynamics model for data center thermal management

Although computational fluid dynamics (CFD) has been widely adopted to improve data center thermal management, the high computational demand limits its applications, such as multivariate optimal design and operation. Fast fluid dynamics (FFD), which has been applied for fast airflow simulation, shows great potential. However, few research applied FFD for optimal design and operation of data center thermal management. This research improves the FFD model for data centers and conducts a comprehensive evaluation and demonstration. First, the FFD model is improved by solving the advection and diffusion equations together using an upwind scheme instead of a semi-Lagrangian advection solver in the conventional FFD model. Second, new features for data centers are added, such as a pressure correction method to simulate plenum airflow and dynamic boundary conditions for IT racks. The new FFD model is first validated with two indoor environment cases and the results show that the new FFD model has slightly better overall prediction accuracy and faster speed compared to the conventional FFD model. It is also observed that both FFD models achieve acceptable accuracy, except for a few localized disparities with experimental data, which might be due to simplified handling of turbulence viscosity near the boundaries. Furthermore, validation with a real data center shows that the FFD model achieves a similar level of accuracy as CFD when compared to the experimental measurements with some level of uncertainties. It is then demonstrated for data center optimal design and operation, which saves 53.4–58.8% of annual energy while still meeting the thermal requirements. In conclusion, with a much faster speed and comparable accuracy compared to CFD, the FFD model parallelized on a graphics processing unit is promising for practical model-based data center early design and operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Processes for Quality Improvements in Radiation Oncology Clinical Trials

Quality assurance in radiotherapy (RT) has been an integral aspect of cooperative group clinical trials since 1970. In early clinical trials, data acquisition was nonuniform and inconsistent and computational models for radiation dose calculation varied significantly. Process improvements developed for data acquisition, credentialing, and data management have provided the necessary infrastructure for uniform data. With continued improvement in the technology and delivery of RT, evaluation processes for target definition, RT planning, and execution undergo constant review. As we move to multimodality image-based definitions of target volumes for protocols, future clinical trials will require near real-time image analysis and feedback to field investigators. The ability of quality assurance centers to meet these real-time challenges with robust electronic interaction platforms for imaging acquisition, review, archiving, and quantitative review of volumetric RT plans will be the primary challenge for future successful clinical trials.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

The Ontologies Community of Practice: A CGIAR Initiative for Big Data in Agrifood Systems

Heterogeneous and multidisciplinary data generated by research on sustainable global agriculture and agrifood systems requires quality data labeling or annotation in order to be interoperable. As recommended by the FAIR principles, data, labels, and metadata must use controlled vocabularies and ontologies that are popular in the knowledge domain and commonly used by the community. Despite the existence of robust ontologies in the Life Sciences, there is currently no comprehensive full set of ontologies recommended for data annotation across agricultural research disciplines. In this paper, we discuss the added value of the Ontologies Community of Practice (CoP) of the CGIAR Platform for Big Data in Agriculture for harnessing relevant expertise in ontology development and identifying innovative solutions that support quality data annotation. The Ontologies CoP stimulates knowledge sharing among stakeholders, such as researchers, data managers, domain experts, experts in ontology design, and platform development teams.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Data Bookkeeping Service 3 - Providing Event Metadata in CMS

The Data Bookkeeping Service 3 provides a catalog of event metadata for Monte Carlo and recorded data of the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) at CERN, Geneva. It comprises all necessary information for tracking datasets, their processing history and associations between runs, files and datasets, on a large scale of about 200, 000 datasets and more than 40 million files, which adds up in around 700 GB of metadata. The DBS is an essential part of the CMS Data Management and Workload Management (DMWM) systems [1], all kind of data-processing like Monte Carlo production, processing of recorded event data as well as physics analysis done by the users are heavily relying on the information stored in DBS.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

AMRZone: A Runtime AMR Data Sharing Framework For Scientific Applications

Frameworks that facilitate runtime data sharing across multiple applications are of great importance for scientific data analytics. Although existing frameworks work well over uniform mesh data, they can not effectively handle adaptive mesh refinement (AMR) data. Among the challenges to construct an AMR-capable framework include: (1) designing an architecture that facilitates online AMR data management; (2) achieving a load-balanced AMR data distribution for the data staging space at runtime; and (3) building an effective online index to support the unique spatial data retrieval requirements for AMR data. Towards addressing these challenges to support runtime AMR data sharing across scientific applications, we present the AMRZone framework. Experiments over real-world AMR datasets demonstrate AMRZone's effectiveness at achieving a balanced workload distribution, reading/writing large-scale datasets with thousands of parallel processes, and satisfying queries with spatial constraints. Moreover, AMRZone's performance and scalability are even comparable with existing state-of-the-art work when tested over uniform mesh data with up to 16384 cores; in the best case, our framework achieves a 46% performance improvement.

Zhang, Wenzhao↗

Canopus: A Paradigm Shift Towards Elastic Extreme-Scale Data Analytics on HPC

Scientific simulations on high performance computing (HPC) platforms generate large quantities of data. To bridge the widening gap between compute and I/O, and enable data to be more efficiently stored and analyzed, simulation outputs need to be refactored, reduced, and appropriately mapped to storage tiers. However, a systematic solution to support these steps has been lacking on the current HPC software ecosystem. To that end, this paper develops Canopus, a progressive JPEGlike data management scheme for storing and analyzing big scientific data. It co-designs the data decimation, compression and data storage, taking the hardware characteristics of each storage tier into considerations. With reasonably low overhead, our approach refactors simulation data into a much smaller, reduced-accuracy base dataset, and a series of deltas that is used to augment the accuracy if needed. The base dataset and deltas are compressed and written to multiple storage tiers. Data saved on different tiers can then be selectively retrieved to restore the level of accuracy that satisfies data analytics. Thus, Canopus provides a paradigm shift towards elastic data analytics and enables end users to make trade-offs between analysis speed and accuracy on-the-fly. We evaluate the impact of Canopus on unstructured triangular meshes, a pervasive data model used by scientific modeling and simulations. In particular, we demonstrate the progressive data exploration of Canopus using the “blob detection” use case on the fusion simulation data.

Lu, Tao↗

SIRIUS: Enabling Progressive Data Exploration for Extreme-Scale Scientific Data

Scientific simulations on high performance computing (HPC) platforms generate large quantities of data. To bridge the widening gap between compute and I/O, and enable data to be more efficiently stored and analyzed, simulation outputs need to be refactored, reduced, and appropriately mapped to storage tiers. However, a systematic solution to support these steps has been lacking in the current HPC software ecosystem. To that end, this paper develops SIRIUS, a progressive JPEG-like data management scheme for storing and analyzing big scientific data. It co-designs data decimation, compression, and data storage, taking the hardware characteristics of each storage tier into considerations. With reasonably low overhead, our approach refactors simulation data, using either topological or uniform decimation, into a much smaller, reduced-accuracy base dataset, and a series of deltas that is used to augment the accuracy if needed. The base dataset and deltas are compressed and written to multiple storage tiers. Data saved on different tiers can then be selectively retrieved to restore the level of accuracy that satisfies data analytics. Thus, SIRIUS provides a paradigm shift towards elastic data analytics and enables end users to make trade-offs between analysis speed and accuracy on-the-fly. This paper further develops algorithms to preserve statistics for data decimation, a common requirement for reducing data. Here, we assess the impact of SIRIUS on unstructured triangular meshes, a pervasive data model used in scientific simulations. In particular, we evaluate two realistic use cases: the blob detection in fusion and high-pressure area extraction in computational fluid dynamics.

97 MATHEMATICS AND COMPUTING↗

Anomaly Detection, Localization and Classification using Drifting Synchrophasor Data Streams

With ongoing automation and digitization of the electric power system, several Phasor Measurement Units(PMUs) have been deployed for monitoring and control. PMU data can have multiple anomalies, and many of the researchers in the past have concentrated on training machine/deep learning algorithms offline for anomaly detection over PMU data (i.e., not in real time). These machine/deep learning algorithms, when trained offline on a sample rather than a population of the dataset, fail to consider the dynamic behavior of the power grid in real-time, resulting in low accuracy. Considering the dynamic behavior of the power grid (e.g., change in load, generation, distributed energy resources (DERs) switching, network, controls), the definition of data anomalies varies in time and requires online training. A fundamental challenge is to enable online (i.e., real-time) training of machine/deep learning algorithms for anomaly detection over streaming PMU data. While machine/deep learning is often desirable to manage data streams, training a deep learning algorithm over streaming PMU data is nontrivial due to changes in data statistics caused by dynamic streaming data. This paper proposes PMUNET: a novel device-level deep learning-based data-driven approach for anomaly detection, localization, and classification over streaming PMU data, using online learning and multivariate data-drift detection algorithm .Two variants of PMUNET, Dynamic data Change Driven Learning (DCDL) and Continuity Driven Learning (CDL), are proposed and compared. DCDL aims to train the deep learning algorithm whenever the definition of anomaly changes due to the power grid dynamics. On the other hand, CDL continuously trains the deep learning algorithm over the PMU data-stream. The experimental results verify that DCDL outperforms CDL and other efficient anomaly detection methods over multiple events such as faults and load/ generator/capacitor/DERs variations/switching for IEEE 14 and 39 Bus test system as well as real PMU industrial data. The result verifies that DCDL variant of PMUNET improves over existing approach with a gain of 2% - 10% in terms of accuracy, false-positive rate, and false-negative rate.

adversarial deep learning↗

LANL Contributions to pyiron: An integrated development environment for computational materials science

To support and accelerate the development of simulation protocols in atomistic modelling, we introduce an integrated development environment (IDE) for computational materials science called pyiron (http://pyiron.org). The pyiron IDE combines a web based source code editor, a job management system for build automation, and a hierarchical data management solution. The core components of the pyiron IDE are pyiron objects based on an abstract class, which links application structures such as atomistic structures, projects, jobs, simulation protocols and computing resources with persistent storage and an interactive user environment. The simulation protocols within the pyiron IDE are constructed using the Python programming language. To highlight key concepts of this tool as well as to demonstrate its ability to simplify the implementation and testing of simulation protocols we discuss two applications. In these examples we show how pyiron supports the whole life cycle of a typical simulation, seamlessly combines ab initio with empirical potential calculations, and how complex feedback loops can be implemented. While originally developed with focus on ab initio thermodynamics simulations, the concepts and implementation of pyiron are general thus allowing to employ it for a wide range of simulation topics.

Janssen, Jan↗

AGR-2 Data Qualification Report for ATR Cycles 149B, 150A, 150B, 151A, and 151B

This report provides the data qualification status of AGR-2 fuel irradiation experimental data from Advanced Test Reactor (ATR) cycles 149B, 150A, 150B, 151A, and 151B), as recorded in the Nuclear Data Management and Analysis System (NDMAS). The AGR-2 data streams addressed include thermocouple temperatures, sweep gas data (flow rate, pressure, and moisture content), and fission product monitoring system (FPMS) data for each of the six capsules in the experiment. A total of 3,307,500 5-minute thermocouple and sweep gas data records were received and processed by NDMAS for this period. There are no AGR-2 data for cycle 150A because the experiment was removed from the reactor. Of these data, 82.2% were determined to be Qualified based on NDMAS accuracy testing and data validity assessment. There were 450,557 Failed temperature records due to thermocouple failures, and 138,528 Failed gas flow records due to gas flow cross-talk and leakage problems that occurred in the capsules after cycle 150A. For FPMS data, NDMAS received and processed preliminary release rate and release-to-birth rate ratio (R/B) data for the first three reactor cycles (cycles 149B, 150B, and 151B). This data consists of 45,983 release rate records and 45,235 R/B records for the 12 radionuclides reported. The qualification status of these FPMS data has been set to In Process until receipt of QA-approved data generator reports. All of the above data have been processed and tested using a SAS®-based enterprise application software system, stored in a secure Structured Query Language database, and made available on the NDMAS Web portal (http://ndmas.inl.gov) for both internal and external VHTR project participants.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

AGR-2 Data Qualification Report for ATR Cycles 151B-2, 152A, 152B, 153A, 153B and 154A

This report documents the data qualification status of AGR-2 fuel irradiation experimental data from Advanced Test Reactor (ATR) Cycles 152A, 152B, 153A, 153B, and 154A, as recorded in the Nuclear Data Management and Analysis System (NDMAS). The AGR-2 data streams addressed include thermocouple (TC) temperatures, sweep gas data (flow rate, pressure, and moisture content), and fission product monitoring system (FPMS) data for each of the six capsules in the experiment. A total of 13,400,520 every minute instantaneous TC and sweep gas data records were received and processed by NDMAS for this period. Of these data, 8,911,791 records (66.5% of the total) were determined to be Qualified based on NDMAS accuracy testing and data validity assessment. For temperature, there were 4,266,081 records (74% of the total TC data) that were Failed due to TC instrument failures. For sweep gas flows, there were 222,648 gas flow records (2.91% of the flow data) that were Failed. The inlet gas flow failures due to gas flow cross-talk and leakage problems that occurred after Cycle 150A were corrected by using the same gas mixture in all six capsules and the Leadout. For FPMS data, NDMAS received and processed preliminary release rate and release-to-birth rate ratio (R/B) data for three reactor cycles (Cycles 149B, 150B, and 151A) . This data consists of 45,983 release rate records and 45,235 R/B records for the 12 radionuclides reported. The qualification status of these FPMS data has been set to In Process until receipt of Quality Assurance-approved data generator reports. All of the above data have been processed and tested using a SAS®-based enterprise application software system, stored in a secure Structured Query Language database, made available on the NDMAS Web portal (http://ndmas.inl.gov), and approved by the INL STIM for release to both internal and appropriate external Very High Temperature Reactor Program participants.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

CaveMan Enterprise version 1.0 Software Validation and Verification.

The U.S. Department of Energy Strategic Petroleum Reserve stores crude oil in caverns solution-mined in salt domes along the Gulf Coast of Louisiana and Texas. The CaveMan software program has been used since the late 1990s as one tool to analyze pressure mea- surements monitored at each cavern. The purpose of this monitoring is to catch potential cavern integrity issues as soon as possible. The CaveMan software was written in Microsoft Visual Basic, and embedded in a Microsoft Excel workbook; this method of running the CaveMan software is no longer sustainable. As such, a new version called CaveMan Enter- prise has been developed. CaveMan Enterprise version 1.0 does not have any changes to the CaveMan numerical models. CaveMan Enterprise represents, instead, a change from desktop-managed work- books to an enterprise framework, moving data management into coordinated databases and porting the numerical modeling codes into the Python programming language. This document provides a report of the code validation and verification testing.

Hart, David↗

A National Infrastructure for Artificial Intelligence on the Grid (NI4AI) (Final Scientific/Technical Report)

Electric utilities have traditionally taken a very pragmatic yet myopic approach with grid sensors and the resulting collected data. Sensors are purchased and deployed to solve a specific, known problem that has risen to sufficient awareness as to justify the effort of deploying sensors and the needed capital investment. This sensor data flows into proprietary software packages with limited functionality intended only to address the initial problem. This approach aligns with the financial incentives of the utility to deploy capital into fixed hardware assets for which the corporations earn a rate of return. This mentality stands in stark contrast to the big data revolution that started nearly 25 years ago with the rise of Google. In this worldview, data is a fundamental business asset; successful organizations collect, store, explore, merge, and exploit as much data as possible to not only solve problems well understood today but also to tackle new problems that will inevitably rise tomorrow. The ARPA-E Open Innovation 2018 project entitled A National Infrastructure for Artificial Intelligence on the Grid or NI4AI for short was designed to demonstrate this alternative paradigm for using data. To do this, the project was composed of three key thrust areas. The first major component deployed a variety of high-frequency grid sensors and captured terabytes of both wide-scale and localized grid measurements, generating high-value datasets for grid research and algorithm development. The second aspect made available PingThings’ PredictiveGridTM, a horizontally scalable, cloud-based data management and AI platform built for time series data to explore and exploit the collected data. Finally, the project fostered a diverse and open research community composed of experts from numerous fields through focused educational content, code sharing, and data science competitions. Shifting away from “single use” sensors and closed data silos within electric utilities is a major benefit to the public at large. This legacy approach to data is incredibly (1) capital intensive (new sensors must be deployed for each new problem and problems tend to arise continuously) and (2) painfully slow (new problems must be identified first and then new sensors must be deployed to collect data to begin to address the issue). The transition to a carbon neutral grid requires a massive transformation of the existing grid infrastructure and will continue to challenge the legacy grid in unforeseen ways. The only way to make the energy transition cost effective is for utilities to abandon this dated data paradigm and adopt more contemporary approaches. NI4AI has shown that it is technically possible and economically feasible to ingest, explore, and exploit grid data collected from even very high frequency sensing, such as continuous point on wave sensors collecting measurements 10,000 times a second. In fact, the PredictiveGrid platform used is commercially available and deployed at several utilities in the United States. Project accomplishments were numerous and included (1) making available a state of the art time series platform to the community, (2) collecting over 520 streams of time series data from grid sensors totaling over 1 trillion grid measurements, and (3) developing and nurturing a community within the industry focused on the use of data to create value for utilities and, ultimately, end consumers.

97 MATHEMATICS AND COMPUTING↗

IGDS/TRAP Interface Program (ITIP). Software User Manual (SUM)

This specification establishes the requirements, concepts, and preliminary design for a set of software known as the IGDS/TRAP Interface Program (ITIP). This software provides the capability to develop at an Interactive Graphics Design System (IGDS) design station process flow diagrams for use by the NASA Coal Gasification Task Team. In addition, ITIP will use the Data Management and Retrieval System (DMRS) to maintain a data base from which a properly formatted input file to the Time-Line and Resources Analysis Program (TRAP) can be extracted. This set of software will reside on the PDP-11/70 and will become the primary interface between the Coal Gasification Task Team and IGDS, DMRS, and TRAP. The user manual for the computer program is presented.

Jefferys, S.↗

Interactive design and analysis of future large spacecraft concepts

Recent developments in interactive computer-aided design and analysis of future large spacecraft concepts are presented. The primary emphases are on automatic spacecraft modeling for lattice (truss-like) structures and simplified multidiscipline design and analysis modules. Capabilities and performance of the twenty some applications modules, the executive and data management software, and graphics display features are reviewed. Data generated for an earth-orbiting large diameter (750 meter) antenna satellite are used to illustrate current capabilities. A single user at an interactive terminal can create, design, analyze, and conduct parametric studies of earth-orbiting spacecraft with relative ease. The approach is particularly useful in the conceptual design phase of advanced space missions when a multiplicity of concepts must be evaluated in a cost-effective and timely manner

Garrett, L. B.↗

Stability Statistics Data Base System

Stability Statistics Data Base System is written entirely in FORTRAN and does not require use of outside data-management routines. Is used with directly-accessible data base containing simple card images. Four FORTRAN modules implement edit, sort, report-generation, and other functions.

Kwok, G. Y.↗

Data archaeology at ICES

This paper provides a brief overview of the function of the International Council for the Exploration of the Sea (ICES), both past and present, in particular in the context of its interest in compiling oceanographic data sets. Details are provided of the procedures it adopted to ensure adequate internationally collaborative marine investigations during the first part of the century, such as how it provided a forum for action by its member states, how it coordinated and published the results of scientific programs, and how it provided a foundation, through scientists employed in the ICES Office, for the establishment of the original oceanographic marine databases and associated products, and the scientific interpretation of the results. The growth and expansion of this area of ICES activity is then traced, taking into account the changing conditions for oceanographic data management resulting from the establishment of the National Data Centres, as well as the World Data Centres for Oceanography, which were created to meet the needs of the International Geophysical Year (IGY). Finally, there is a discussion of the way in which the very existence of ICES has proved to be a valuable source of old data, some of which have not yet been digitized, but which can be readily retrieved because they have been very carefully documented throughout the years. Lessons from this activity are noted, and suggestions are made on how the past experiences of ICES can be utilized to ensure the availability of marine data to present and future generations of scientists.

Dooley, Harry D.↗