Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

AGR-2 Data Qualification Interim Report

Projects for the very high temperature reactor (VHTR) Technology Development Office program provide data in support of Nuclear Regulatory Commission licensing of the VHTR. Fuel and materials to be used in the reactor are tested and characterized to quantify performance in high temperature and high fluence environments. The VHTR program established the NGNP Data Management and Analysis System (NDMAS) to manage and document VHTR data qualification, for storage of the data in a readily accessible electronic form, and to assist in the analysis and presentation of the data. This document gives the status of NDMAS processing and qualification of data associated with the initial reactor cycle (147A) of the second Advanced Gas Reactor (AGR-2) experiment which began on June 21, 2010. Because it is early in the AGR-2 experiment, data from only two AGR-2 data streams are reported on: Fuel Fabrication and Fuel Irradiation data. As of August 1, 2010, approximately 311,000 irradiation data records have been stored in NDMAS, and qualification tests are in progress. Preliminary information indicates that TC 2 in Capsule 2 failed prior to start of the experiment, and NDMAS testing has thus far identified only two invalid data values from the METSO data collection system Data from the Fission Product Monitoring System (FPMS) are not currently processed until after reactor cycle shutdown and have not yet been received. A description of the ATR operating conditions data associated with the AGR-2 experiment (e.g., power levels) are summarized in the AGR-1 data qualification report (INL/EXT-09-16460). Since ATR data are collected under ATR program data quality requirements (i.e., outside the VHTR program), the NGNP program and NDMAS do not take additional actions to qualify these data other than NDMAS capture testing. Data qualification of graphite characterization data collected under the Graphite Technology Development Project is reported in a separate status report (Hull 2010).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Development of Streamlined Nuclear Safety Analysis Tool for Spent Nuclear Fuel Applications

To understand the changing nuclear and mechanical characteristics of spent nuclear fuel (SNF) or used nuclear fuel (UNF) and the different storage, transportation, and disposal systems at various stages within the waste management system, different types of analyses are required. These analyses require the use of assorted tools and numerous types of data. Using the appropriate modeling and simulation (M&S) parameters and selecting from the diversity of analytic tools to conduct SNF analyses can be a tedious, error-prone, and time-consuming undertaking for analysts and reviewers alike. A new, integrated data and analysis system was designed to simplify and automate performance of accurate, efficient evaluations for characterizing the input to the overall U.S. nuclear waste management system—the UNF-Storage, Transportation & Disposal Analysis Resource and Data System (UNF-ST&DARDS). A relational database has been assembled to provide a standard means by which UNF-ST&DARDS can succinctly store and retrieve M&S parameters for specific SNF analysis. A library of various analysis model templates is used to communicate M&S parameters for the most appropriate M&S application. A process manager facilitates performance of actual as-loaded, assembly-specific, and cask-specific evaluations. Interactive visualization capabilities facilitate data analysis and results interpretation. To date, UNF-ST&DARDS has completed (1) explicit depletion and decay analysis of every fuel assembly (~245 000) discharged from commercial U.S. reactors through June 2013, with 13 cooling time steps (results include isotopic compositions for 142 isotopes, and radiation and thermal source terms); (2) SNF radiation dose rate evaluations at 1 m for all the fuel assemblies discharged through June 2013; and (3) criticality, shielding, thermal, and containment analyses of hundreds of loaded casks. In conclusion, UNF-ST&DARDS also provides various automated report generation capabilities with dynamic figure and table update capabilities based on changes to the Unified Database.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Used Nuclear Fuel-Storage, Transportation & Disposal Analysis Resource and Data System (UNF-ST&DARDS) 4.x

A new, integrated data and analysis system has been designed to simplify and automate the performance of accurate and efficient evaluations for characterizing the input to the overall waste management system-UNF-Storage, Transportation & Disposal Analysis Resource and Data System (UNF-ST&DARDS). A relational database has been assembled to provide a standard means by which UNF-ST&DARDS can succinctly store and retrieve M&S parameters for specific SNF analysis. Users can import SNF data into this database for user-specific analysis. A library of various analysis model templates provides the ability to communicat the various set of M&S parameters to the most appropriate M&S application. A process manager facilitates performance of actual, as-loaded, assembly-specific and cask-specific evaluations. Interactive visualization capabilities facilitate data analysis and results interpretation.

Banerjee, Kaushik↗

Karlsruhe Database for Radioactive Wastes (KADABRA) - Accounting and Management System for Radioactive Waste Treatment - 12275

The data management system KADABRA was designed according to the purposes of the Cen-tral Decontamination Department (HDB) of the Wiederaufarbeitungsanlage Karlsruhe Rueckbau- und Entsorgungs-GmbH (WAK GmbH), which is specialized in the treatment and conditioning of radioactive waste. The layout considers the major treatment processes of the HDB as well as regulatory and legal requirements. KADABRA is designed as an SAG ADABAS application on IBM system Z mainframe. The main function of the system is the data management of all processes related to treatment, transfer and storage of radioactive material within HDB. KADABRA records the relevant data concerning radioactive residues, interim products and waste products as well as the production parameters relevant for final disposal. Analytical data from the laboratory and non destructive assay systems, that describe the chemical and radiological properties of residues, production batches, interim products as well as final waste products, can be linked to the respective dataset for documentation and declaration. The system enables the operator to trace the radioactive material through processing and storage. Information on the actual sta-tus of the material as well as radiological data and storage position can be gained immediately on request. A variety of programs accessed to the database allow the generation of individual reports on periodic or special request. KADABRA offers a high security standard and is constantly adapted to the recent requirements of the organization. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

A comprehensive data acquisition and management system for an ecosystem-scale peatland warming and elevated CO 2 experiment

Ecosystem-scale manipulation experiments represent large science investments that require well-designed data acquisition and management systems to provide reliable, accurate information to project participants and third party users. The SPRUCE project (Spruce and Peatland Responses Under Climatic and Environmental Change, http://mnspruce.ornl.gov ) is such an experiment funded by the Department of Energy's (DOE), Office of Science, Terrestrial Ecosystem Science (TES) Program. The SPRUCE experimental mission is to assess ecosystem-level biological responses of vulnerable, high carbon terrestrial ecosystems to a range of climate warming manipulations and an elevated CO 2 atmosphere. SPRUCE provides a platform for testing mechanisms controlling the vulnerability of organisms, biogeochemical processes, and ecosystems to climatic change (e.g., thresholds for organism decline or mortality, limitations to regeneration, biogeochemical limitations to productivity, and the cycling and release of CO 2 and CH 4 to the atmosphere). The SPRUCE experiment will generate a wide range of continuous and discrete measurements. In order to successfully manage SPRUCE data collection, achieve SPRUCE science objectives, and support broader climate change research, the research staff has designed a flexible data system using proven network technologies and software components. The primary SPRUCE data system components are the following; 1. data acquisition and control system – set of hardware and software to retrieve biological and engineering data from sensors, collect sensor status information, and distribute feedback to control components; 2. data collection system – set of hardware and software to deliver data to a central depository for storage and further processing; and 3. data management plan – set of plans, policies, and practices to control consistency, protect data integrity, and deliver data. This publication presents our approach to meeting the challenges of designing and constructing an efficient data system for managing high volume sources of in situ observations in a remote, harsh environmental location. Finally, the approach covers data flow starting from the sensors and ending at the archival/distribution points, discusses types of hardware and software used, examines design considerations that were used to choose them, and describes the data management practices chosen to control and enhance the value of the data.

54 ENVIRONMENTAL SCIENCES↗

dCache on Steroids - Delegated Storage Solutions

For over a decade, dCache.org has delivered a robust software used at more than 80 Universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments as well as many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from a SoC based all-in-one Raspberry-Pi up to hundreds of nodes in a multipetabyte installation. Due to lack of managed storage at the time, dCache implemented data placement, replication and data integrity directly. Today, many alternatives are available: S3, GlusterFS, CEPH and others. While such solutions position themselves as scalable storage systems, they cannot be used by many scientific communities out of the box. The absence of community-accepted authentication and authorization mechanisms, the use of product specific protocols and the lack of namespace are some of the reasons that prevent wide-scale adoption of these alternatives. Most of these limitations are already solved by dCache. By delegating low-level storage management functionality to the above-mentioned new systems and providing the missing layer through dCache, we provide a solution which combines the benefits of both worlds - industry standard storage building blocks with the access protocols and authentication required by scientific communities. In this paper, we focus on CEPH, a popular software for clustered storage that supports file, block and object interfaces. CEPH is often used in modern computing centers, for example as a backend to OpenStack services. We will show prototypes of dCache running with a CEPH backend and discuss the benefits and limitations of such an approach. As a result, we will also outline the roadmap for supporting ‘delegated storage’ within the dCache releases.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Object storage model for CMS data

In CMS, data access and management is organized around the data-tier model: a static definition of what subset of event information is available in a particular dataset, realized as a collection of files. In previous work, we have proposed a novel data management model that obviates the need for data tiers by exploding files into individual event data product objects. In this work, we estimate the potential savings in data volume based on user analysis patterns.

Smith, Nick↗

Using Dashboards to Improve Energy and Comfort in Federal Buildings

Federal agencies are taking many steps to improve the sustainability of their operations, including improving the energy efficiency of their buildings, promoting recycling and reuse of materials, encouraging carpooling and alternative transit schemes, and installing low flow water fixture units are just a few of the common examples. However, an often overlooked means of energy savings is to provide feedback to building users about their energy use through information dashboards connected to a building?s energy information system. An Energy Information System (EIS), broadly defined, is a package of performance monitoring software, data acquisition hardware, and communication systems that is used to collect, store, analyze, and display energy information. At a minimum, the EIS provides the whole-building energy-use information (Granderson 2009a). We define a ?dashboard? as a display and visualization tool that utilizes the EIS data and technology to provide critical information to users. This information can lead to actions resulting in energy savings, comfort improvements, efficient operations, and more. The tools to report analyzed information have existed in the information technology as business intelligence (Few 2006). The dashboard is distinguished from the EIS as a whole, which includes additional hardware and software components to collect and storage data, and analysis for resources and energy management (Granderson 2009b). EIS can be used for a variety of uses, including benchmarking, base-lining, anomaly detection, off-hours energy use evaluation, load shape optimization, energy rate analysis, retrofit and retro-commissioning savings (Granderson 2009a). The use of these EIS features depends on the specific users. For example, federal and other building managers may use anomaly detection to identify energy waste in a specific building, or to benchmark energy use in similar buildings to identify energy saving potential and reduce operational cost. There are several vendors of EIS technology that provide information on energy and other environmental variables in buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Used Nuclear Fuel-Storage, Transportation & Disposal Analysis Resource and Data System (UNF-ST&DARDS)

This will be licensed via RSICC. A new, integrated data and analysis system has been designed to simplify and automate the performance of accurate and efficient evaluations for characterizing the input to the overall nuclear waste management system -UNF-Storage, Transportation & Disposal Analysis Resource and Data System (UNF-ST&DARDS). A relational database within UNF-ST&DARDS provides a standard means by which UNF-ST&DARDS can succinctly store and retrieve modeling and simulation (M&S) parameters for specific spent nuclear fuel analysis. A library of various analysis model templates provides the ability to communicate the various set of M&S parameters to the most appropriate M&S application. Interactive visualization capabilities facilitate data analysis and results interpretation. UNF-ST&DARDS current analysis capabilities include (1) assembly-specific depletion and decay, (2) and spent nuclear fuel cask-specific criticality and shielding. Currently, UNF-ST&DARDS uses SCALE nuclear analysis code system for performing nuclear analysis.

Banerjee, Kaushik↗

FJET Database Project: Extract, Transform, and Load

The Data Mining & Knowledge Management team at Kennedy Space Center is providing data management services to the Frangible Joint Empirical Test (FJET) project at Langley Research Center (LARC). FJET is a project under the NASA Engineering and Safety Center (NESC). The purpose of FJET is to conduct an assessment of mild detonating fuse (MDF) frangible joints (FJs) for human spacecraft separation tasks in support of the NASA Commercial Crew Program. The Data Mining & Knowledge Management team has been tasked with creating and managing a database for the efficient storage and retrieval of FJET test data. This paper details the Extract, Transform, and Load (ETL) process as it is related to gathering FJET test data into a Microsoft SQL relational database, and making that data available to the data users. Lessons learned, procedures implemented, and programming code samples are discussed to help detail the learning experienced as the Data Mining & Knowledge Management team adapted to changing requirements and new technology while maintaining flexibility of design in various aspects of the data management project.

excel vba↗

Optimizing tertiary storage organization and access for spatio-temporal datasets

We address in this paper data management techniques for efficiently retrieving requested subsets of large datasets stored on mass storage devices. This problem represents a major bottleneck that can negate the benefits of fast networks, because the time to access a subset from a large dataset stored on a mass storage system is much greater that the time to transmit that subset over a network. This paper focuses on very large spatial and temporal datasets generated by simulation programs in the area of climate modeling, but the techniques developed can be applied to other applications that deal with large multidimensional datasets. The main requirement we have addressed is the efficient access of subsets of information contained within much larger datasets, for the purpose of analysis and interactive visualization. We have developed data partitioning techniques that partition datasets into 'clusters' based on analysis of data access patterns and storage device characteristics. The goal is to minimize the number of clusters read from mass storage systems when subsets are requested. We emphasize in this paper proposed enhancements to current storage server protocols to permit control over physical placement of data on storage devices. We also discuss in some detail the aspects of the interface between the application programs and the mass storage system, as well as a workbench to help scientists to design the best reorganization of a dataset for anticipated access patterns.

Chen, Ling Tony↗

Simulating Secure Data Exchange and Storage for Urban Air Mobility Environments

Urban Air Mobility (UAM) defines an environment for managing operations of vertical takeoff and landing (VTOL) and short takeoff and landing (STOL) vehicles in an urban environment. Within a UAM environment, UAM operators manage fleets of vehicles, relying on Providers of Services for UAM (PSUs) for managing flights in a region of airspace. Flight plan deconfliction is primarily performed by the Discovery and Synchronization Service (DSS), and the Federal Aviation Administration (FAA) maintains control over the UAM space via the FAA-Industry Exchange Protocol (FIDXP). UAM is a federated environment with many different entities owning and operating vehicles, PSUs, and other services. These entities often need to interoperate or access data generated by other organizations. This paper demonstrates the feasibility of using blockchain to facilitate a secure data exchange and storage for this flight information in a UAM environment. In particular, this paper is focused on flight plans and telemetry data. A blockchain network was developed with a set of smart contracts for managing relevant flight data. Hyperledger Fabric was chosen as it is performent, scalable, and allows organizations to reuse existing public key infrastructure (PKI) for identity management. A set of simulated UAM services were also developed. These services propose flight plans and negotiate with other UAM services for airspace access. All interactions between UAM services, as well as vehicle telemetry data, is recorded onto the blockchain. Vehicle telemetry data is generated by a vehicle flight simulation service. This paper successfully demonstrates the feasibility of using blockchain as a secure data exchange and storage mechanism in a UAM environment.

UAM↗

Medipix3 array high performance read-out board for synchrotron research

The Medipix3 ASIC is one of the most advanced chip that is presently available to build photon counting area detectors. The capabilities of the chip include adjacent pixels charge summing circuitry to sort out the distortion due to charge sharing, simultaneous counting and read-out that enables frames to be acquired without dead time, the colour mode of operation that enables up to eight energy bands to be acquired. In order to fully exploit the capabilities of the Medipix3 chip in synchrotron research, a high performance electronic board capable of driving large arrays of chips is necessary. We propose a parallel read-out board of Medipix3 chip arrays with a scalable architecture that allows driving the Medipix3 chip in all of its modes of operation. The board functions include the control of the chip arrays, data formatting and data compression, the management of the communications with the data storage devices, and operation in various trigger modes. In addition to this the board will have some 'intelligence' embedded. This will add some very important features to the final detector such as pattern recognition, capability of variable frame duration as a function of the photon flux, feedback to other equipment and real time calculations of data relevant to experiments such as the autocorrelation function.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The Kepler End-to-End Data Pipeline: From Photons to Far Away Worlds

The Kepler mission is described in overview and the Kepler technique for discovering exoplanets is discussed. The design and implementation of the Kepler spacecraft, tracing the data path from photons entering the telescope aperture through raw observation data transmitted to the ground operations team is described. The technical challenges of operating a large aperture photometer with an unprecedented 95 million pixel detector are addressed as well as the onboard technique for processing and reducing the large volume of data produced by the Kepler photometer. The technique and challenge of day-to-day mission operations that result in a very high percentage of time on target is discussed. This includes the day to day process for monitoring and managing the health of the spacecraft, the annual process for maintaining sun on the solar arrays while still keeping the telescope pointed at the fixed science target, the process for safely but rapidly returning to science operations after a spacecraft initiated safing event and the long term anomaly resolution process.The ground data processing pipeline, from the point that science data is received on the ground to the presentation of preliminary planetary candidates and supporting data to the science team for further evaluation is discussed. Ground management, control, exchange and storage of Kepler's large and growing data set is discussed as well as the process and techniques for removing noise sources and applying calibrations to intermediate data products.

data archiving↗

ASCR Workshop on In Situ Data Management: Enabling Scientific Discovery from Diverse Data Sources

In January 2019, the U.S. Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions for in situ data management (ISDM). The workshop defined ISDM as the practices, capabilities, and procedures to control the organization of data and enable the coordination and communication among heterogeneous tasks, executing simultaneously in a high-performance computing system, cooperating toward a common objective. The workshop revealed two primary, interdependent motivations for processing and managing data in situ. The first motivation is that the in situ methodology enables scientific discovery from a broad range of data sources over a wide scale of computing platforms: leadership-class systems, clusters, clouds, workstations, and embedded devices at the edge. The successful development of ISDM capabilities will benefit real-time decision-making, design optimization, and data-driven scientific discovery. The second motivation is the need to decrease data volumes. ISDM can make critical contributions to managing large data volumes from computations and experiments to minimize data movement, save storage space, and boost resource efficiency, often while simultaneously increasing scientific precision. A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery across the Office of Science mission space. Applications wanting to use the in situ capabilities include those where data analysis feeds back to the simulation, decisions are made autonomously, big data or machine learning is among the tasks to be coordinated, and computations need to be completed in real time. The workshop identified six priority research directions that highlight the components and capabilities needed for ISDM to be successful for the wide variety of applications discussed: making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗

Mic-hackathon 2024: hackathon on machine learning for electron and scanning probe microscopy

Microscopy is one of the primary sources of information on materials structure and functionality at the nanometer and atomic scales. The data generated through microscopy is often contained in well-structured datasets, enriched with extensive metadata and sample histories, although not always with the same level of detail or storage format. The broad incorporation of data management plans by major funding agencies ensures the preservation and accessibility of this data. However, deriving insights from these rich datasets remains challenging due to the lack of established code ecosystems, standardized benchmarks, and integration strategies. Correspondingly, the efficiency of data usage is very low, and time expenditures at the analysis stage are enormous. In addition to post-acquisition data analysis, the emergence of application programming interfaces by major microscope manufacturers now creates opportunities for real-time ML-based data analytics to enable automated decision making, and particularly ML-agent controlled real-time microscope operation. Despite these opportunities, there is a significant gap in integrating the ML community with the broader microscopy community, limiting the value that these methods bring to physics and materials discovery and materials optimization. Hackathons address these challenges by fostering collaboration between ML experts and microscopy professionals, encouraging the development of innovative solutions that leverage ML for microscopy and preparing the workforce of the future both for microscopy-intensive domains areas, instrument manufacturers, and ML scientists interested in real world applications for fundamental research, materials optimization, and manufacturing. The hackathon generated benchmark datasets and digital twins of microscopes that further contribute to the development of the field and establish data analysis ecosystems. All the codes can be found at GitHub(https://github.com/KalininGroup/Mic-hackathon-2024-codes-publication/tree/1.0.0.1) and Zenodo (https://zenodo.org/records/15579940).

97 MATHEMATICS AND COMPUTING↗

Storage for Advanced Scientific Use-Cases and Beyond

The dCache project provides open-source storage software deployed internationally to satisfy ever more demanding scientific storage requirements. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, through wide access and easy integration with existing systems. In this paper, we describe some of the recent features that facilitate the use of storage to maximise the gain from stored data, including quality-of-service management, heterogeneous systems - both through integrated tertiary storage support and geographical locality - the parallel NFS (pNFS) extension, and innovative delegated authorisation schemes.

43 PARTICLE ACCELERATORS↗