Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Catalyzing deep decarbonization with federated battery diagnosis and prognosis for better data management in energy storage systems

Industrial data analytics methods play a central role in improving energy storage performance and efficiency, impacting the future of electrified transportation and renewable electricity generation. However, significant challenges hinder the large-scale deployment of batteries. Conventional methods rely on centralized collection and processing of fleet-level data, leading to database size issues and privacy concerns due to potential data breaches. To enable scalable deployment of battery management systems, this article proposes a federated battery diagnosis and prognosis model, which distributes the processing of battery standard current-voltage-time-usage data in a privacy-preserving manner. Instead of transferring the raw data, this approach communicates only the locally processed parameters, thus reducing communication load and preserving data confidentiality. The federated model offers a paradigm shift in battery health management through privacy-preserving distributed methods for battery data processing and lifetime prediction, ensuring the reliable and sustainable deployment of lithium-ion batteries in a rapidly evolving world.

asset health management↗
DOE OSTI2017

MaDaTS: Managing Data on Tiered Storage for Scientific Workflows

Scientific workflows are increasingly used in High Performance Computing (HPC) environments to manage complex simulation and analyses, often consuming and generating large amounts of data. However, workflow tools have limited support for managing the input, output and intermediate data. The data elements of a workflow are often managed by the user through scripts or other ad-hoc mechanisms. Technology advances for future HPC systems is redefining the memory and storage subsystem by introducing additional tiers to improve the I/O performance of data-intensive applications. These architectural changes introduce additional complexities to managing data for scientific workflows. Thus, we need to manage the scientific workflow data across the tiered storage system on HPC machines. In this paper, we present the design and implementation of MaDaTS (Managing Data on Tiered Storage for Scientific Workflows), a software architecture that manages data for scientific workflows. We introduce Virtual Data Space (VDS), an abstraction of the data in a workflow that hides the complexities of the underlying storage system while allowing users to control data management strategies. We evaluate the data management strategies with real scientific and synthetic workflows, and demonstrate the capabilities of MaDaTS. Our experiments demonstrate the flexibility, performance and scalability gains of MaDaTS as compared to the traditional approach of managing data in scientific workflows.

Ghoshal, Devarshi↗
NASA NTRS1986

Data Management System (DMS) testbed user's manual development, volumes 1 and 2

A critical review of the network communication services contained in the Tinman User's Manual for Data Management System Test Bed (Tinman DMS User's Manual) is presented. The review is from the perspective of applying modern software engineering principles and using the Ada language effectively to ensure the test bed network communication services provide a robust capability. Overall the material on network communication services reflects a reasonably good grasp of the Ada language. Language features are appropriately used for most services. Design alternatives are offered to provide improved system performance and a basis for better application software development. Section two contains a review and suggests clarifications of the Statement of Policies and Services contained in Appendix B of the Tinman DMS User's Manual. Section three contains a review of the Network Communication Services and section four contains concluding comments.

Mcbride, John G.↗
NASA NTRS1992

Space Station Freedom data management system growth and evolution report

The Information Sciences Division at the NASA Ames Research Center has completed a 6-month study of portions of the Space Station Freedom Data Management System (DMS). This study looked at the present capabilities and future growth potential of the DMS, and the results are documented in this report. Issues have been raised that were discussed with the appropriate Johnson Space Center (JSC) management and Work Package-2 contractor organizations. Areas requiring additional study have been identified and suggestions for long-term upgrades have been proposed. This activity has allowed the Ames personnel to develop a rapport with the JSC civil service and contractor teams that does permit an independent check and balance technique for the DMS.

Bartlett, R.↗
NASA NTRS1976

Program on Earth Observation Data Management Systems (EODMS), appendixes

The needs of state, regional, and local agencies involved in natural resources management in Illinois, Iowa, Minnesota, Missouri, and Wisconsin are investigated to determine the design of satellite remotely sensed derivable information products. It is concluded that an operational Earth Observation Data Management System (EODMS) will be most beneficial if it provides a full range of services - from raw data acquisition to interpretation and dissemination of final information products. Included is a cost and performance analysis of alternative processing centers, and an assessment of the impacts of policy, regulation, and government structure on implementing large scale use of remote sensing technology in this community of users.

Eastwood, L. F., Jr.↗
NASA NTRS1978

A data management system for weight control and design-to-cost

The definition of the mass properties data of aircraft changed on a daily basis as do design details of the aircraft. This dynamic nature of the definition has generally encouraged those responsible for the data to update the data on a weekly or monthly basis. The by-product of these infrequent updates was the requirement of manual records to maintain daily activity. The development of WAVES changed the approach to management of mass properties data. WAVES has given the ability to update the data on a daily basis thereby eliminating the need for manual records. WAVES has demonstrated that a software product can support a data management system for engineering data.

Bryant, J. C.↗
NASA NTRS1992

Data management in NOAA

NOAA has 11 terabytes of digital data stored on 240,000 computer tapes. There are an additional 100 terabytes (TB) of geostationary satellite data stored in digital form on specially configured SONY U-Matic video tapes at the University of Wisconsin. There are over 90,000,000 non-digital form records in manuscript, film, printed, and chart form which are not easily accessible. The three NOAA Data Centers service 6,000 requests per year and publish 5,000 bulletins which are distributed to 40,000 subscribers. Seventeen CD-ROM's have been produced. Thirty thousand computer tapes containing polar satellite data are being copied to 12 inch WORM optical disks for research applications. The present annual data accumulation rate of 10 TB will grow to 30 TB in 1994 and to 100 TB by the year 2000. The present storage and distribution technologies with their attendant support systems will be overwhelmed by these increases if not improved. Increased user sophistication coupled with more precise measurement technologies will demand better quality control mechanisms, especially for those data maintained in an indefinite archive. There is optimism that the future will offer improved media technologies to accommodate the volumes of data. With the advanced technologies, storage and performance monitoring tools will be pivotal to the successful long-term management of data and information.

Callicott, William M.↗
DOE OSTI2017

Design of the protoDUNE raw data management infrastructure

The Deep Underground Neutrino Experiment (DUNE) will employ a set of Liquid Argon Time Projection Chambers (LArTPC) with a total mass of 40 kt as the main components of its Far Detector. In order to validate this technology and characterize the detector performance at full scale, an ambitious experimental program (called “protoDUNE”) has been initiated which includes a test of the large-scale prototypes for the single-phase and dual-phase LArTPC technologies, which will run in a beam at CERN. The total raw data volume that is slated to be collected during the scheduled 3-month beam run is estimated to be in excess of 2.5 PB for each detector. This data volume will require that the protoDUNE experiment carefully design the DAQ, data handling and data quality monitoring systems to be capable of dealing with challenges inherent with peta-scale data management while simultaneously fulfilling the requirements of disseminating the data to a worldwide collaboration and DUNE associated computing sites. Here in this paper, we present our approach to solving these problems by leveraging the design, expertise and components created for the LHC and Intensity Frontier experiments into a unified architecture that is capable of meeting the needs of protoDUNE.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1971

ALSEP data management.

Nine million measurements a day are transmitted from each functional ALSEP on the moon and continuously collected on earth for use in long-term scientific analysis. Periodically, this massive flow of intricate data is examined on a real-time basis to evaluate ALSEP performance and to study the need for new commands to any of the ALSEP systems. A worldwide complex of facilities and personnel has the challenging task of efficient and timely data processing and dissemination. This paper describes facility characteristics and support activities and details certain constraints imposed on ALSEP hardware design by data handling requirements. The highlights of the first two years of lunar data management are reviewed.

Murtaugh, C. R.↗
DOE OSTI2018

Toward Transparent Data Management in Multi-layer Storage Hierarchy for HPC Systems

Upcoming exascale high performance computing (HPC) systems are expected to comprise multi-tier storage hierarchy, and thus will necessitate innovative storage and I/O mechanisms. Traditional disk and block-based interfaces and file systems face severe challenges in utilizing capabilities of storage hierarchies due to the lack of hierarchy support and semantic interfaces. Object-based and semantically-rich data abstractions for scientific data management on large scale systems offer a sustainable solution to these challenges. Such data abstractions can also simplify users involvement in data movement. Here, we take the first steps of realizing such an object abstraction and explore storage mechanisms for these objects to enhance I/O performance, especially for scientific applications. We explore how an object-based interface can facilitate next generation scalable computing systems by presenting the mapping of data I/O from two real world HPC scientific use cases: a plasma physics simulation code (VPIC) and a cosmology simulation code (HACC). Our storage model stores data objects in different physical organizations to support data movement across layers of memory/storage hierarchy. Our implementation sclaes well to 16K parallel processes, and compared to the state of the art, such as MPI-IO and HDF5, our object-based data abstractions and data placement strategy in multi-level storage hierarchy achieves up to 7 X I/O performance improvement for scientific data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A customizable data management framework for high-repetition-rate high-energy-density science

The high-energy-density (HED) physics community is moving toward a new paradigm of high-repetition-rate (HRR) operation. To fully leverage the scientific power of HRR HED facilities, all of the components of each subsystem (laser, targetry, and performance diagnostics) must be connected and synchronized in a reliable and robust manner while the data acquired are tagged and archived in real time. To this end, GA has begun developing a generalized NoSQL-database framework, the MongoDB repository for information and archiving. An organizational strategy has been developed that shifts HED data organization from a shot-based to a diagnostic-based approach in order to increase archival and retrieval efficiency that lends itself to optimization applications. This work is a first step in pushing HRR HED science toward data management solutions that emphasize machine actionability and aim to stimulate community engagement to define data standards in HED science.

Instruments & Instrumentation↗
NASA NTRS1993

Modeling the data management system of Space Station Freedom with DEPEND

Some of the features and capabilities of the DEPEND simulation-based modeling tool are described. A study of a 1553B local bus subsystem of the Space Station Freedom Data Management System (SSF DMS) is used to illustrate some types of system behavior that can be important to reliability and performance evaluations of this type of spacecraft. A DEPEND model of the subsystem is used to illustrate how these types of system behavior can be modeled, and shows what kinds of engineering and design questions can be answered through the use of these modeling techniques. DEPEND's process-based simulation environment is shown to provide a flexible method for modeling complex interactions between hardware and software elements of a fault-tolerant computing system.

Olson, Daniel P.↗
NASA NTRS1987

An overview of the Hubble Space Telescope command and data management system

The Hubble Space Telescope (HST) will be an orbiting observatory which will extend the capability of large earth-based observatories to view the universe by approximately 350 times in volume and approximately seven times in distance. The HST program, which includes the Orbiting Observatory, the Space Telescope Science Institute, and the Space Telescope Operations Control Center, is supported by the Space Shuttle, the Tracking and Data Relay Satellite System, and the NASA Communications Network. This paper describes the Command and Data Management System for the HST spacecraft, including flexibility, operation modes, and end-to-end data flow paths used to meet stringent synchronization and data quality requirements. The results of system tests indicate that the use of concatenated coding techiques consisting of both block and convolutional codes as a protection against random and burst errors has been successful. Diagrams of the HST data system are included.

Clubb, Jerry↗
DOE OSTI2018

Scalable Data Management, Analysis, and Visualization (SDAV) Institute

Analytics-driven data layout optimizations and compression (NCSU): We have developed solutions that address challenges related to scientific data indexing, compression, and storage layout problems. Moreover, in addition to focusing on mainstream uniform mesh data, we have also addressed Adaptive Mesh Refinement (AMR) data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1997

Data Management System

CENTRA 2000 Inc., a wholly owned subsidiary of Auto-trol technology, obtained permission to use software originally developed at Johnson Space Center for the Space Shuttle and early Space Station projects. To support their enormous information-handling needs, a product data management, electronic document management and work-flow system was designed. Initially, just 33 database tables comprised the original software, which was later expanded to about 100 tables. This system, now called CENTRA 2000, is designed for quick implementation and supports the engineering process from preliminary design through release-to-production. CENTRA 2000 can also handle audit histories and provides a means to ensure new information is distributed. The product has 30 production sites worldwide.

Source record↗
DOE OSTI2015

AsyncStageOut: Distributed User Data Management for CMS Analysis

AsyncStageOut (ASO) is a new component of the distributed data analysis system of CMS, CRAB, designed for managing users' data. It addresses a major weakness of the previous model, namely that mass storage of output data was part of the job execution resulting in inefficient use of job slots and an unacceptable failure rate at the end of the jobs. ASO foresees the management of up to 400k files per day of various sizes, spread worldwide across more than 60 sites. It must handle up to 1000 individual users per month, and work with minimal delay. This creates challenging requirements for system scalability, performance and monitoring. ASO uses FTS to schedule and execute the transfers between the storage elements of the source and destination sites. It has evolved from a limited prototype to a highly adaptable service, which manages and monitors the user file placement and bookkeeping. To ensure system scalability and data monitoring, it employs new technologies such as a NoSQL database and re-uses existing components of PhEDEx and the FTS Dashboard. We present the asynchronous stage-out strategy and the architecture of the solution we implemented to deal with those issues and challenges. The deployment model for the high availability and scalability of the service is discussed. The performance of the system during the commissioning and the first phase of production are also shown, along with results from simulations designed to explore the limits of scalability.

Riahi, H.↗