SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

All sources NASA NTRS DOE OSTI

NASA NTRS2018

The Planetary Materials Database

NASA provides funds for a variety of research programs whose principal focus is to collect and analyze terrestrial analog materials. These data are used to (1) understand and interpret planetary geology; (2) identify and characterize habitable environments and pre-biotic/biotic processes; (3) interpret returned data from present and past missions; and (4) evaluate future mission and instrument concepts prior to selection for flight. Data management plans are now required for these programs, but the collected data are still not generally available to the community. There is also little possibility to re-analyze the collected materials by other techniques, since there is no requirement to archive collected samples. The Planetary Materials Database (PMD) is a central, high-quality, long-term data repository, which aims to promote the field of astrobiology and increase scientific returns from NASA funded research by enabling data sharing, collaboration and exposure of non-NASA scientists to NASA research initiatives and missions. The PMD is a linked collection of databases developed using the Open Data Repository (ODR) system. The PMD will include detailed descriptions of terrestrial analog planetary materials as well as data from the instruments used in their analysis. The goal is to provide example patterns/spectra/analyses, etc. and background information suitable for use by the Space Science community. An early example showing the utility of these databases (although not in the ODR format) is the RRUFF mineral database. RRUFF, comprising 4,000+ pure mineral standards, is the most popular and widely used dataset of minerals and receives more than 180,000 queries per week from geologists and mineralogists worldwide. The PMD will be patterned after the CheMin database [3], a resource that contains all of the data collected by the MSL CheMin XRD instrument on Mars. Raw and processed CheMin data can be viewed, downloaded, reprocessed and reanalyzed using cloud-based “applications” linked to the data.

Blake, David↗

NASA NTRS2019

A Unified Level of Service Model for NASA Earth Science Data Stewardship

During the past year, the Interagency Implementation and Concepts Team (IMPACT) reviewed existing service models in use at various NASA data centers in an effort to produce a unified, cohesive, and comprehensive Level of Service Model for all of NASA Distributed Active Archive Centers (DAACs). NASA DAACs are responsible for ensuring NASA Earth Science data are accurately and securely ingested, distributed, supported, and preserved. The term “Service" as used here refers to the spectrum of data management activities and outputs provided by DAACs in support of the data cared fo by each data cente. The unified Level-of- Service (LoS) model described in this presentation utilizes both the NASA-defined data product category and the data processing level to easily identify an appropriate level-of-service to be applied to a data product throughout the full data life cycle. This LoS model is to be used by DAACs when appraising incoming data in order to determine the appropriate and required services to provide. The LoS model utilizes a 3-level system in which services build upon previous levels and thereby require greater commitment and effort both on the part of the DAAC personnel and the data producer at the highest level. The LoS model description also contains examples of ways to communicate with data producers and data users what services can be expected, thereby bringing more consistent user experiences across the enterprise. In this presentation, we will outline the features of the LoS model and describe how it relates to the FAIR data practices and the NOAA Maturity Matrix model.

Smith, Deborah↗

NASA NTRSDate not supplied

Intelligent Change Detection System: Autonomous Intelligent Machine Agent Model Development

NASA’s significant role in facilitating the harmonious integration of unmanned aircraft systems (UAS), with other aerial vehicles operating in the National Airspace System (NAS), has revealed a need for more advanced technological tools than are being utilized currently. This technology would lend itself to significantly assisting Direct-Action Aviation Personnel (DAAP) with the ingress and egress of UAS operations within the NAS. Providing research findings that would reduce technical barriers, associated with UAS-NAS integration, has been a persistent effort by both NASA and the FAA. One such research effort, pursued by NASA’s Transformative Tools and Technologies – Revolutionary Aviation Mobility (T3-RAM) project, is the development of an autonomous intelligent machine (AIM) agent that would aid DAAP functioning as implemented in remote ground control stations (RGCS). This evolution of the “human-machine” symbiosis, within the aviation environment, is necessary for many reasons. For example, there are projections of large increases to the 864,000 registered UAS and 45,000 aviation operations taking place in the NAS each day. With a data output range from 1 to 20 terabytes per flight or each aerial vehicle, which is projected to have proportional rate increase to that of registered UASs. It is evident, that due to the projected increase of UASs and their generated data, the human-agent’s data managing capabilities will be quickly overwhelmed by the enormous amounts of data emanating in the NAS. The research efforts presented in this paper puts forward results from the development, assessment, and verification of a previously conceptualized AIM-Agent that combats actionable-data (information) errors resulting from the visual perception phenomenon known as “change blindness” (CB). CB has been identified as one of the main culprits of information erroring encountered within the ground control station operator (GCSO) community.

Change Blindness↗

DOE OSTI2021

Seamless integration of commercial Clouds with ATLAS Distributed Computing

The CERN ATLAS Experiment successfully uses a worldwide dis-tributed computing Grid infrastructure to support its physics programme at the Large Hadron Collider (LHC). The Grid workflow system PanDA routinely manages up to 700,000 concurrently running production and analysis jobs to process simulation and detector data. In total more than 500 PB of data are distributed over more than 150 sites in the WLCG and handled by the ATLAS data management system Rucio. To prepare for the ever growing data rate in future LHC runs new developments are underway to embrace industry accepted protocols and technologies, and utilize opportunistic resources in a standard way. This paper reviews how the Google and Amazon Cloud computing ser-vices have been seamlessly integrated as a Grid site within PanDA and Rucio. Performance and brief cost evaluations will be discussed. Such setups could offer advanced Cloud tool-sets and provide added value for analysis facilities that are under discussions for LHC Run-4.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Investigating Scientific Data Change with User Research Methods

Scientific datasets are continually expanding and changing due to fluctuations with instruments, quality assessment and quality control processes, and modifications to software pipelines. Datasets include minimal information about these changes or their effects requiring scientists manually assess modifications through a number of labor intensive and ad-hoc steps. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists systematically identify and make decisions around data changes. Currently, there is a lack of understanding, and common practices, for identifying and evaluating changes in datasets since systematically measuring and managing data change is under explored in scientific work. We are conducting user research to address this need by exploring scientist's conceptualizations, behaviors, needs, and motivations when dealing with changing datasets. Our user research utilizes multiple methods to produce foundational, generative insights and evaluate research products produced by our team. In this paper, we detail our user research process and outline our findings about data change that emerge from our studies. Our work illustrates how scientific software teams can push beyond just usability testing user interfaces or tools to better probe the underlying ideas they are developing solutions to address.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2025

A digital twin platform for building performance monitoring and optimization: Performance simulation and case studies

Advancements in sensor technology, data analytics, affordable compute, and communication infrastructure have paved the way for Digital Twin technology in optimizing building operations and controls. This study presents the development of an open and interoperable web-based Digital Twin platform for integrating diverse data streams and facilitating effective user interactions. The platform utilizes modern technologies for the web framework and time-series data management, ensuring scalability and responsiveness. The backend supports seamless integration of diverse data sources and emulators, incorporating data from building sensors and meters, external weather Application Programming Interfaces, and advanced EnergyPlus simulation models of the building and its energy systems including the Distributed Energy Resources that are formulated in Functional Mockup Units. A simulation case study was conducted with FlexLab, a test facility on Lawrence Berkeley National Laboratory campus. The case study includes normal operations, Distributed Energy Resource integration, and power outage scenarios, to illustrate the Digital Twin’s ability to provide critical insights into energy performance and thermal resilience. The results demonstrated the platform’s potential as a decision-support tool for optimizing building energy performance and enhancing resilience against extreme weather events. Future work will focus on deploying the Digital Twin platform to a real building for field validation, extending its capabilities to cover more scenarios such as bidirectional Electric Vehicle interactions, and enhancing user engagement.

EnergyPlus↗

DOE OSTI2024

HydraGNN v3.0

New or improved capabilities included in v3.0 release are as follows: 1. Enhancement in message passing layers through generalization of the class inheritance to enable the inclusion of a broader set of message passing policies Inclusion of equivariant message passing layers from the original implementations of: SchNet (https://pubs.aip.org/aip/jcp/article/148/24/241722/962591/SchNet-A-deep-learning-architecture-for-molecules); DimeNet++ (https://arxiv.org/abs/2011.14115); EGNN models (https://arxiv.org/pdf/2102.09844.pdf) 2. Restructuring of class inheritance for data management 3. Support of DDStore https://github.com/ORNL/DDStore capabilities for improved distributed data parallelism on large volumes of data that cannot fit on intra-node memory capacities 4. Large-scale system support for OLCF-Crusher and OLCF-Frontier

Lupo Pasini, Massimiliano [Oak Ridge National Labo↗

DOE OSTI2024

PIPES (Pipeline for Integrated Projects in Energy Systems) [SWR-24-89]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. https://github.com/nrel-pipes/pipes-api https://github.com/nrel-pipes/pipes-web https://github.com/nrel-pipes/nrel-pipes

Gu, Jianli↗

DOE OSTI2022

Data reduction through optimized scalar quantization for more compact neural networks

Raw data generation for several existing and planned large physics experiments now exceeds TB/s rates, generating untenable data sets in very little time. Those data often demonstrate high dimensionality while containing limited information. Meanwhile, Machine Learning algorithms are now becoming an essential part of data processing and data analysis. Those algorithms can be used offline for post processing and post data analysis, or they can be used online for real time processing providing ultra low latency experiment monitoring. Both use cases would benefit from data throughput reduction while preserving relevant information: one by reducing the offline storage requirements by several orders of magnitude and the other by allowing ultra fast online inferencing with low complexity Machine Learning models. Moreover, reducing the data source throughput also reduces material cost, power and data management requirements. In this work we demonstrate optimized nonuniform scalar quantization for data source reduction. This data reduction allows lower dimensional representations while preserving the relevant information of the data, thus enabling high accuracy Tiny Machine Learning classifier models for online fast inferences. We demonstrate this approach with an initial proof of concept targeting the CookieBox, an array of electron spectrometers used for angular streaking, that was developed for LCLS-II as an online beam diagnostic tool. We used the Lloyd-Max algorithm with the CookieBox dataset to design an optimized nonuniform scalar quantizer. Optimized quantization lets us reduce input data volume by 69% with no significant impact on inference accuracy. When we tolerate a 2% loss on inference accuracy, we achieved 81% of input data reduction. Finally, the change from a 7-bit to a 3-bit input data quantization reduces our neural network size by 38%.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

A robust approach to Gaussian process implementation

Abstract. Gaussian process (GP) regression is a flexible modeling technique used to predict outputs and to capture uncertainty in the predictions. However, the GP regression process becomes computationally intensive when the training spatial dataset has a large number of observations. To address this challenge, we introduce a scalable GP algorithm, termed MuyGPs, which incorporates nearest-neighbor and leave-one-out cross-validation during training. This approach enables the evaluation of large spatial datasets with state-of-the-art accuracy and speed in certain spatial problems. Despite these advantages, conventional quadratic loss functions used in the MuyGPs optimization, such as root mean squared error (RMSE), are highly influenced by outliers. We explore the behavior of MuyGPs in cases involving outlying observations and, subsequently, develop a robust approach to handle and mitigate their impact. Specifically, we introduce a novel leave-one-out loss function based on the pseudo-Huber function (LOOPH) that effectively accounts for outliers in large spatial datasets within the MuyGPs framework. Our simulation study shows that the LOOPH loss method maintains accuracy despite outlying observations, establishing MuyGPs as a powerful tool for mitigating unusual observation impacts in the large data regime. In the analysis of US ozone data, MuyGPs provides accurate predictions and uncertainty quantification, demonstrating its utility in managing data anomalies. Through these efforts, we advance the understanding of GP regression in spatial contexts.

Mukangango, Juliette↗

NASA NTRS1972

Engineering study for the functional design of a multiprocessor system

The results are presented of a study to generate a functional system design of a multiprocessing computer system capable of satisfying the computational requirements of a space station. These data management system requirements were specified to include: (1) real time control, (2) data processing and storage, (3) data retrieval, and (4) remote terminal servicing.

Miller, J. S.↗

NASA NTRS1972

A Preliminary Study of a Buoy System for Acquisition, Transmission, and Management of Hydrological Data Obtained from In-situ Measurements

The requirements for a system of remotely located, data collection buoys are considered first for a prototype system to be used in conjunction with the Earth Resources Technology Satellite (ERTS-A), and then for a more advanced system. The necessary sensor characteristics for compatibility with the ERTS-A data collection platforms are considered, as well as possible sites for location of the prototype buoys. The advanced system is considered from the standpoint of continuous data collection both through satellite data relay and ground telemetry systems. Management of the data from a buoy system is analyzed, especially with regard to the advanced system.

Elliott, J. M.↗

NASA NTRS1976

A microprocessor controlled pressure scanning system

A microprocessor-based controller and data logger for pressure scanning systems is described. The microcomputer positions and manages data from as many as four 48-port electro-mechanical pressure scanners. The maximum scanning rate is 80 pressure measurements per second (20 ports per second on each of four scanners). The system features on-line calibration, position-directed data storage, and once-per-scan display in engineering units of data from a selected port. The system is designed to be interfaced to a facility computer through a shared memory. System hardware and software are described. Factors affecting measurement error in this type of system are also discussed.

Anderson, R. C.↗

NASA NTRS1975

Command and data handling of science signals on Spacelab

The Orbiter Avionics and the Spacelab Command and Data Management System (CDMS) combine to provide a relatively complete command, control, and data handling service to the instrument complement during a Shuttle Sortie Mission. The Spacelab CDMS services the instruments and the Orbiter in turn services the Spacelab. The CDMS computer system includes three computers, two I/O units, a mass memory, and a variable number of remote acquisition units. Attention is given to the CDMS high rate multiplexer, CDMS tape recorders, closed circuit television for the visual monitoring of payload bay and cabin area activities, methods of science data acquisition, questions of transmission and recording, CDMS experiment computer usage, and experiment electronics.

Mccain, H. G.↗

NASA NTRS1976

A microprocessor controlled pressure scanning system

Anderson, R. C.↗

NASA NTRS1983

The UARS and open data concept and analysis study

Alternative concepts for a common design for the UARS and OPEN Central Data Handling Facility (CDHF) are offered. Costs for alternative implementations of the UARS designs are presented, showing that the system design does not restrict the implementation to a single manufacturer. Processing demands on the alternative UARS CDHF implementations are then discussed. With this information at hand together with estimates for OPEN processing demands, it is shown that any shortfall in system capability for OPEN support can be remedied by either component upgrades or array processing attachments rather than a system redesign. In addition to a common system design, it is shown that there is significant potential for common software design, especially in the areas of data management software and non-user-unique production software. Archiving the CDHF data are discussed. Following that, cost examples for several modes of communications between the CDHF and Remote User Facilities are presented. Technology application is discussed.

Mittal, M.↗

NASA NTRS1990

High rate science data handling on Space Station Freedom

A study by NASA's User Information System Working Group for Space Station Freedom (SSF) has determined that the proposed onboard Data Management System, as initially configured, will be incapable of handling the data-generation rates typical of numerous scientific sensor payloads; many of these generate data at rates in excess of 10 Mbps, and there are at least four cases of rates in excess of 300 Mbps. The SSF Working Group has accordingly suggested an alternative conceptual architecture based on technology expected to achieve space-qualified status by 1995. The architecture encompasses recorders with rapid data-ingest capabilities and massive storage capabilities, optical delay lines allowing the recording of only the phenomena of interest, and data flow-compressing image processors.

Handley, Thomas H., Jr.↗

NASA NTRS1991

Performance evaluation of Platform Data ManagementSystem under various degrees of protocol implementation

The Platform Data Management System (DMS) collects Housekeeping (H/K), Payload (P/L) Engineering, and Payload Science data from various subsystems and payloads on the platform for transmission to the ground through the downlink via TDRSS. The DMS also distributes command data received from the ground to various subsystems and payloads. In addition, DMS distributes timing and safemode data. The function of collection and distribution of various types of data is performed by the Command and Data Handling (C&DH) subsystem of DMS. The C&DH subsystem uses for this purpose a number of data buses namely, Housekeeping, Payload Engineering, Payload Science, and Time and Safemode buses. Out of these buses, the H/K, P/L Engineering, and P/L Science buses are planned to be implemented by using MIL-STD 1553 bus. Most of the period covered was spent in developing a queue theoretic model of the 1553 Bus as used in the DMS. The aim is to use this model to test the performance and suitability of the 1553 Bus to the DMS under a number of alternative design scenarios.

Arozullah, Mohammed↗