Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A Visual Analytics Framework for Reviewing Streaming Performance Data

Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.

Human-centered computing↗
NASA NTRS2018

Medical Data Architecture Project Status

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions. This gap identifies that the current in-flight medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are a variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable a more medically autonomous crew than the current paradigm. The medical system requirements are being developed in parallel with the exploration mission architecture and vehicle design. ExMC has recognized that in order to make informed decisions about a medical data architecture framework, current methods for medical data management must not only be understood, but an architecture must also be identified that provides the crew with actionable insight to medical conditions. This medical data architecture will provide the necessary functionality to address the challenges of executing a self-contained medical system that approaches crew health care delivery without assistance from ground support. Hence, the products supported by current prototype development will directly inform exploration medical system requirements.

medical data architecture↗
NASA NTRS2018

Medical Data Architecture (MDA) Project Status

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions. This gap identifies that the current in-flight medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are a variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable a more medically autonomous crew than the current paradigm. The medical system requirements are being developed in parallel with the exploration mission architecture and vehicle design. ExMC has recognized that in order to make informed decisions about a medical data architecture framework, current methods for medical data management must not only be understood, but an architecture must also be identified that provides the crew with actionable insight to medical conditions. This medical data architecture will provide the necessary functionality to address the challenges of executing a self-contained medical system that approaches crew health care delivery without assistance from ground support. Hence, the products supported by current prototype development will directly inform exploration medical system requirements.

medical data architecture↗
NASA NTRS2019

Medical Data Architecture (MDA) Project Status

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions. This gap identifies that the current in-flight medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are a variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable a more medically autonomous crew than the current paradigm. The medical system requirements are being developed in parallel with the exploration mission architecture and vehicle design. ExMC has recognized that in order to make informed decisions about a medical data architecture framework, current methods for medical data management must not only be understood, but an architecture must also be identified that provides the crew with actionable insight to medical conditions. This medical data architecture will provide the necessary functionality to address the challenges of executing a self-contained medical system that approaches crew health care delivery without assistance from ground support. Hence, the products supported by current prototype development will directly inform exploration medical system requirements.In fiscal year 2018, the MDA project developed Test Bed 2, the second iteration in a series of prototypes with functionality focused on data security through role-based access control and encryption, integration with One Portal exercise software and ingestion of an ultrasound Digital Imaging and Communications in Medicine (DICOM) file and image display. Test Bed 2 advances the medical data system architecture framework by providing these functionalities in a scalable system that maintained a layered, modular design. The architecture framework uses a data services approach with role-based access to data in a customized medical record system suitable for space exploration. These functionalities were demonstrated as part of the Next Space Technologies for Exploration Partnerships (NextSTEP) ground test demonstrated at the NASA Johnson Space Center Integrated Power, Avionics and Software (iPAS) facility. Interfacing to a Core Flight Software (CFS) system, the MDA system, using Consultative Committee for Space Data Systems (CCSDS) protocol, transferred an exercise file from the simulated flight MDA system to a mirrored MDA system on the ground through the CFS system. The selection of data sources and demonstrations enabled the team to address stakeholder concerns throughout the development process. In the next iteration, the MDA team will work with stakeholders to identify additional relevant functionalities to further advance system data models, standards and principles that will inform the medical system requirements development.

medical data architecture↗
NASA NTRS1986

Data reduction, management, and analysis software for CID

In an overview of the Data Reduction System, three major steps are examined. First, the raw data tapes were selected from the onboard recorders. These tapes should provide the best quality data for the data reduction software system. These tapes contained 352 channels of data, plus the monitor channels recorded in 8 bit Pulsed Coded Modulation (PCM) words. The next step consists of transcribing the PCM tapes from 8 bit serial digital data to 8 bit parallel digital data. This puts the data in the correct format for processing. The transcription process was accomplished here at LaRC in the Central Data Transportation Facility (CDTF). The last step in this 3 step process is to process the data through the reduction system developed for the Impact Dynamic Research Facility in the early part of 1980. Processing system criteria, system interface routines, and engineering units program that reads digitized data from tapes, and file management programs are discussed.

Davis, C. W.↗
NASA NTRS1979

Multipurpose interactive NASA information system

Multipurpose Interactive NASA Information System (MINIS) is data management system capable of retrieving descriptive data from LANDSAT photos. General enough to be used with other user-defined data bases, interactive data management and information retrieval system was especially developed for small and medium-sized computers. It uses free-form data base that allows one to create entirely new and different data bases and to control format of output products.

Hill, J. M.↗
NASA NTRSDate not supplied

NASA’s Commercial Smallsat Data Acquisition (CSDA) Program Data System

The National Aeronautic and Space Administration (NASA) Commercial Smallsat Data Acquisition (CSDA) Program was established to identify, evaluate, and acquire data from commercial satellite companies that support and complement NASA’s Earth sciences missions and research goals. Since becoming a sustained program in 2020, CSDA has been developing a data system to provide scalable, efficient, continuous, and repeatable data management processes for all commercial data acquired by NASA. This includes providing access to commercial data to approved science investigators through NASA developed and vendor operated interfaces. As the commercial data user community and volume of data acquired by CSDA has grown, the team managing these data has adapted technologies and services to better address the data and information search, discovery, and access needs of the science user community. This presentation will provide an overview of CSDA Program data system activities including data availability, development of CSDA user interfaces, challenges in managing large-volume diverse datasets, and long-term data preservation activities to retain data for scientific reproducibility.

data management↗
NASA NTRS1983

Results of data base management system parameterized performance testing related to GSFC scientific applications

The results of a variety of tests designed to demonstrate and evaluate the performance of several commercially available data base management system (DBMS) products compatible with the Digital Equipment Corporation VAX 11/780 computer system are summarized. The tests were performed on the INGRES, ORACLE, and SEED DBMS products employing applications that were similar to scientific applications under development by NASA. The objectives of this testing included determining the strength and weaknesses of the candidate systems, performance trade-offs of various design alternatives and the impact of some installation and environmental (computer related) influences.

Carchedi, C. H.↗
NASA NTRS2017

Medical Data Architecture Project Status

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions. This gap identifies that the current International Space Station (ISS) medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable an increasingly autonomous crew than the current ISS paradigm. The MDA will develop capabilities that support automated data collection, and the necessary functionality and challenges in executing a self-contained medical system that approaches crew health care delivery without assistance from ground support. To attain this goal, the first year of the MDA project focused on reducing technical risk, developing documentation and instituting iterative development processes that established the basis for the first version of MDA software (or Test Bed 1). Test Bed 1 is based on a nominal operations scenario authored by the ExMC Element Scientist. This narrative was decomposed into a Concept of Operations that formed the basis for Test Bed 1 requirements. These requirements were successfully vetted through the MDA Test Bed 1 System Requirements Review, which permitted the MDA project to begin software code development and component integration. This paper highlights the MDA objectives, development processes, and accomplishments, and identifies the fiscal year 2017 milestones and deliverables in the upcoming year.

Data management↗
NASA NTRS2017

Medical Data Architecture Project Status

The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions. This gap identifies that the current International Space Station (ISS) medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable an increasingly autonomous crew than the current ISS paradigm. The MDA will develop capabilities that support automated data collection, and the necessary functionality and challenges in executing a self-contained medical system that approaches crew health care delivery without assistance from ground support.To attain this goal, the first year of the MDA project focused on reducing technical risk, developing documentation and instituting iterative development processes that established the basis for the first version of MDA software (or Test Bed 1). Test Bed 1 is based on a nominal operations scenario authored by the ExMC Element Scientist. This narrative was decomposed into a Concept of Operations that formed the basis for Test Bed 1 requirements. These requirements were successfully vetted through the MDA Test Bed 1 System Requirements Review, which permitted the MDA project to begin software code development and component integration. This paper highlights the MDA objectives, development processes, and accomplishments, and identifies the fiscal year 2017 milestones and deliverables in the upcoming year.

Exploration medical system↗
NASA NTRS1983

Description of data base management systems activities

One of the major responsibilities of the JPL Computing and Information Services Office is to develop and maintain a JPL plan for providing computing services to the JPL management and administrative community that will lead to improved productivity. The CISO plan to accomplish this objective has been titled 'Management and Administrative Support Systems' (MASS). The MASS plan is based on the continued use of JPL's IBM 3032 Computer system for administrative computing and for the MASS functions. The current candidate administrative Data Base Management Systems required to support the MASS include ADABASE, Cullinane IDMS and TOTAL. Previous uses of administrative Data Base Systems have been applied to specific local functions rather than in a centralized manner with elements common to the many user groups. Limited capacity data base systems have been installed in microprocessor based office automation systems in a few Project and Management Offices using Ashton-Tate dBASE II. These experiences plus some other localized in house DBMS uses have provided an excellent background for developing user and system requirements for a single DBMS to support the MASS program.

Source record↗
DOE OSTI2019

DOE Data Day 2019 Report

The Department of Energy (DOE) has joined the larger scientific community in the promotion of data management as a means to higher quality, more efficient research and analysis, and as a critical component of data science. Tools and platforms to support data management and analysis are rapidly evolving and provide enormous opportunities. They also pose challenges that can be specific to DOE but are common across DOE mission areas and organizations. The DOE Data Day (D3) workshop was held on September 25–26, 2019, and organized by a team at Lawrence Livermore National Laboratory (LLNL) in an effort to gather the data management practitioners at the DOE labs together to share their work and results, facilitating knowledge transfers and best practices across project teams. Over 100 attendees participated from multiple national labs—LLNL, Sandia (SNL), Lawrence Berkeley (LBNL), Los Alamos (LANL), Oak Ridge (ORNL), Argonne (ANL), Pacific Northwest (PNNL), Stanford Linear Accelerator (SLAC)—along with Mission Support and Test Services (MSTS), National Energy Technology Laboratory (NETL), Pantex, National Security Complex (Y-12), National Nuclear Security Administration (NNSA), the DOE Office of Scientific and Technical Information (OSTI), and elsewhere in the DOE complex.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Automated Network Services for Exascale Data Movement

The Large Hadron Collider (LHC) experiments distribute data by leveraging a diverse array of National Research and Education Networks (NRENs), where experiment data management systems treat networks as a “blackbox” resource. After the High Luminosity upgrade, the Compact Muon Solenoid (CMS) experiment alone will produce roughly 0.5 exabytes of data per year. NREN Networks are a critical part of the success of CMS and other LHC experiments. However, during data movement, NRENs are unaware of data priorities, importance, or need for quality of service, and this poses a challenge for operators to coordinate the movement of data and have predictable data flows across multi-domain networks. The overarching goal of SENSE (The Software-defined network for End-to-end Networked Science at Exascale) is to enable National Labs and universities to request and provision end-to-end intelligent network services for their application workflows leveraging SDN (Software-Defined Networking) capabilities. This work aims to allow LHC Experiments and Rucio, the data management software used by CMS Experiment, to allocate and prioritize certain data transfers over the wide area network. In this paper, we will present the current progress of the integration of SENSE, Multi-domain end-to-end SDN Orchestration with QoS (Quality of Service) capabilities, with Rucio, the data management software used by CMS Experiment.

Balcas, Justas↗
NASA NTRS2018

A Conceptual Enterprise Framework for Managing Scientific Data Stewardship

Scientific data stewardship is an important part of long-term preservation and the use/reuse of digital research data. It is critical for ensuring trustworthiness of data, products, and services, which is important for decision-making. Recent U.S. federal government directives and scientific organization guidelines have levied specific requirements, increasing the need for a more formal approach to ensuring that stewardship activities support compliance verification and reporting. However, many science data centers lack an integrated, systematic, and holistic framework to support such efforts. The current business- and process-oriented stewardship frameworks are too costly and lengthy for most data centers to implement. They often do not explicitly address the federal stewardship requirements and/or the uniqueness of geospatial data. This work proposes a data-centric conceptual enterprise framework for managing stewardship activities, based on the philosophy behind the Plan-Do-Check-Act (PDCA) cycle, a proven industrial concept. This framework, which includes the application of maturity assessment models, allows for quantitative evaluation of how organizations manage their stewardship activities and supports informed decision-making for continual improvement towards full compliance with federal, agency, and user requirements.

Scientific data stewardship↗
NASA NTRS1985

The Space Flight Operations Center development project

This paper presents an overview of the Jet Propulsion Laboratory's (JPL) plan for developing a multi-mission facility called the Space Flight Operations Center (SFOC). The SFOC development is a five-year project with the objective of developing and adapting up-to-date computing, networking and data management technology to meet the future data processing and data management requirements for JPL's flight project mission operations. Presented are the project background, basic goals and objectives, design concept and project implementation approach.

Ebersole, M. M.↗
DOE OSTI2021

Athena: High-Performance Sparse Tensor Contraction Sequence on Heterogeneous Memory

Sparse tensor contraction sequence has been widely employed in many fields, such as chemistry and physics. However, how to efficiently implement the sequence faces multiple challenges, such as redundant computations and memory operations, massive memory consumption, and inefficient utilization of hardware. To address the above challenges, we introduce Athena, a high-performance framework for SpTC sequences. Athena introduces new data structures, leverages emerging Optane-based heterogeneous memory (HM) architecture, and stage parallelism. In particular, Athena introduces shared hash table-represented sparse accumulator to eliminate unnecessary input processing and data migration; Athena uses a novel data-semantic guided dynamic migration solution to make the best use of the Optane-based HM for high performance; Athena also co-runs execution phases with different characteristics to enable high hardware utilization. Evaluating with 12 datasets, we show that Athena brings 327-7362× speedup over the state-of-the-art SpTC algorithm. With the dynamic data placement guided by data semantics, Athena brings performance improvement on Optane-based HM over a state-of-the-art software-based data management solution, a hardware-based data management solution, and PMM-only by 1.58×, 1.82×, and 2.34× respectively.

Liu, Jiawen↗
NASA NTRS2014

Climate Analytics as a Service

Climate science is a big data domain that is experiencing unprecedented growth. In our efforts to address the big data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS). CAaaS combines high-performance computing and data-proximal analytics with scalable data management, cloud computing virtualization, the notion of adaptive analytics, and a domain-harmonized API to improve the accessibility and usability of large collections of climate data. MERRA Analytic Services (MERRA/AS) provides an example of CAaaS. MERRA/AS enables MapReduce analytics over NASA's Modern-Era Retrospective Analysis for Research and Applications (MERRA) data collection. The MERRA reanalysis integrates observational data with numerical models to produce a global temporally and spatially consistent synthesis of key climate variables. The effectiveness of MERRA/AS has been demonstrated in several applications. In our experience, CAaaS is providing the agility required to meet our customers' increasing and changing data management and data analysis needs.

big data↗