Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Overview of the Telescience Testbed Program

The NASA's Telescience Testbed Program (TTP) conducted by the Ames Research Center is described with particular attention to the objectives, the approach used to achieve these objectives, and the expected benefits of the program. The goal of the TTP is to gain operational experience for the Space Station Freedom and the Earth Observing System programs, using ground testbeds, and to define the information and communication systems requirements for the development and operation of these programs. The results of TTP are expected to include the requirements for the remote coaching, command and control, monitoring and maintenance, payload design, and operations management. In addition, requirements for technologies such as workstations, software, video, automation, data management, and networking will be defined.

Rasmussen, Daryl N.↗
NASA NTRS1991

Traditional anthropology and geographical information systems in the collaborative study of Cassava in Africa

Cross-cultural, village-level, and farmer surveys have been used with a geographical information system to describe the distribution and relative importance of cassava (manioc, yuca, Manihot esculenta) in its cultural, economic, and ecological contexts. It presents examples of data management for mapping, sample selection, cross-tabulation of characteristics, combination of data types for indices and hypothesis testing. The methods used are reviewed, and some of the main conclusions of the study are presented.

Romanoff, Steven↗
NASA NTRS1995

Selecting a general-purpose data compression algorithm

The National Space Science Data Center's Common Data Formate (CDF) is capable of storing many types of data such as scalar data items, vectors, and multidimensional arrays of bytes, integers, or floating point values. However, regardless of the dimensionality and data type, the data break down into a sequence of bytes that can be fed into a data compression function to reduce the amount of data without losing data integrity and thus remaining fully reconstructible. Because of the diversity of data types and high performance speed requirements, a general-purpose, fast, simple data compression algorithm is required to incorporate data compression into CDF. The questions to ask are how to evaluate and compare compression algorithms, and what compression algorithm meets all requirements. The object of this paper is to address these questions and determine the most appropriate compression algorithm to use within the CDF data management package that would be applicable to other software packages with similar data compression needs.

Mathews, Gary Jason↗
NASA NTRS2004

Assurance Technology Challenges of Advanced Space Systems

The initiative to explore space and extend a human presence across our solar system to revisit the moon and Mars post enormous technological challenges to the nation's space agency and aerospace industry. Key areas of technology development needs to enable the endeavor include advanced materials, structures and mechanisms; micro/nano sensors and detectors; power generation, storage and management; advanced thermal and cryogenic control; guidance, navigation and control; command and data handling; advanced propulsion; advanced communication; on-board processing; advanced information technology systems; modular and reconfigurable systems; precision formation flying; solar sails; distributed observing systems; space robotics; and etc. Quality assurance concerns such as functional performance, structural integrity, radiation tolerance, health monitoring, diagnosis, maintenance, calibration, and initialization can affect the performance of systems and subsystems. It is thus imperative to employ innovative nondestructive evaluation methodologies to ensure quality and integrity of advanced space systems. Advancements in integrated multi-functional sensor systems, autonomous inspection approaches, distributed embedded sensors, roaming inspectors, and shape adaptive sensors are sought. Concepts in computational models for signal processing and data interpretation to establish quantitative characterization and event determination are also of interest. Prospective evaluation technologies include ultrasonics, laser ultrasonics, optics and fiber optics, shearography, video optics and metrology, thermography, electromagnetics, acoustic emission, x-ray, data management, biomimetics, and nano-scale sensing approaches for structural health monitoring.

Chern, E. James↗
NASA NTRS2010

Terrestrial Hydrological Data from NASA's Hydrology Data and Information Services Center (HDISC): Products, Services, and Applications

Terrestrial hydrological variables are important in global hydrology, climate, and carbon cycle studies. The North American and Global Land Data Assimilation Systems (NLDAS and GLDAS, respectively) have been generating a series of land surface states (soil moisture, snow, and temperature) and fluxes (evapotranspiration, radiation, and heat flux) variables. These data, hosted at and available from NASA s Hydrology Data and Information Services Center (HDISC), include the NLDAS hourly 1/8 degree products and the GLDAS 3-hourly 0.25 and 1.0 degree products. HDISC provides easy access and visualization and analysis capabilities for these products, thus reducing the time and resources spent by scientists on data management and facilitating hydrological research. Users can perform spatial and parameter subsetting, data format transformation, and data analysis operations without needing to first download the data. HDISC is continually being developed as a data and services portal that supports weather and climate forecasts, and water and energy cycle research.

Fang, Hongliang↗
NASA NTRSDate not supplied

TPSAS-NF1676L-33992-DND

The CERES Science Team integrates and fuses observations from 6 CERES instruments aboard the Terra, Aqua, S-NPP, and NOAA-20 missions with data from more than 20 other unique data sources. Following the November 2017 launch of CERES Flight Model 6 (FM6) onboard NOAA20, CERES has now amassed over 80 instrument-years of valuable Earth radiation budget data. The rapidly growing volume of CERES data coupled with the introduction of new data products alongside improvements to existing science algorithms fosters the requirement for faster, more flexible, and scalable data production and orchestration. New virtualized, cloud-centric compute hardware hosted by the NASA Langley Research Center’s (LaRC) Atmospheric Sciences Data Center (ASDC) provides an ideal environment for these ever-increasing data production demands for CERES. This poster discusses updates to the implementation of the CERES Data Management Team’s (DMT) CERES AuTomAted job Loading sYSTem (CATALYST), a custom data processing workflow engine for CERES, to use on-demand computing resources to perform automated CERES data production processing in a Linux-based container environment. Linux containers provide CERES the flexibility to build multiple production environments in containers tailored for specific workloads and allow effortless provisioning of resources based on the CERES Science Team’s data production requirements.

Thomas N. Hillyer↗
NASA NTRSDate not supplied

Use NASA Earthdata in the Cloud

With the impending arrival of new, high-data-volume Earth observation missions, NASA’s ability to effectively ingest, process, and archive large amounts of data requires the most cost-effective, flexible, and scalable data-management architectures and technologies. To meet these demands, NASA's Earth Science Data Systems (ESDS) Program is implementing a strategic vision to develop and operate multiple components of NASA's Earth Observing System Data and Information System (EOSDIS) in a commercial cloud environment. As more earth observing data is moved into the cloud, NASA EOSDIS wants to share our reasoning for moving data into the commercial cloud, as well as demonstrate how end-users can take advantage of both data in the cloud and cloud-deployed services. This presentation explains the enterprise reasons for moving data into the cloud, specifically the scalability of cloud systems, the flexibility to pivot to the most optimal language/system/architecture, and the reduced redundancy of a common architecture. Then we primarily focus on the user-driven reasons for moving to the cloud. Equal access to high performance computing helps all parties, especially users that don't have access to their own on-premise computing infrastructure. Rapid deployment allows users to spin up their own systems without needing the extensive platform previously required for large scale development. Cost effectiveness allows users a wide breadth of computing options, from super computer ability to small scale development, which can help underserved communities. Finally we link to internal resources that users can utilize to begin their cloud based development journey without having to be an expert in the field.

Nicholas Doty↗
DOE OSTI2024

PyTrac

SAND2025-00635O PyTrac is a software tool that analyzes and visualizes PTRAC event files generated by MCNP 6.3. It converts these files into a graph network that makes it easier to interpret individual histories. The software includes command line tools for viewing the graph data structure in both 2D and 3D formats. PyTrac also integrates with MCNP to run simulations and manage data files. This provides a streamlined approach to analyzing and understanding the complex data generated by MCNP simulations. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Nowack, Aaron [Sandia National Lab. (SNL-CA), Live↗
NASA NTRS2004

Hierarchical Storage Management at the NASA Center for Computational Sciences: From UniTree to SAM-QFS

This paper presents the data management issues associated with a large center like the NCCS and how these issues are addressed. More specifically, the focus of this paper is on the recent transition from a legacy UniTree (Legato) system to a SAM-QFS (Sun) system. Therefore, this paper will describe the motivations, from both a hardware and software perspective, for migrating from one system to another. Coupled with the migration from UniTree into SAM-QFS, the complete mass storage environment was upgraded to provide high availability, redundancy, and enhanced performance. This paper will describe the resulting solution and lessons learned throughout the migration process.

Salmon, Ellen↗
NASA NTRS2022

An Advanced Open-Source Platform for Air Quality Analysis, Visualization, and Prediction

Ambient air pollution is the largest environmental health risk factor, leading to several million premature deaths globally per year. The challenge of combating poor air quality is exacerbated by growing urban populations, changing emissions, and a warming climate. While there have been many advances monitoring and modeling of atmospheric composition, reflected in the dramatic increase in archived Earth Observations, there is no single measurement or method that alone can provide an accurate depiction of the entire atmosphere. The rapidly growing collections of observational and modeling data require us to be smarter about what data to include, and how such data is used. In recent years, NASA has invested significantly in advancing the concepts for Analytics Collaborative Framework (ACF) [5] and New Observing Strategies (NOS) [4] to tackle our software infrastructure need for harmonized data management and dynamic acquisition of diverse measurements for on-demand, interactive, multivariate analysis, and access [3]. It is not enough to have a big data, standalone analytics solution; it is critical that we start integrating data from remote sensing, modeling, and in-situ networks in a harmonized manner that enables timely and data-driven decision-making for air quality management. This work presents the design and development of an Air Quality Analytics Collaborative Framework (AQ ACF), as part of NASA’s Advanced Information Systems Technology (AIST) effort, to establish a data, machine-learning, and numerically driven platform for air quality analysis, visualization, and prediction.

Liu, Qian↗
NASA NTRS1973

Utilization of remotely-sensed data in the management of inland wetlands

The author has identified the following significant results. ERTS-1 data and aerial photography are proving to be a useful tool for the inventory and management of inland wetlands. Two examples of the application of remotely-sensed data to specific wetland management needs or requirements are discussed. Studies of the Great Dismal Swamp are utilizing ERTS-1 imagery and color IR photography in: (1) study area selection; (2) field inspection; (3) vegetation mapping; (4) identification of drainage characteristics and moisture regime; (5) location of intensive study areas; and (6) detection of change. Thematic extractions of ERTS-1 data made using the United States Geological Survey's Autographic Theme Extraction System are aiding analyses of swamp hydrologic regime and providing information pertinent to quick recognition and inventory of wetlands from ERTS-1. DCP'S in south Florida wetlands provide near-real time data for water resources managers. Data relayed by satellite can be entered into models to provide predictive data and water storage information for long-term and short-term decision making.

Carter, V.↗
NASA NTRSDate not supplied

A PI’s Guide to Analog Research

Congratulation on being awarded a grant to conduct research in a spaceflight analog! NASA funded research in a Human Research Program (HRP) managed analog environment is very different from research in a traditional laboratory. This presentation will provide a guide to help investigators successfully plan studies for integration and implementation in an HRP analog platform. There are many things to consider when planning a study for implementation in an analog facility and campaign or mission. How do you get from grant approval to data analysis? This is where HRP’s Research Operations and Integration (ROI) team comes in. The ROI team works with sponsoring HRP elements and PI teams to capture requirements and help the PI teams through the phases of research complement development, integration and implementation. A complement is comprised of a group of studies requiring a common platform and/or scenario that are able to be integrated on a noninterference basis for implementation. Properly defining and documenting requirements and study needs is crucial in successful integration and implementation. This presentation will outline and provide the information needed for PI teams to understand the integration process and the role of the ROI team in the successful integration and implementation of their study. Topics to be discussed will include but not be limited to science requirement definition and documentation, NASA IRB submissions, subject recruitment, screening and selection, study integration process, efficiency in data collection and data sharing, daily crew schedules, hardware and software shipping, receipt and checkout, biological sample collection, mission support, data management and receipt of data.

B. Caldwell↗
NASA NTRS1983

RIMS: Resource Information Management System

An overview is given of the capabilities and functions of the resource management system (RIMS). It is a simple interactive DMS tool which allows users to build, modify, and maintain data management applications. The RIMS minimizes programmer support required to develop/maintain small data base applications. The RIMS also assists in bringing the United Information Services (UIS) budget system work inhouse. Information is also given on the relationship between the RIMS and the user community.

Symes, J.↗
NASA NTRS2003

Monitoring Outpatient Care

Each year, health care costs for managing chronically ill patients increase as the life expectancy of Americans continues to grow. To handle this situation, many hospitals, doctors practices, and home care providers are turning to disease management, a system of coordinated health care interventions and communications, to improve outpatient care. By participating in daily monitoring programs, patients with congestive heart failure, chronic obstructive pulmonary disease, diabetes, and other chronic conditions requiring significant self-care are facing fewer emergency situations and hospitalizations. Cybernet Medical, a division of Ann Arbor, Michigan-based Cybernet Systems Corporation, is using the latest communications technology to augment the ways health care professionals monitor and assess patients with chronic diseases, while at the same time simplifying the patients interaction with technology. Cybernet s newest commercial product for this purpose evolved from research funded by NASA, the National Institute of Mental Health, and the Advanced Research Projects Agency. The research focused on the physiological assessment of astronauts and soldiers, human performance evaluation, and human-computer interaction. Cybernet Medical's MedStar Disease Management Data Collection System is an affordable, widely deployable solution for improving in-home-patient chronic disease management. The system's battery-powered and portable interface device collects physiological data from off-the-shelf instruments.

Source record↗
DOE OSTI2023

Orchestration of materials science workflows for heterogeneous resources at large scale

In the era of big data, materials science workflows need to handle large-scale data distribution, storage, and computation. Any of these areas can become a performance bottleneck. We present a framework for analyzing internal material structures (e.g., cracks) to mitigate these bottlenecks. We demonstrate the effectiveness of our framework for a workflow performing synchrotron X-ray computed tomography reconstruction and segmentation of a silica-based structure. Our framework provides a cloud-based, cutting-edge solution to challenges such as growing intermediate and output data and heavy resource demands during image reconstruction and segmentation. Specifically, our framework efficiently manages data storage, scaling up compute resources on the cloud. The multi-layer software structure of our framework includes three layers. A top layer uses Jupyter notebooks and serves as the user interface. A middle layer uses Ansible for resource deployment and managing the execution environment. A low layer is dedicated to resource management and provides resource management and job scheduling on heterogeneous nodes (i.e., GPU and CPU). At the core of this layer, Kubernetes supports resource management, and Dask enables large-scale job scheduling for heterogeneous resources. The broader impact of our work is four-fold: through our framework, we hide the complexity of the cloud’s software stack to the user who otherwise is required to have expertise in cloud technologies; we manage job scheduling efficiently and in a scalable manner; we enable resource elasticity and workflow orchestration at a large scale; and we facilitate moving the study of nonporous structures, which has wide applications in engineering and scientific fields, to the cloud. While we demonstrate the capability of our framework for a specific materials science application, it can be adapted for other applications and domains because of its modular, multi-layer architecture.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Mic-hackathon 2024: hackathon on machine learning for electron and scanning probe microscopy

Microscopy is one of the primary sources of information on materials structure and functionality at the nanometer and atomic scales. The data generated through microscopy is often contained in well-structured datasets, enriched with extensive metadata and sample histories, although not always with the same level of detail or storage format. The broad incorporation of data management plans by major funding agencies ensures the preservation and accessibility of this data. However, deriving insights from these rich datasets remains challenging due to the lack of established code ecosystems, standardized benchmarks, and integration strategies. Correspondingly, the efficiency of data usage is very low, and time expenditures at the analysis stage are enormous. In addition to post-acquisition data analysis, the emergence of application programming interfaces by major microscope manufacturers now creates opportunities for real-time ML-based data analytics to enable automated decision making, and particularly ML-agent controlled real-time microscope operation. Despite these opportunities, there is a significant gap in integrating the ML community with the broader microscopy community, limiting the value that these methods bring to physics and materials discovery and materials optimization. Hackathons address these challenges by fostering collaboration between ML experts and microscopy professionals, encouraging the development of innovative solutions that leverage ML for microscopy and preparing the workforce of the future both for microscopy-intensive domains areas, instrument manufacturers, and ML scientists interested in real world applications for fundamental research, materials optimization, and manufacturing. The hackathon generated benchmark datasets and digital twins of microscopes that further contribute to the development of the field and establish data analysis ecosystems. All the codes can be found at GitHub(https://github.com/KalininGroup/Mic-hackathon-2024-codes-publication/tree/1.0.0.1) and Zenodo (https://zenodo.org/records/15579940).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package

Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.

58 GEOSCIENCES↗
DOE OSTI2022

Enhancing Risk Analysis of Accidental Release Using CFD Modeling and Machine Learning

This project report presents the results from the AU31 project that is entitled “Enhancing Risk Analysis of Accidental Release Using CFD Modeling and Machine Learning” performed between FY2021 and FY2022. The technical objective of this project is to enhance risk analysis of accidental release of radioactive materials and provide novel risk forecasting and capabilities for assessing unplanned hazardous chemical releases for the U.S. Department of Energy (DOE). The approach includes a cross-platform finite element analysis of airborne chemical release, machine learning (ML) for simulation and forecasting, and a big data approach to data management, transformation and analysis capable of handling petabyte-scale unstructured and structured data. The outcome of these findings were assembled and accessible as a final output via an intuitive web hosted geospatial-based user interface for demonstration and general dissemination. These new results also provide a mechanism of combining this knowledge in a risk analysis framework tool. The geospatial analysis and output can utilize 5 years of qualified meteorological data to illustrate projected plume footprints that are generated from computational aspects of this study. Specifically, we show the development of a geospatial risk analysis capabilities to respond to and plan for accidental release of radioactive materials and to provide a risk forecasting tool for assessing hazardous chemical releases for the DOE. The focus of this effort is germane to common practices in risk analysis; and it is highly relevant to the timely and transparent release of information related to materials at risk including hazardous materials and chemical vapors and rapid assessment of the potential impacts on the workforce at active sites across the DOE complex.

42 ENGINEERING↗