Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

An approach for management of geometry data

The strategies for managing Integrated Programs for Aerospace Design (IPAD) computer-based geometry are described. The computer model of geometry is the basis for communication, manipulation, and analysis of shape information. IPAD's data base system makes this information available to all authorized departments in a company. A discussion of the data structures and algorithms required to support geometry in IPIP (IPAD's data base management system) is presented. Through the use of IPIP's data definition language, the structure of the geometry components is defined. The data manipulation language is the vehicle by which a user defines an instance of the geometry. The manipulation language also allows a user to edit, query, and manage the geometry. The selection of canonical forms is a very important part of the IPAD geometry. IPAD has a canonical form for each entity and provides transformations to alternate forms; in particular, IPAD will provide a transformation to the ANSI standard. The DBMS schemas required to support IPAD geometry are explained.

Dube, R. P.↗

Power System Waveform Datasets for Machine Learning

The desire for increased visibility across the electricity grid will necessarily increase the deployment of sensing and measurement devices and associated data management needs to unprecedented levels. For the existing sensing and measurement infrastructure, there remains a great amount of “value” yet to be extracted through advanced data management and analytics. Availability of more data will not, by itself, lead to changes in grid visibility, security, and resiliency. To create the predictive and prescriptive environment required to enable new markets and transactions for customer revenue and a reliable grid, the data must be collected, organized, evaluated, and analyzed using sophisticated algorithms to provide actionable information allowing operators and customers to reliably manage an increasingly complex grid. Progress in artificial intelligence (AI) has been largely driven by large, publicly available datasets that can be used to train AI algorithms such as MNIST, a database of handwritten images of digits, and ImageNet, an image database of everyday objects. These types of publicly available databases of real-world training datasets have been largely credited for advancement of image processing, computer vision, and deep learning algorithms that these use cases deploy. However, in the power systems industry to date, there are few databases with proper event labeling, and data access to a publicly available collection of power system event waveforms that will allow users to interact with grid signature data. Publicly available datasets of power system event waveforms, such as the DOE/EPRI dataset, often lack critical metadata or contain limited examples of each event type, and data formats vary widely across these datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗

Biological Research and Space Health Enabled by Machine Learning to Support Deep Space Missions

A key science goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth independence and autonomy of mission operations. Here we present an overview of AI/ML architecture to support deep space mission goals, developed with leaders in the field. First, we focus on the fundamental biological research that supports our understanding of physiological responses to spaceflight, and we describe current efforts to support AI/ML research including data standardization and data engineering through maximally open and FAIR (findable, accessible, interoperable, reusable) databases and the generation of AI-ready datasets for reuse and analysis. We also discuss remote data management frameworks for research data as well as environmental and health data that are generated during deep space missions. We highlight several research projects that leverage data standardization and management for fundamental biological discovery to uncover the complex effects of space travel on living systems. Next, we provide an overview of cutting-edge AI/ML approaches that can be integrated to support remote monitoring and analysis during deep space missions, including generative models and large language models to learn the underlying biomedical patterns and predict outcomes or answer questions during off world medical scenarios. We also describe current AI/ML methods to support this research and monitoring through automated cloud-based labs which enable limited human intervention and closed-loop experimentation in remote settings. These labs could support mission autonomy by analyzing environmental data streams, and would be facilitated through in situ analytics capabilities to avoid sending large raw data files through low bandwidth communications. Finally, in the context of deep space missions with limited communications or access to medical advice from Earth, we describe a solution for integrated, real-time mission biomonitoring across hierarchical levels from continuous environmental monitoring, to wearables and point-of-care devices, to molecular and physiological monitoring. We introduce a precision space health system that will ensure that the future of space health is predictive, preventative, participatory and personalized.

artificial intelligence↗

OCEANIDS: Autonomous Data Acquisition, Management and Distribution System

OCEANIDS is a clearinghouse for mission essential and near-real-time satellite data streams. This viewgraph presentation describes this mission, and includes the following topics: 1) OCEANIDS Motivation; 2) High-Level Architecture; 3) OCEANIDS Features; 4) OCEANIDS GUI: Nodes; 5) OCEANIDS GUI: Cluster; 6) Data Streams; 7) Statistics; and 8) GHRSST-PP.

automated monitoring↗

Modifying the Heliophysics Data Policy to Better Enable Heliophysics Research

The Heliophysics (HP) Science Data Management Policy, adopted by HP in June 2007, has helped to provide a structure for the HP data lifecycle. It provides guidelines for Project Data Management Plans and related documents, initiates Resident Archives to maintain data services after a mission ends, and outlines a route to the unification of data finding, access, and distribution through Virtual observatories. Recently we have filled in missing pieces that assure more coherence and a home for the VxOs (through the 'Heliophsyics Data and Model Consortium'), and provide greater clarity with respect to long term archiving. In particular, the new policy which has been vetted with many community members, details the 'Final Archives' that are to provide long-term data access. These are distinguished from RAs in that they provide little additional service beyond servicing data, but critical to their success is that the final archival materials include calibrated data in useful formats such as one finds in CDAWeb and various ASCII or FITS archives. Having a clear goal for legacy products, to be detailed as part of the Mission Archives Plans presented at Senior Reviews, will help to avoid the situation so common in the past of having archival products that preserve bits well but not readily usable information. We hope to avoid the need for the large numbers of 'data upgrade' projects that have been necessary in recent years.

Hayes, Jeffrey↗

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. iDDS has been designed to intelligently orchestrate workflow and data management systems, decoupling data pre-processing, delivery, and main processing in various workflows. It is an experiment-agnostic service around a workflow-oriented structure to work with existing and emerging use cases in ATLAS and other experiments. Here we will present the motivation for iDDS, its design schema and architecture, use cases and current status, and plans for the future.

97 MATHEMATICS AND COMPUTING↗

Data Transfer Tool (HPC Annual Report V.3.1)

Sandia has been developing and supporting data transfer tools for over 20 years and has the expertise to take DOE into the Extreme Scale era. In looking at Exascale and beyond (Extreme Scale Computing), data sets can be thousands of 500TBs in size, a single file can be in the 100TB range, and billions of files are expected. Huge bursts of data need to be transferred, even today. While data archiving is often not thought about, it is an integral part of the full data management path when data is generated on HPC systems. In order to move generated data to its final resting place (data archive) or to transfer between file systems, a capable data transfer tool is required.

97 MATHEMATICS AND COMPUTING↗

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. It has been designed to intelligently orchestrate workflows and data management systems, decoupling data pre-processing, delivery, and primary processing in large scale workflows. It is an experiment-agnostic service that has been deployed to serve data carousel (orchestrating efficient processing of tape-resident data), machine learning hyperparameter optimization, active learning, and other complex multi-stage workflows defined via DAG (Directed Acyclic Graph), CWL (Common Workflow Language) and other descriptions, including a growing number of analysis workflows. We will at first introduce some deployed use cases in a summary. Then we will focus on new improvements and use cases under developments in ATLAS, Rubin Observatory and sPHENIX, together with future efforts.

97 MATHEMATICS AND COMPUTING↗

Quo vadis, remote sensing

The use of satellite remote sensing data for resource management is discussed. The evaluation of the need for management data is reviewed, and some legislative programs which require the monitoring of environmental resources are summarized. Several characteristics of data used in the monitoring of dynamic processes are analyzed, and the implications of routine processing of extensive remote sensing data for the development of a new world view are considered.

Billingsley, F. C.↗

Techniques for the creation of land use maps and tabulations from Landsat imagery

Methods for creating color thematic maps and land use tabulations, employing both Landsat imagery and computer image processing, are discussed. The system, the Multiple Input Land Use System (MILUS) has been tested in the metropolitan section of Dayton, Ohio. Training areas for land use were first digitized by coordinates and then transformed onto an image of white lines on a black background. This image was added to a Landsat image of the same area. Then multispectral classification was performed. A tape of digitized census tract boundaries was computer interfaced to yield an image of tract boundaries on a background registered to the thematic land-use map. Using a data management system, the data were then used to produce figures for the area and percent of land use in each tract. Future work is expected to convert most of the steps into interactive processing. This would greatly reduce the time needed to edit and register the data sets.

Angelici, G. L.↗

A Simplified Shuttle Payload Thermal Analyzer /SSPTA/ program

A simple thermal analysis program for Space Shuttle payloads has been developed to accommodate the user who requires an easily understood but dependable analytical tool. The thermal analysis program includes several thermal subprograms traditionally employed in spacecraft thermal studies, a data management system for data generated by the subprograms, and a master program to coordinate the data files and thermal subprograms. The language and logic used to run the thermal analysis program are designed for the small user. In addition, analytical and storage techniques which conserve computer time and minimize core requirements are incorporated into the program.

Bartoszek, J. T.↗

National meeting to review IPAD status and goals

A joint NASA/industry project called Integrated Programs for Aerospace-vehicle Design (IPAD) is described, which has the goal of raising aerospace-industry productivity through the application of computers to integrate company-wide management of engineering data. Basically a general-purpose interactive computing system developed to support engineering design processes, the IPAD design is composed of three major software components: the executive, data management, and geometry and graphics software. Results of IPAD activities include a comprehensive description of a future representative aerospace vehicle design process and its interface to manufacturing, and requirements and preliminary design of a future IPAD software system to integrate engineering activities of an aerospace company having several products under simultaneous development.

Fulton, R. E.↗

LANDSAT-D: The new era of earth resources survey

The history of the development and utilization of LANDSAT satellites is summarized. Objectives of the LANDSAT D mission are listed and the capabilities of the 4-channel multispectral band scanner and the 7-channel radiometer (thematic mapper) to be carried on the satellite are described. Satellite components are illustrated and diagrams show the ground segment data management system, the data assessment system, and the operations control center. The flight segment of the mission and flight segment specifications are also described. Examples of digitally analyzed and enhanced imagery are included.

Source record↗

U.S. data processing for the IRAS project

The JPL's Scientific Data Analysis System (SDAS), which will process IRAS data and produce a catalogue of perhaps a million infrared sources in the sky, as well as other information for astronomical records, is described. The purposes of SDAS are discussed, and the major SDAS processors are shown in block diagram. The catalogue processing is addressed, mentioning the basic processing steps which will be applied to raw detector data. Signal reconstruction and conversion to astrophysical units, source detection, source confirmation, data management, and survey data products are considered in detail.

Duxbury, J. H.↗

Proceedings of the Second Pilot Climate Data System Workshop

The proceedings of the workshop held on January 29 and 30, 1986 are discussed. Data management, satellite radiance data, clouds, ultraviolet flux variations in the upper atmosphere, rainfall during El Nino events, and the use of optical disks are among the topics covered.

Source record↗

Science on the Space Station: The opportunity and the challenge - A NASA view

The Science and Applications Information System (SAIS) which will provide users with remote, interactive, and iterative access from their home locations to the many information system services is introduced. Such aspects as transaction management, standard format data units, and the use of intelligent data management systems are discussed. It is concluded that new and innovative approaches to information management must be developed so that the full potential of the Space Station can be realized with respect to the demands of the entire science community.

Devirian, Michael↗

Telescience, an operational approach to science investigation

The NASA Science and Applications Information System, which is based on telescience and must provide remote interaction between information system services in space and on the ground, is discussed. An infrastructure of networked facilities and institutionally provided support services is being developed. The technologies involved with providing telescience capability are examined, including automated data management services, new data acquisition systems, user support environment for system access, and the capability to access heterogeneous data bases and computational facilities from remote locations.

Weiss, James R.↗