SEARCH · Engineering Papers

Results for “data and data science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

NASA NTRS2021

Calibration and Performance of Juno Radio Science Data

Juno Radio Science measures the frequency of X- and Ka-band radio links between the Juno spacecraft and the Earth-based observing stations of NASA’s Deep Space Network (DSN) in order to determine the gravitational field of Jupiter. The received frequency contains information on the gravitational field and is also perturbed by the propagation environment, including Earth troposphere and ionosphere, electrons in the solar plasma, electrons in the Io Plasma Torus around Jupiter, and instrumental effects on both the spacecraft and the ground electronics. Each of these effects must be calibrated out of the data to ensure an accurate estimation of the Jupiter gravitational field. This work discusses the data processing, calibration, and performance of the frequency measurements. The precision of the frequency measurements average 1.1 mHz (1σ standard deviation at 60 second integration time), or 5.3 microns/sec in units of velocity. The remaining noise sources are primarily from residual troposphere and charged particles. Further improvement could be made in future radio science experiments with the addition of a cross-link and stiffer ground antennas.

Oudrhiri, Kamal↗

NASA NTRS2017

Giovanni in the Cloud: Earth Science Data Exploration in Amazon Web Services

Giovanni is an exploration tool at the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), providing 22 analysis and visualization services for over 1600 Earth Science data variables. Owing to its popularity, Giovanni has experienced a consistent growth in overall demand, with periodic usage spikes attributed to trainings by education organizations, extensive data analysis in response to natural disasters, preparations for science meetings, etc. Furthermore, the new generation of spaceborne sensors and high resolution models have resulted in an exponential growth in data volume with data distributed across the traditional boundaries of data centers. Seamless exploration of data (without users having to worry about data center boundaries) has been a key recommendation of the GES DISC User Working Group. These factors have required new strategies for delivering acceptable performance. The cloud-based Giovanni, built on Amazon Web Services (AWS), evaluates (1) AWS native solutions to provide a scalable, serverless architecture; (2) open standards for data storage in the Cloud; (3) a cost model for operations; and (4) end-user performance. Our preliminary findings indicate that the use of serverless architecture has a potential to significantly reduce development and operational cost of Giovanni. The combination of using AWS managed services, storage of data in open standards, and schema-on-read data access strategy simplifies data access and analytics, in addition to making data more accessible to the end users of Giovanni through popular programming languages.

Giovanni↗

NASA NTRS1993

NASA space and Earth science data on CD-ROM

The National Space Science Data Center (NSSDC) is very interested in facilitating the widest possible use of the scientific data acquired through NASA spaceflight missions. Therefore, NSSDC has participated with projects and data management elements throughout the NASA science environment in the creation, archiving, and dissemination of data using Compact Disk-Read Only Memory (CD-ROM). This CD-ROM technology has the potential to enable the dissemination of very large data volumes at very low prices to a great many researchers, students and their teachers, and others. This catalog identifies and describes the scientific CD-ROM's now available from NSSDC including the following data sets: Einstein Observatory CD-ROM, Galileo Cruise Imaging on CD-ROM, International Halley Watch, IRAS Sky Survey Atlas, Infrared Thermal Mapper (IRTM), Magellan (MIDR), Magellan (ARCDR's), Magellan (GxDR's), Mars Digital Image Map (MDIM), Outer Planets Fields & Particles Data, Pre-Magellan, Selected Astronomical Catalogs, TOMS Gridded Ozone Data, TOMS Ozone Image Data, TOMS Update, Viking Orbiter Images of Mars, and Voyager Image.

Towheed, Syed S.↗

NASA NTRS1993

Multimission image processing and science data visualization

The Operational Science Analysis (OSA) Functional area supports science instrument data display, analysis, visualization and photo processing in support of flight operations of planetary spacecraft managed by the Jet Propulsion Laboratory (JPL). This paper describes the data products generated by the OSA functional area, and the current computer system used to generate these data products. The objectives on a system upgrade now in process are described. The design approach to development of the new system are reviewed, including use of the Unix operating system and X-Window display standards to provide platform independence, portability, and modularity within the new system, is reviewed. The new system should provide a modular and scaleable capability supporting a variety of future missions at JPL.

Green, William B.↗

NASA NTRS2015

Improving the Accessibility and Use of NASA Earth Science Data

Many of the NASA Langley Atmospheric Science Data Center (ASDC) Distributed Active Archive Center (DAAC) multidimensional tropospheric and atmospheric chemistry data products are stored in HDF4, HDF5 or NetCDF format, which traditionally have been difficult to analyze and visualize with geospatial tools. With the rising demand from the diverse end‐user communities for geospatial tools to handle multidimensional products, several applications, such as ArcGIS, have refined their software. Many geospatial applications now have new functionalities that enable the end user to: Store, serve, and perform analysis on each individual variable, its time dimension, and vertical dimension. Use NetCDF, GRIB, and HDF raster data formats across applications directly. Publish output within REST image services or WMS for time and space enabled web application development. During this webinar, participants will learn how to leverage geospatial applications such as ArcGIS, OPeNDAP and ncWMS in the production of Earth science information, and in increasing data accessibility and usability.

Tisdale, Matthew↗

NASA NTRS2000

Cassini/Huygens Program Archive Plan for Science Data

The purpose of this document is to describe the Cassini/Huygens science data archive system which includes policy, roles and responsibilities, description of science and supplementary data products or data sets, metadata, documentation, software, and archive schedule and methods for archive transfer to the NASA Planetary Data System (PDS).

NASA Planetary Science Data System↗

NASA NTRS2018

Enabling Communities in the Use and Application of NASA's Earth Science Data and Information Products: Interactive Webinars that Work

The vast active archives of NASA's Earth science data reflect the agency's ongoing commitment to producing the highest quality data, services and tools intended for a myriad of science and application uses worldwide. These data span the whole of Earth science domains, and most have direct uses in societal applications. As many internet-based tools mature they enable NASA to increase the availability to and, most importantly, the uses of these disparate data types. This poster/paper highlights one particularly effective tool the NASA Earth Science Data and Information System (ESDIS) Project is utilizing to increase our user communities' understanding of these data, services and tools and their varied uses. Webinars, or video-based, on-line, interactive discussions and presentations have become a staple for exchanging information via the internet. The NASA ESDIS Project is using this platform to create a rich, highly-available, source of cutting-edge insights into the uses of Earth science data and applications. As evidence, we present a representative sample of the webinar offerings created by ESDIS beginning in 2013.

Lindsay, Francis E.↗

NASA NTRSDate not supplied

MISR Level 1A CCD Science data, all cameras (MIL1A_V2)

The Level 1A data are raw MISR data that are decommutated, reformatted 12-bit Level 0 data shifted to byte boundaries, i.e., reversal of square-root encoding applied and converted to 16 bit, and annotated (e.g., with time information). These data are used by the Level 1B1 processing algorithm to generate calibrated radiances. The science data output preserves the spatial sampling rate of the Level 0 raw MISR CCD science data. CCD data are collected during routine science observations of the sunlit portion of the Earth. Each product represents one 'granule' of data. A 'granule' is defined to be the smallest unit of data required for MISR processing. Also, included in the Level 1A product are pointers to calibration coefficient files provided for Level 1B processing. [Location=GLOBAL] [Temporal_Coverage: Start_Date=2000-02-24; Stop_Date=] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180].

AM-1↗

NASA NTRSDate not supplied

MISR Level 1A CCD Science data, all cameras (MIL1A_V1)

CCD↗

NASA NTRS1966

Science data processing

Processing and storage of space science data from Mariner IV space probe

SPACE SCIENCE↗

NASA NTRS2018

The Advantages of Synergy-Quantitative Earth Science Data Visualization and Analysis with Giovanni, Panoply, and Excel

The NASA Giovanni data analysis system provides a multitude of basic analysis capabilities for numerous Earth science data products which are available in the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) archive, as well as for additional selected data products provided by other NASA Distributed Active Archive Center (DAAC) archives. In Giovanni, users can easily generate time-averaged data maps, area-averaged time-series, Latitude-Time and Longitude-Time Hovmoeller diagrams, correlation maps, accumulation maps, and map animations (22 analysis options are available in total). While ASCII text output is available for time-series plots, it is not included as an option for data maps. In order to provide a quantitative, easy-to-use numerical output in ASCII text form, the NetCDF file output from a Giovanni visualization is downloaded and then opened with the free NASA visualization software package Panoply. Panoply provides the capability of translating the Giovanni file into comma-separated-variable (CSV) output. Panoply also provides additional visualization options, including the facile calculation of difference maps and quasi-anomaly maps using Giovanni output files. The CSV files from Panoply can then be imported into an Excel spreadsheet, where an Excel macro converts the CSV files. The output consists of latitude-longitude-data value triads in text form for maps, and either longitude-time-data value or latitude-time-data value triads in text form for Hovmoeller diagrams. This presentation will explicate the basic procedure for the conversion, and then provide several examples where the procedure is applied to Giovanni output from different analysis options.

Earth science↗

NASA NTRS2008

The NPOESS Preparatory Project (NPP) Science Data Segment (SDS) Data Depository and Distribution Element (SD3E) System Architecture

The National Polar-orbiting Operational Environmental Satellite System (NPOESS), the U.S. Government's future low-Earth orbiting satellite system, will monitor global weather and environmental conditions. Serving as a risk reduction for NPOESS, the NPOESS Preparatory Project (NPP) will provide remotely sensed atmospheric, land, ocean, ozone, and sounder data that will serve the meteorological and global climate change scientific communities. The National Aeronautics and Space Administration (NASA) NPP Science Data Segment's (SDS) primary role is to independently assess the quality of the NPP science and environmental data records for their ability to support climate research. The SDS is composed of nine elements; an input element that receives data from the operational agencies and acts as a buffer, a calibration analysis element, five elements devoted to measurement based quality assessment, an element used to test algorithmic improvements, and an element that provides overall science direction. Each element requires a set of sensor specific science data products for their evaluation. There are four NPP sensors that will be flown on the NPP observatory. They are the Visible Infrared Imagining Radiometer Suite (VIIRS), the Advanced Technology Microwave Sounder (ATMS), the Cross-Track Infrared Sounder (CrIS), and the Ozone Mapper/Profiler Suite (OMPS). It is estimated that these four sensors combined will make daily data requests for approximately six terabytes of NPP science products from the operational data providers. As a result, issues associated with duplicate data requests, data transfers of large volumes of diverse products, and data transfer failures raised concerns with respect to the network traffic and bandwidth consumption. Therefore, a central data broker system for receiving and buffering data requests and data products for the SDS was developed. The data element for this system is called the SDS Data Depository and Distribution Element (SD3E). It supports science mission data assessment by assuring the timely and validated acquisition and subsequent transfer of the NPP Science Mission data to the SDS Elements and NPP Science Team. The six science elements that interface with the SD3E span across the NASA Goddard Space Flight Center (GSFC), the NASA Jet Propulsion Laboratory (JPL), and the University of Wisconsin. As the primary communication vehicle for the science elements and science team, the SD3E has an interface to the operational data providers: National Environment Satellite, Data, and Information Service (NESDIS) Interface Data Processing System (IDPS) and the National Oceanic Atmospheric Administration's (NOAA) Comprehensive Large Array-data Stewardship system (CLASS) Archive Data System (ADS), that are responsible for product generation and archive and distribution respectively. The SD3E is designed to be a semi-customizable and semi-automated system. This system is designed to provide flexibility and ease of use for the science users in accessing the latest data products by creating a rolling data cache that temporarily stores the products locally before transferring the data to the SDS Measurement based elements for the land, ocean, atmosphere, sounder, and ozone. This paper describes the design and architecture of one of the nine SDS elements, the SD3E, and how this system has provided a mechanism for efficient data exchange, how it has helped in alleviating some of the network traffic and usage, and how it has contributed to reducing operational costs.

Ho, Evelyn L.↗

DOE OSTI2020

Veridical data science

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, composed of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. As part of the PCS workflow, we develop PCS inference procedures, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others. Moreover, we demonstrate its favorable performance over existing methods in terms of receiver operating characteristic (ROC) curves in high-dimensional, sparse linear model simulations, including a wide range of misspecified models. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2008

Aquarius's Instrument Science Data System (ISDS) Automated to Acquire, Process, Trend Data and Produce Radiometric System Assessment Reports

The Aquarius Radiometer, a subsystem of the Aquarius Instrument required a data acquisition ground system to support calibration and radiometer performance assessment. To support calibration and compose performance assessments, we developed an automated system which uploaded raw data to a ftp server and saved raw and processed data to a database. This paper details the overall functionalities of the Aquarius Instrument Science Data System (ISDS) and the individual electrical ground support equipment (EGSE) which produced data files that were infused into the ISDS. Real time EGSEs include an ICDS Simulator, Calibration GSE, Labview controlled power supply, and a chamber data acquisition system. ICDS Simulator serves as a test conductor primary workstation, collecting radiometer housekeeping (HK) and science data and passing commands and HK telemetry collection request to the radiometer. Calibration GSE (Radiometer Active Test Source) provides source choice from multiple targets for the radiometer external calibration. Power Supply GSE, controlled by labview, provides real time voltage and current monitoring of the radiometer. And finally the chamber data acquisition system produces data reflecting chamber vacuum pressure, thermistor temperatures, AVG and watts. Each GSE system produce text based data files every two to six minutes and automatically copies the data files to the Central Archiver PC. The Archiver PC stores the data files, schedules automated uploads of these files to an external FTP server, and accepts request to copy all data files to the ISDS for offline data processing and analysis. Aquarius Radiometer ISDS contains PHP and MATLab programs to parse, process and save all data to a MySQL database. Analysis tools (MATLab programs) in the ISDS system are capable of displaying radiometer science, telemetry and auxiliary data in near real time as well as performing data analysis and producing automated performance assessment reports of the Aquarius Radiometer.

Source record↗

NASA NTRS2019

Federated Cloud Challenges in NASA's Earth Science Data Systems (Why So Difficult?)

NASA is presented with a number of opportunities and challenges in federating its Earth Science Data Systems in the burgeoning world of cloud computing. Cloud hosting of Earth Science data provides a new way of bringing data together, at least from a virtual location sense, and is one of the main motives for NASA to host data there. However, NASA is also faced with a Big Data Variety challenge, brought on by the variety of the EO datasets in its archives. This diversity requires many diverse science archives to service the different science communities. As a result, nearly every major function in the Earth Observing System Data and Information System (EOSDIS) must also be federated across its data centers. This pattern is repeated with many of the outside agencies and organizations that EOSDIS federates with, such as the Committee for Earth Observing Satellites, leading to pioneering work on "deep federation" in a joint project with the European Space Agency to develop a Multi-Mission Algorithm and Analysis Platform.

Lynnes, Christopher↗

DOE OSTI2024

Machine Learning and Data Science to Advance Laboratory Earthquake Prediction and Illuminate the Mechanics of Precursors to Failure

Earthquakes represent one of our greatest natural hazards and in recent years human induced seismicity is adding to the threat. Even a modest improvement in the ability to forecast devastating large earthquakes or smaller shallow events associated with fluid injection could save thousands of lives and billions of dollars. Current efforts to forecast earthquakes are limited by knowledge of earthquake physics and hampered by a lack of reliable lab or field observations. However, recent work has provided a critical opportunity for advancement. We have found: 1) clear and consistent precursors prior to earthquake-like failure in the laboratory and 2) that lab earthquakes can be predicted using machine learning (ML). These works show that stick-slip failure events –the lab equivalent of earthquakes– are preceded by a cascade of micro-failure events that radiate elastic energy in a manner that foretells catastrophic failure. Remarkably, ML predicts the fault zone stress state, the failure time and in some cases the magnitude of lab earthquakes. In addition, the observations include clear precursors to failure in the form of changes in fault zone properties prior to lab earthquakes. Precursors have been observed in previous laboratory studies but their origin is poorly understood and their possible connection to ML based earthquake prediction is unknown. The work conducted under our project has dramatically expanded these efforts. We have developed an integrated data science approach to illuminate the physics of earthquake precursors and lab earthquake prediction. Our work has accelerated the development of ML, artificial intelligence (AI), and related data science approaches by providing massive data sets that are tightly connected to critical scientific problems and by bringing together leading subject matter experts and data scientists. Earthquake physics involves phenomena that are far from equilibrium. Our work has leveraged data science methods to illuminate these phenomena and investigate how they relate to earthquake prediction. In addition to a large database with many types of labeled events that is available to everyone, our work has advanced the fundamental understanding of seismic forecasting, earthquake physics, and fault rheology

58 GEOSCIENCES↗

NASA NTRS2004

Science Data Processing for the Advanced Microwave Scanning Radiometer: Earth Observing System

The National Aeronautics and Space Administration established the framework for the Science Investigator-led Processing Systems (SIPS) to enable the Earth science data products to be produced by personnel directly associated with the instrument science team and knowledgeable of the science algorithms. One of the first instantiations implemented for NASA was the Advanced Microwave Scanning Radiometer - Earth Observing System (AMSR-E) SIPS. The AMSR-E SIPS is a decentralized, geographically distributed ground data processing system composed of two primary components located in California and Alabama. Initial science data processing is conducted at Remote Sensing Systems (RSS) in Santa Rosa, California. RSS ingests antenna temperature orbit data sets from JAXA and converts them to calibrated, resampled, geolocated brightness temperatures. The brightness temperatures are sent to the Global Hydrology and Climate Center in Huntsville, Alabama, which generates the geophysical science data products (e.g., water vapor, sea surface temperature, sea ice extent, etc.) suitable for climate research and applications usage. These science products are subsequently sent to the National Snow and Ice Data Center Distributed Active Archive Center in Boulder, Colorado for archival and dissemination to the at-large science community. This paper describes the organization, coordination, and production techniques employed by the AMSR-E SIPS in implementing, automating and operating the distributed data processing system.

Goodman, H. Michael↗

NASA NTRS2018

NASA GES DISC Earth Science Data Support

It is the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) mission statement to facilitate data access and evaluation, as well as scientific exploration and discovery. Recently, GES DISC has been evolving and improving our data management and services in order to promote GES DISC data to be easily discovered, improve usage and made more interoperable with common tools. As a result, we will present a brief review of our recent data services at the GES DISC including our new science-data driven website, subsetting and resampling services across multitude of satellite processing levels, visualization services, and how we utilize social media tools to interact with user communities.

data support↗