SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

All sources NASA NTRS DOE OSTI

NASA NTRS2020

Assessment of SNPP VIIRS RSB detector-to-detector differences using deep convective clouds and deserts

An accurate on-orbit characterization of the Visible Infrared Imaging Radiometer Suite (VIIRS) onboard the Suomi National Polar-orbiting Partnership satellite is essential to satisfy the requirements from NOAA, NASA, and the general research community for high-quality operational and research products. NASA’s Land Science Investigator-led Processing System (LSIPS) sensor data records (SDRs) are utilized to assess the stability of the calibrated top of atmosphere reflectance over the deep convective clouds (DCCs) and over the Libya-4desert site. The results from DCCs and desert show detector-to-detector (D2D) reflectance differences in the VIIRS reflective solar bands, 10 moderate-resolution bands (M-bands, M1to M5, M7 to M11), and three imagery resolution bands (I-bands, I1-I3). More importantly, the D2D differences exhibit an increase in magnitude over time for bands M1 to M5 and I1to I2, which induces noticeable striping and higher uncertainties in the downstream data products. The largest D2D difference is observed in bands M1 and M2, with magnitudes greater than1.8% in trends among the 16 detectors and standard deviations less than 0.12% over time. TheD2D stability assessment results over DCCs are consistent with those observed over the Libya-4desert site. To improve the quality of the calibrated SDR reflectance data, NASA’s VIIRS characterization support team has improved the calibration algorithm to incorporate corrections based on these DCC measurements to mitigate the striping (detector differences) observed in the current version of the SDR. These improvements are planned to be included in the next mission reprocessing of the LSIPS land products.

SNPP VIIRS↗

NASA NTRS1999

The Transport of Solar Ions Through the Earth's Magnetosphere

This report covers the initial phase of an investigation that was originally selected by NASA Headquarters for funding by a grant but was later transferred to NASA GSFC for continued funding under a new and separate contract. The principal objective of the investigation, led by Dr. O.W. Lennartsson, is to extract information about the solar origin plasma in Earth's magnetosphere, specifically about the entry and transport of this plasma, using energetic (10 eV/e to 18 keV/e) ion composition data from the Lockheed Plasma Composition Experiment on the NASA/ESA International Sun-Earth Explorer One (ISEE 1) satellite. These data were acquired many years ago, from November 1977 through March of 1982, but, because of subsequent failures of similar experiments on several other spacecraft, they are still the only substantial ion composition data available from Earth's magnetotail, beyond 10 R(sub E), in the critically important sub-kev to keV energy range. All of the Lockheed data now exist in a compacted scientific format, suitable for large-scale statistical investigations, which has been archived both at Lockheed Martin in Palo Alto and at the National Space Science Data Center (NSSDC) in Greenbelt. The completion of the archiving, by processing the remaining half of the data, was made possible by separate funding through a temporary NASA program for data restoration and was given priority over the data analysis by a no-cost extension of the subject grant. By chance, the period of performance coincided with an international study of source and loss processes of magnetospheric plasma, sponsored by the International Space Science Institute (ISSI) in Bern, Switzerland, for which Dr. Lennartsson was invited to serve as one of 12 co-chairs. This study meshed well with the continued analysis of the NASA/Lockheed ISEE ion composition data and provided a natural forum for a broader discussion of the results from this unique experiment. What follows is arranged, for the most part, in the context of the ISSI project.

Lennartsson, O. W.↗

DOE OSTI2021

DESC DC2 Data Release Note

In preparation for cosmological analyses of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), the LSST Dark Energy Science Collaboration (LSST DESC) has created a 300 deg$^2$ simulated survey as part of an effort called Data Challenge 2 (DC2). The DC2 simulated sky survey, in six optical bands with observations following a reference LSST observing cadence, was processed with the LSST Science Pipelines (19.0.0). In this Note, we describe the public data release of the resulting object catalogs for the coadded images of five years of simulated observations along with associated truth catalogs. We include a brief description of the major features of the available data sets. To enable convenient access to the data products, we have developed a web portal connected to Globus data services. We describe how to access the data and provide example Jupyter Notebooks in Python to aid first interactions with the data. We welcome feedback and questions about the data release via a GitHub repository.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRS1992

Earth observing system. Output data products and input requirements, version 2.0. Volume 1: Instrument data product characteristics

Information on Earth Observing System (EOS) output data products and input data requirements that has been compiled by the Science Processing Support Office (SPSO) at GSFC is presented. Since Version 1.0 of the SPSO Report was released in August 1991, there have been significant changes in the EOS program. In anticipation of a likely budget cut for the EOS Project, NASA HQ restructured the EOS program. An initial program consisting of two large platforms was replaced by plans for multiple, smaller platforms, and some EOS instruments were either deselected or descoped. Updated payload information reflecting the restructured EOS program superseding the August 1991 version of the SPSO report is included. This report has been expanded to cover information on non-EOS data products, and consists of three volumes (Volumes 1, 2, and 3). Volume 1 provides information on instrument outputs and input requirements. Volume 2 is devoted to Interdisciplinary Science (IDS) outputs and input requirements, including the 'best' and 'alternative' match analysis. Volume 3 provides information about retrieval algorithms, non-EOS input requirements of instrument teams and IDS investigators, and availability of non-EOS data products at seven primary Distributed Active Archive Centers (DAAC's).

Lu, Yun-Chi↗

DOE OSTI2026

Performance and Reliability Assessment of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) Data Advisor (ADA)

The Atmospheric Radiation Measurement (ARM) User Facility provides one of the world's largest openly accessible repositories of atmospheric observations through the ARM Data Discovery platform. Although the repository contains more than three decades of measurements collected from permanent observatories, mobile facilities, aircraft campaigns, and field experiments, identifying appropriate datasets can be challenging, particularly for new users unfamiliar with ARM instrumentation and datastream organization. To improve data accessibility, the ARM Data Center developed the ARM Data Advisor (ADA), an artificial intelligence-powered assistant designed to facilitate scientific data discovery, dataset interpretation, and user guidance. This report evaluates ADA's performance as a domain-specific scientific assistant using realistic atmospheric science workflows. The evaluation examines five key capabilities: data retrieval and curation efficiency, hallucination resistance, scientific reasoning, response to ambiguous queries, and content retention and session continuity. Representative prompts were developed to simulate typical interactions between researchers and the ARM Data Discovery platform, and ADA's responses were assessed for retrieval completeness, scientific accuracy, consistency, and practical usefulness. In these representative tests, ADA reduced the complexity of discovering and accessing ARM datasets by recommending appropriate datastreams, explaining instrumentation, interpreting metadata, and assisting with data processing workflows. ADA also exhibits strong domain knowledge of atmospheric science terminology and generally resists hallucination by acknowledging unavailable datasets and requesting clarification when appropriate. Overall, the results indicate that ADA represents a promising advancement in scientific data discovery within the ARM User Facility and has considerable potential to improve researcher productivity, particularly for new users and interdisciplinary scientists seeking efficient access to ARM observations.

Salvador, Christian [ORNL] (ORCID:0000000283287777↗

DOE OSTI2015

Robot for Investigations and Assessments of Nuclear Areas

RIANA is a remote controlled Robot dedicated for Investigations and Assessments of Nuclear Areas. The development of RIANA is motivated by the need to have at disposal a proven robot, tested in hot cells; a robot capable of remotely investigate and characterise the inside of nuclear facilities in order to collect efficiently all the required data in the shortest possible time. It is based on a wireless medium sized remote carrier that may carry a wide variety of interchangeable modules, sensors and tools. It is easily customised to match specific requirements and quickly configured depending on the mission and the operator's preferences. RIANA integrates localisation and navigation systems. The robot will be able to generate / update a 2D map of its surrounding and exploring areas. The position of the robot is given accurately on the map. Furthermore, the robot will be able to autonomously calculate, define and follow a trajectory between 2 points taking into account its environment and obstacles. The robot is configurable to manage obstacles and restrict access to forbidden areas. RIANA allows an advanced control of modules, sensors and tools; all collected data (radiological and measured data) are displayed in real time in different format (chart, on the generated map...) and stored in a single place so that may be exported in a convenient format for data processing. This modular design gives RIANA the flexibility to perform multiple investigation missions where humans cannot work such as: visual inspections, dynamic localization and 2D mapping, characterizations and nuclear measurements of floor and walls, non destructive testing, samples collection: solid and liquid. The benefits of using RIANA are: - reducing the personnel exposures by limiting the manual intervention time, - minimizing the time and reducing the cost of investigation operations, - providing critical inputs to set up and optimize cleanup and dismantling operations. (authors)

42 ENGINEERING↗

DOE OSTI2018

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro- Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

analytics-hub↗

NASA NTRSDate not supplied

Biomass Harmonization and SAR Analysis with the Multi-mission Algorithm and Analysis Platform (MAAP)

The Multi‐mission Algorithm and Analysis Platform (MAAP) is a collaborative effort between NASA and the European Space Agency (ESA) to support above ground biomass (AGB) research in an open science framework. MAAP brings together relevant data, algorithms, and computing capabilities in a common cloud environment to address the challenges of sharing and processing data from field, airborne and satellite measurements. MAAP was publicly released in October 2021, providing computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of data. MAAP has allowed scientists from both North America and Europe to collaborate on the generation and analysis/visualization of data derived from multiple, discipline-adjacent missions in an open, collaborative environment that has reached beyond traditional scientific investigation. MAAP has been used to support multiple scientific activities. To date, existing LiDAR data from multiple platforms has been calibrated with field measurements and combined for more comprehensive and accurate estimates of above ground biomass AGB; these LiDAR platforms include airborne (e.g. LVIS), the International Space Station (NASA’s Global Ecosystem Dynamics Investigation (GEDI), and satellites (e.g. ICESat-2). The current challenge is to effectively and seamlessly combine the aforementioned LiDAR-based data with new data sources such as P-band RADAR from ESA’s upcoming BIOMASS mission, existing ESA Sentinel-1 C-band SAR, and the 30 PB/yr of high cadence global coverage L-band SAR data from the upcoming NASA-ISRO SAR (NISAR) mission. Recent analysis using MAAP merged ICESat-2 and optical data (Harmonized Landsat Sentinel) produced the most comprehensively precise estimate of boreal-wide AGB to date. Another effort using MAAP is the production and open distribution of global comparisons of AGB map estimates, including from ICESat-2 and GEDI, to bolster stakeholder uptake for policy applications. These map estimates will feed into the Intergovernmental Panel on Climate Change (IPCC) database, likely aiding the next Global Carbon Stocktake of the UNFCCC. Furthermore, the biomass retrieval intercomparison exercise BRIX-2 could benefit from the MAAP providing standardized test cases (based on airborne campaign and spaceborne data) allowing the community to develop and apply retrieval algorithms based on these test cases, while forthcoming SAR data training curricula could also use the MAAP as a teaching and learning platform. The MAAP is meeting the challenges inherent in international, open science collaboration and large scale computing with a platform that is entirely open source and cloud native, using open standards for data access, manipulation, protocols, and formats. The MAAP data system consists of a dedicated data store whose data is indexed in an online catalog conforming to established metadata, application programmatic interfaces (APIs), and service interface standards, using an implementation of the open sourced NASA Common Metadata Repository. Federation of user identities allows users from either NASA or ESA to access and consume services from the other using a unified metadata catalog for the data utilized across the ESA and NASA MAAP platforms. Similarly, we are exploring how to increase interoperability to achieve a common approach to packaging, orchestrating and executing algorithms, with interoperable access to data for subsetting, fast browse, and cloud-optimized access, all using interoperable standards such as those from the Open Geospatial Consortium (OGC). Designed for interoperability, ESA and NASA utilize a common architecture for the software platform. It provides a cloud-based algorithm development environment (ADE) that enables scientists to develop algorithms collaboratively with access to the MAAP data catalog as well as other data archives. MAAP provides an Eclipse Che-based ADE supporting both Python and R languages, popular in this biomass community. Algorithms developed and containerized within the ADE can be deployed to run to thousands of computational nodes in the MAAP’s data processing system (DPS), dramatically speeding up processing and giving scientists a rapid, iterative turnaround of results. NASA’s implementation of the DPS is based on the Hybrid Science Data System (HySDS) framework, used by NASA flight projects to produce Earth science standard products.

cloud computing↗

DOE OSTI2023

Critical parameters in the faculty application process: A data-driven analysis

We report the process of applying for science and engineering faculty positions requires a significant time investment for both the applicant and hiring committees. Applicants face the daunting task of sifting through a seemingly endless amount of information related to crafting their application packages in preparation for a review process that is inherently highly subjective. Furthermore, while scientific supervisors are often the most valuable resource during this stage of career development, many years may have passed since they went through this process themselves, and they may not have experience with recent developments in the application process. Because expectations and job responsibilities of new faculty members are rapidly evolving, certain guidelines are often not clearly specified, and it is therefore becoming difficult for applicants to most effectively prepare their application documents. Without access to adequate information and assistance, early-career researchers in particular can be placed at a significant disadvantage in the hiring process. Several recent articles can help prospective applicants in materials science and engineering land a faculty position,1,2,3 but there is nevertheless a need for clarification on expectations for different aspects of the faculty application materials.

36 MATERIALS SCIENCE↗

NASA NTRSDate not supplied

TPSAS-NF1676L-17867-DND

In response to the exponential growth in science data analysis and visualization capabilities, data centers have been developing new processes to package and deliver large volumes of aggregated subsets of archived data. New standards are evolving to help data providers and application programmers manage the growing needs of the science community. These standards evolve from the best practices gleaned from new products and capabilities. The NASA Atmospheric Sciences Data Center (ASDC) has developed and deployed production provider-specific search and subset web applications for the CALIPSO, CERES, TES, and MOPITT missions. This presentation explores a CERES CCCM (CALIPSO, CloudSat, CERES, MODIS) data validation use case that leverages aggregated subset results from CERES CCCM (Level2), CERES SSF (Level2), and CALIPSO LIDAR (Level) datasets. Additionally, it examines the standards and formats that ASDC developers have applied to the delivered files as well as the implementation strategies for subsetting and processing the aggregated products.

Walter E Baskin↗

NASA NTRSDate not supplied

Complications of Metadata Curation for NASA Airborne and Field Campaigns, Platforms, and Instruments

The Airborne Data Management Group (ADMG) curates metadata that describe NASA's airborne and field campaigns, platforms and instruments. This activity is vital to building a useful inventory of sub-orbital Earth science data that improves data discovery and access. During the curation process, many metadata issues were identified that required improvement to campaign and data product metadata. In some cases, locating the needed metadata to add to the inventory was a simple process. For other cases, the information was hard to find. In addition, identifying accurate investigation instrument details to add to the inventory was especially complicated because of the variety of definitions used in the Earth science community for the same concepts. One example of this is the concept of instruments' spatial and temporal resolution. The spatial resolution is one of the more difficult elements to curate given the variations in meaning across various disciplines. Clarified definitions are needed to enable consistency of information across campaigns and instruments. In this presentation, we introduce results from a survey of scientists from various fields in which we asked for definitions of spatial and temporal resolution. Our survey results highlight the importance of creating more universally acceptable definitions for certain metadata elements. By curating sub-orbital field campaign and instrument metadata, ADMG is enabling more efficient discovery and access to NASA observations by allowing science data users to search for certain clearly defined criteria and metadata values.

Ashlyn Shirey↗

NASA NTRSDate not supplied

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗

NASA NTRS2023

Floating Potential Measurement Unit (FPMU) Data Processing Algorithm Development and Analysis Assessment

The NASA Engineering and Safety Center (NESC) was requested to provide space plasma instrumentation data analysis expertise in support of Floating Potential Measurement Unit (FPMU) data analysis and data processing algorithm development. Subject matter expertise was requested to support modification of FPMU data processing algorithms to account for aging of the FPMU instrument suite, improve the science quality of the FPMU data sets that are made available to the science community, and provide analysis of FPMU data for complex spacecraft orientation and geophysical situations. This document contains the results of the NESC assessment.

Floating Potential Measurement Unit↗

NASA NTRS2015

Next Generation UAS Based Spectral Systems for Environmental Monitoring

This presentation provides information on the development of a small Unmanned Aerial System(UAS) with a low power, high performance Intelligent Payload Module (IPM) and a hyperspectral imager to enable intelligent gathering of science grade vegetation data over agricultural fields at about 150 ft. The IPM performs real time data processing over the image data and then enables the navigation system to move the UAS to locations where measurements are optimal for science. This is important because the small UAS typically has about 30 minutes of battery power and therefore over large agricultural fields, resource utilization efficiency is important. The key innovation is the shrinking of the IPM and the cross communication with the navigation software to allow the data processing to interact with desired way points while using Field Programmable Gate Arrays to enable high performance on large data volumes produced by the hyperspectral imager.

onboard processing↗

NASA NTRS1984

Artificial intelligence applications concepts for the remote sensing and earth science community

The following potential applications of AI to the study of earth science are described: (1) intelligent data management systems; (2) intelligent processing and understanding of spatial data; and (3) automated systems which perform tasks that currently require large amounts of time by scientists and engineers to complete. An example is provided of how an intelligent information system might operate to support an earth science project.

Campbell, W. J.↗

NASA NTRS1986

User's guide to image processing applications of the NOAA satellite HRPT/AVHRR data. Part 1: Introduction to the satellite system and its applications. Part 2: Processing and analysis of AVHRR imagery

The use of NOAA Advanced Very High Resolution Radar/High Resolution Picture Transmission (AVHRR/HRPT) imagery for earth resource applications is provided for the applications scientist for use within the various Earth science, resource, and agricultural disciplines. A guide to processing NOAA AVHRR data using the hardware and software systems integrated for this NASA project is provided. The processing steps from raw data on computer compatible tapes (1B data format) through usable qualitative and quantitative products for applications are given. The manual is divided into two parts. The first section describes the NOAA satellite system, its sensors, and the theoretical basis for using these data for environmental applications. Part 2 is a hands-on description of how to use a specific image processing system, the International Imaging Systems, Inc. (I2S) Model 75 Array Processor and S575 software, to process these data.

Huh, Oscar Karl↗

DOE OSTI2015

Quasar classification using color and variability

We conduct a pilot investigation to determine the optimal combination of color and variability information to identify quasars in current and future multi-epoch optical surveys. We use a Bayesian quasar selection algorithm to identify 35,820 type 1 quasar candidates in a 239 deg{sup 2} field of the Sloan Digital Sky Survey (SDSS) Stripe 82, using a combination of optical photometry and variability. Color analysis is performed on 5-band single- and multi-epoch SDSS optical photometry to a depth of r∼22.4. From these data, variability parameters are calculated by fitting the structure function of each object in each band with a power-law model using 10 to >100 observations over timescales from ∼1 day to ∼8 years. Selection was based on a training sample of 13,221 spectroscopically confirmed type-1 quasars, largely from the SDSS. Using variability alone, colors alone, and combining variability and colors we achieve 91%, 93%, and 97% quasar completeness and 98%, 98%, and 97% efficiency, respectively, with particular improvement in the selection of quasars at 2.7<z<3.5 where quasars and stars have similar optical colors. The 22,867 quasar candidates that are not spectroscopically confirmed reach a depth of i∼22.0; 21,876 (95.7%) are dimmer than coadded i-band magnitude of 19.9, the cutoff for spectroscopic follow-up for SDSS on Stripe 82. Brighter than 19.9, we find 5.7% more quasar candidates without confirming spectra in sky regions otherwise considered complete. The resulting quasar sample has sufficient purity (and statistically correctable incompleteness) to produce a luminosity function comparable to those determined by spectroscopic investigations. We discuss improvements that can be made to the process in preparation for performing similar photometric selection and science on data from post-SDSS sky surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2011

Data Serving Climate Simulation Science at the NASA Center for Climate Simulation

The NASA Center for Climate Simulation (NCCS) provides high performance computational resources, a multi-petabyte archive, and data services in support of climate simulation research and other NASA-sponsored science. This talk describes the NCCS's data-centric architecture and processing, which are evolving in anticipation of researchers' growing requirements for higher resolution simulations and increased data sharing among NCCS users and the external science community.

Salmon, Ellen M.↗