Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Analysis-Ready Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Fostering Open Science Inclusiveness for Interdisciplinary Users of Earth Observations

The term Open Science is subject to a variety of interpretations because of a key (and useful) ambiguity in the meaning of “Open”. Open in the sense of Transparency enables more trust in science research by making the details of the scientific process visible and accessible to anyone. “Open” in the sense of Inclusiveness enables more scientists from other disciplines to participate in research in a given discipline, thus producing more interdisciplinary research. Data Systems can play a major role in enabling Open (Inclusive) Science by making it easier for users from other disciplines to work with data within a given discipline. This is challenging for Earth Observation datasets, most of which are the product of advanced instrumentation and sophisticated, specialized variable retrieval algorithms and code. Serving the “extra-disciplinary”communities begins with simple things, like accessible, readable data documentation with adequate scaffolding. But just as important is provisioning Analysis-Ready data that does not require expert pre-processing. Disciplines also often have dominant toolsets, such as R in the biomass community or GIS in many applications communities. Ensuring that EO data are easy to use in the tools favored in other communities will enable more interdisciplinary research. Ideally, interdisciplinary research also benefits from scientists with different domain expertise. Platforms and frameworks that facilitate frictionless collaboration with discipline experts, together with capacity building efforts in those external disciplines also improve the inclusiveness aspect of Open Science. In short, Open Science is at root a way of thinking about how users from diverse discipline can best access and use data and services from a particular discipline.

Christopher Lynnes

The NASA Earth Science Applied Sciences Disasters Program: Making EO Data and Expertise Available through the Disasters Mapping Portal to Inform Decision-Makers throughout the Disaster Cycle

The NASA Earth Science Applied Sciences Disasters Program promotes the use of Earth Observations (EO) to inform disaster risk reduction and resilience throughout the disaster cycle, from local to global scales, by harnessing NASA science and technology capabilities, through engagement with end users to demonstrate the value and impact of EO to support decision-making, and by supporting end users in their use of EO in decision-making while developing relationships to grow as a trusted source of relevant science and useful results. Through the NASA Earth Science Applied Sciences Disasters Program Mapping Portal, an Esri-backed hub of geospatially enabled disaster products, event-based and near-real-time products are hosted to provide end users with analysis-ready data and data services, keeping in mind their expressed EO needs. In this presentation, case studies of past events will be highlighted, showcasing how EO data and services were provided by the Program and utilized by end users during the disaster cycle. Additionally, this presentation will highlight some of the Program’s ongoing collaboration activities to assist end users in understanding and preparing for utilization of EO data from the Mapping Portal to inform their decision-making throughout the disaster cycle. Ongoing efforts to advance the science provided by the Program on the Mapping Portal through new developments and capabilities will be highlighted, and common challenges and data needs end users often encounter when using EO data and our Program’s innovative solutions to these challenges will be addressed. Furthermore, this presentation will touch on the challenges faced and the solutions created by the Mapping Portal team in managing and hosting a plethora of EO data for end users across various disciplines.

Ronan Lucey

NASA POWER: Providing Analysis-Ready, Cloud-Optimized Data for AI /ML Training and Applications in Earth Science

As global demand for sustainable development grows, the integration of Earth Observation (EO) data into decision making frameworks has become a primary objective for the scientific community. The NASA Prediction of Worldwide Energy Resources (POWER) project serves as a bridge between NASA EO data and the specialized needs of the renewable energy, sustainable infrastructure and agroclimatology communities. In this poster presentation we will present an overview of POWER data products and services along with its use in diverse research to decision-making workflows. By providing over 40 years of high-resolution historical, hourly and daily solar and meteorological data, POWER transforms satellite observations and global model reanalysis into actionable, Analysis-Ready Dataset (ARD). Currently, the project delivers over 250 industry-friendly parameters to the users from different NASA datasets like CERES SYN1Deg, MERRA-2, and IMERG alongside downscaled CMIP6 climate model data, fulfilling over 16 million requests from 50,000 unique users monthly. To ensure data quality and traceability, these parameters are rigorously validated against the ground-based observations from the Baseline Surface Radiation Network (BSRN) and the Global Surface Summary of the Day (GSOD) – these results will be discussed in the presentation. A newly introduced web-based PaRameter Uncertainty ViEwer (PRUVE) tool will be presented that provides an online validation platform to the users that benchmarks satellite-based and assimilation data products against these surface measurements. To reduce technical barriers to data adoption, POWER data is accessible through RESTful APIs, ESRI ArcGIS Image Services, a web-based Data Access Viewer tool, allowing users to visualize, validate and apply the dataset. For efficient data delivery POWER data is cloud-optimized into Zarr datastore accessible through NASA managed Amazon S3 ensures high-performance allowing users to integrate EO directly into operational pipelines. These customized services will be presented. Use cases from application will be presented from the energy sector - such as for design of generation systems, performance monitoring of solar power plants, in infrastructure sector- optimizing building energy efficiency and thermal comfort, in agriculture – such as driving crop simulation and yield forecasting models to enable climate resilient farming. Furthermore, the shift toward machine learning (ML) in EO research that has positioned POWER as a key provider for training datasets which will be discussed. Use-cases will be presented to showcase how NASA data is enabling the development of predictive tools for climate variability and resource management. The poster will present POWER’s future plans including technology development to enhance data traceability and reproducibility and improving I/O performance to support the rapid integration of new EO products, ensuring that POWER remains a robust scalable backend for the evolving landscape of AI-driven Earth Science. Additionally, POWER is developing an AI Agent and an MCP-Server to enable industry AI-Agentic workflows.

Neha Khadka

Assessing the Needs of NASA's Near Real-Time Earth Observation Products

"The 2017-2027 Decadal Survey for Earth Science and Applications from Space stated that NASA's Earth Science with planned implementation of applications provides sustained earth observations for societal benefits [1]. The Decadal Survey indicated that data latency is invaluable for time-sensitive applications including disaster risk reduction, wildland fire carbon emissions quantification, real-time measurements of the state of the hydrologic systems and many more. Data latency refers to the time between earth observation and data products available to users. During the past 13 years, NASA's Land, Atmosphere Near Real-Time Capability for Earth Observing Systems (LANCE) continues to provide free access to earth observation products that are made available much quicker than routine processing allows. The latency of most LANCE data products is Near Real-time (NRT) which is defined as less than three hours from satellite observations [2]. LANCE is managed by the Earth Science Data and Information System (ESDIS) Project at NASA Goddard Space Flight Center [3], and a User Working Group (UWG) is responsible for providing guidance to LANCE. LANCE data are used by direct users and brokers who add value to the data [4]. NASA Earth Applied Sciences Program (ASP) is one of the primary users of LANCE, which collaborates with partner organizations and provides support to scientists to solve problems in applications of earth observations. ASP promotes the use of LANCE NRT data products to demonstrate applications in decision making, facilitates end-user feedback to the science team to improve data products, and provides information on future demands for research. LANCE supports applications that need a rapid response including detecting wildland fires and volcanic eruptions, tracking smoke, ash and dust plumes, monitoring air quality and tracking extreme weather events such as hurricanes, landslides, and floods. To gather feedback regarding the availability, accessibility and actionability of NASA's NRT data products for societal benefit, three surveys and a few discussions with experts involved in the topic within ASP were conducted from the perspective of users. Feedback has been collected from users who are interested in using low latency NASA data within application communities of agriculture, disasters, water resources, health and air quality, ecological conservation, wildland fires and capacity building. Analysis-ready NRT data products in a variety of formats have been mentioned many times in the collected feedback, especially for applied users with little to no experience using research-grade earth observation products. Users prefer to have products that can be easily integrated into their existing workflows and take their analysis to the data. HDF5 is a commonly used data format for research, but typically requires some conversion to a more friendly format for applications and regular use in decision-making. Users prefer the GeoTIFF data format that can be directly ingested into a GIS mapping software and platform for data analysis and visualization. For example, LANCE’s fire, flood, SO2 and Black Marble Nighttime Blue/Yellow Composite data products have been integrated into NASA Disasters Mapping Portal, which is an GIS-based open data portal, for users in the disaster management community. There are 291 LANCE NRT layers available through GIBS and Worldview, where users can download a snapshot in GeoTIFF format. Operational users expect data to be processed as close to the user as possible. The collected feedback indicates that LANCE fire products within 3 hours latency would meet the needs of the wildland fire community. The ideal latency for volcanic application is 10-15 minutes. Users in Volcanic Ash Advisory Centers (VAAC) reported that the first forecast volcanic product should be issued within 75 minutes from the volcano eruption [5]. Overall, for disaster applications, data latency within 3 hours is useful while latency greater than 12 hours is not timely enough for operational use. Capacity building and training are critical for users to be able to access, interpret and use data products and tools for their decision making, especially for applied users with limited experience using earth observation products. LANCE data products have been used in a number of capacity building projects domestically and internationally [6]. As LANCE continues to bring new products into the system, users request training to utilize LANCE new and upcoming data products and capabilities in their applications. Due to the limitation of bandwidth and downstream flow paths, users in some developing countries need tools to select and download data for a specific area of interest instead of bulk downloads. The collected feedback also shows the lack of available SAR satellite low latency data products. The advantages of SAR to monitor conditions and changes on the ground through darkness, clouds, volcanic ash, and other atmospheric conditions, are appealing to low latency users. For example, terabytes of low latency but cloudy optical images are not helpful in rapidly identifying the extent of flood or fire impacts. LANCE could be complemented with low latency measurements via the upcoming NASA-ISRO Synthetic Aperture Radar (NISAR) mission [7]. Requests for higher spatial resolution products are expressed. A user from the wildland fire management community reported that products with 30-m spatial resolution could be used to detect small fires. The 30-m Landsat OLI fire data is now part of NASA’s Fire Information for Resource Management System (FIRMS) US/Canada [8]. Within the open and free NASA resources, LANCE disseminates NRT data products in a manner that allows them to be accessible and understandable to both scientific and applied users. In many application areas, latency plays an important or even decisive role where low latency earth observations help people to observe areas of interest, detect and track changes in the environment and make timely decisions. NASA’s Earth Applied Sciences Program promotes the use of LANCE NRT products and builds a bridge between application users and research teams. The collected feedback indicates data latency within 3 hours is useful for most of the applications, and shows the needs of user-friendly, analysis-ready products, and requests training on LANCE’s new and upcoming data products. User feedback has been provided to LANCE UWG for guidance and recommendations, and for translating findings into something actionable.

Tian Yao

NASA Power: Global Solar Insolation, Meteorological Parameter Data, and Web Services to Support Sustainable Building Design and Operations

The buildings industry is currently striving to adopt green solutions to make infrastructure more energy-efficient in order to meet the 2050 net-zero climate goals. This planning requires reliable environmental datasets that are crucial in designing, building, and maintaining our world’s-built environment, as well as other energy-related processes and investments. This webinar for the National Institute of Building Sciences provides an overview of NASA’s Prediction Of Worldwide Energy Resources (POWER) Project that informs decision-making and development for sustainable building design and operations by enabling public open discovery, efficient access, and convenient distribution of NASA’s Earth Observations and global atmospheric model datasets. POWER’s datastore is comprised of solar radiation and surface meteorology parameters, spanning nearly 40 years of hourly data, that are easily accessible via several access methods and tools to support three focus areas: 1) renewable energy deployment and management, 2) sustainable infrastructure, and 3) agroclimatology applications. POWER and NASA Earth Science both plan future data parameters, updated tools, and improved observations that could directly support U.S. and international sustainable development goals, climate strategies, and building information modeling. To this end, solar data from several NASA projects and meteorological data from NASA assimilation models have already been reformatted and disseminated to the public via a user-friendly web GIS-enabled based data portal through the POWER Project. POWER data is analysis-ready and accessible through an Application Programming Interface (API), ArcGIS Image Services, and the project’s Data Access Viewer enhanced (DAVe), an interactive online tool. The POWER DAVe also features data consistent with ASHRAE Climate Design Conditions and has developed web image services showing Building Climate Zones and their variability. Through those tools, the data can be downloaded into multiple formats that support the infrastructure community, including CSV and Energy Plus Weather (EPW). POWER’s entire data product catalog is available through Amazon Web Services (AWS) Open Data Registry (ODR) via a free and publicly accessible Simple Storage Service (S3). This webinar provides a full overview of the NASA POWER Project's data and services developed in collaboration with the sustainable infrastructure community. Examples of how the renewable energy and building communities have utilized POWER data products to make decisions and a preview of future data product expansion, including climate projections, and web services will also be provided. Additionally, use case stories from our broad community of users will be presented.

Paul W. Stackhouse

Treating gridded geospatial data as point data to simplify analytics

Gridded geospatial remote sensing (satellite) data has traditionally been stored in file-based multidimensional arrays to preserve the locality of data. Measurements from locations that are physically next to each other on earth remain next to each other in the arrays. Maintaining this locality is useful when running calculations like reprojection, but unnecessary for many other calculations. This talk will go through a real world example of a tool redesign at the Goddard Earth Sciences Data and Information Services Center (GES DISC), showing the advantages of using the data frame model for calculating summary statistics, where measurement proximity is unimportant.

Analysis-ready data

Analysis Ready Satellite Data

Analyais-Ready Data (ARD) specifications have gained rcent prominence in the field of land-related Earth Observations. The ARD label enables users to recognize data that need a minimum of preprocessing before analysis. However, the regular geolocation requirements make Level 2 data in other disciplines problematic. While Level 3 gridded data can satisfy the geolocation requirement, they often sacrifice spatial resolution and other information, such as extreme values. This talk outlines this dilemma with some potential approaches to it.

Analysis-Ready Data

How Can NASA Science Benefit Solar Energy Development and Assessment?

This is a keynote presentation for a workshop highlighting the use of remote sensing and modeling products derived with NASA missions and Earth system modeling for solar energy system development. We broadly introduce NASA's fleet of satellite producing publicly available data products that would be useful for the information needed solar energy development. We also introduce the work in Earth system science modeling that provides additional global gridded data products that are also relevant. We identify that one issue is obtaining and using this data is the level of expertise needed to find it. NASA has developed a new Earth Action program to address this gap and bring the data closer to the user. Now under the Energy Resources element, the NASA LaRC's POWER (Prediction of Worldwide Renewable Energy) project has developed a web services platform that provides key information needed for solar energy development as analysis-ready. This means that the data for particular locations are immediately useful for help provide the information needed by engineers and architects to plan for solar systems. Thus, the presentation shows why NASA products would be relevant and then shows examples of finding the needed data through the POWER web portal.

remote sensing

Satellite Ocean Colour: Current Status and Future Perspective

Spectrally resolved water-leaving radiances (ocean colour) and inferred chlorophyll concentration are key to studying phytoplankton dynamics at seasonal and inter-annual scales, for a better understanding of the role of phytoplankton in marine biogeochemistry; the global carbon cycle; and the response of marine ecosystems to climate variability, change and feedback processes. Ocean colour data also have a critical role in operational observation systems monitoring coastal eutrophication, harmful algal blooms, and sediment plumes. The contiguous ocean-colour record reached 21 years in 2018; however, it is comprised of a number of one-off missions such that creating a consistent time-series of ocean-colour data requires merging of the individual sensors (including MERIS, Aqua-MODIS, SeaWiFS, VIIRS, and OLCI) with differing sensor characteristics, without introducing artefacts. By contrast, the next decade will see consistent observations from operational ocean colour series with sensors of similar design and with a replacement strategy. Also, by 2029 the record will start to be of sufficient duration to discriminate climate change impacts from natural variability, at least in some regions. This paper describes the current status and future prospects in the field of ocean colour focusing on large to medium resolution observations of oceans and coastal seas. It reviews the user requirements in terms of products and uncertainty characteristics and then describes features of current and future satellite ocean-colour sensors, both operational and innovative. The key role of in situ validation and calibration is highlighted as are ground segments that process the data received from the ocean-colour sensors and deliver analysis-ready products to end-users. Example applications of the ocean-colour data are presented, focusing on the climate data record and operational applications including water quality and assimilation into numerical models. Current capacity building and training activities pertinent to ocean colour are described and finally a summary of future perspectives is provided.

ocean colour

Making Better Use of Satellite Data: The Satellite Needs Working Group

The U.S. Group on Earth Observations (USGEO) initiated in 2016 the Satellite Needs Working Group (SNWG) to identify and communicate the Earth observation needs of U.S. federal agencies. The SNWG identifies such needs through a biennial survey followed by interviews and follow-up discussions by the satellite Earth data providers of the U.S. Government: the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration (NOAA), and the U.S. Geological Survey (USGS). Solutions and services are identified that leverage current or upcoming satellite missions to meet the identified needs; implementation of services that are estimated to significantly increase the level of satisfaction of multiple U.S. agencies are funded by NASA. The SNWG process has resulted in the implementation of numerous services that have impacted operations of not only U.S. agencies but of academic and international institutions as well. Notable examples include the Harmonized Landsat Sentinel-2 project that leverages European Space Agency (ESA) and NASA satellite assets to generate a global, analysis-ready, surface reflectance product with a temporal resolution of two days; the Airborne Data Management Group that curates and provides access to relevant resources, information and data from existing and past NASA field campaigns; and the Dynamic Surface Water Extent product which consists of harmonized but independent water extents derived from both from optical and radar data. These and other products are hosted at the NASA Distributed Active Archive Centers (DAACs) for free and open access. The SNWG Management Office at NASA’s Interagency Implementation and Advanced Concepts Team (IMPACT) manages the implementation of selected solutions and, importantly, NASA’s response to the needs of federal agencies through a Stakeholder Engagement Program. The impact of implemented services and solutions is hampered without efforts to build capacity around the use of the services. The Stakeholder Engagement Program ensures relevant training and outreach to SNWG agencies in collaboration with each solution implementation team.

remote sensing

Multidimensional Data Aggregation in the Cloud with Application to Geostationary Satellite-based Air Quality Monitoring

Scientists use satellite data for studying Earth's systems, and the remote sensing data that these satellites collect are typically separated into files of a size small enough for efficient network transfer and storage. However, researchers usually prefer to analyze the data based on real-world dimensions like time, space, or elevation. To help with this, NASA's Atmospheric Science Data Center (ASDC) developed a new cloud-based tool that combines these smaller data chunks into larger, more useful datasets. The tool works on Network Common Data Form (netCDF4) and some HDF5 formatted files, and it is available as a service in NASA's Earthdata Cloud. In this presentation, we showcase this service using data from the Tropospheric Emissions: Monitoring of Pollution (TEMPO) instrument. By combining TEMPO's continuous observations over time, we create longer and more informative analysis-ready time series to facilitate the study of air quality patterns. Insights gained will provide a more comprehensive understanding of pollution sources, transport patterns, and their effects on the environment and human health.

Daniel Kaufman

Constrained Reference Star Differential Imaging: Enabling High-fidelity Imagery of Highly Structured Circumstellar Disks

High-contrast imaging presents us with the opportunity to study circumstellar disks and the planets still embedded within them, providing key insights into the formation and evolution of planetary systems. However, the postprocessing techniques that are often needed to suppress stellar halo light typically result in significant and variable loss of circumstellar light, even when using relatively conservative approaches like reference star differential imaging (RDI). We introduce “constrained reference star differential imaging” (constrained RDI), a new class of RDI point-spread-function (PSF) subtraction techniques for systems with circumstellar disks. Constrained RDI utilizes either high-resolution polarized-intensity (PI) images or disk models to severely limit or even eliminate the signal loss due to oversubtraction that is common to RDI. We demonstrate the ability of constrained RDI utilizing polarimetric data to yield an oversubtraction-free detection of the AB Aurigae protoplanetary disk in total intensity. PI-constrained RDI allows us to decisively recover the spectral signature of the confirmed, recently discovered protoplanet, AB Aurigae b. We further demonstrate that constrained RDI can be a powerful analysis tool for soon-to-be-acquired James Webb Space Telescope coronagraphic imaging of disks. In both cases, constrained RDI provides analysis-ready products that enable more detailed studies of disks and more robust verification of embedded exoplanets.

Kellen Lawson