Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Earth Science Data Analytics: Bridging Tools and Techniques with the Co-Analysis of Large, Heterogeneous Datasets

The continuum of ever-evolving data management systems affords great opportunities to the enhancement of knowledge and facilitation of science research. To take advantage of these opportunities, it is essential to understand and develop methods that enable data relationships to be examined and the information to be manipulated. This presentation describes the efforts of the Earth Science Information Partners (ESIP) Federation Earth Science Data Analytics (ESDA) Cluster to understand, define, and facilitate the implementation of ESDA to advance science research. As a result of the void of Earth science data analytics publication material, the cluster has defined ESDA along with 10 goals to set the framework for a common understanding of tools and techniques that are available and still needed to support ESDA.

science data analysis↗
NASA NTRSDate not supplied

Blast from the Past: ASDC Curation for NASA Suborbital Legacy Missions to Promote Data Discovery and Accessibility

NASA has an extensive history of conducting suborbital field campaigns to further advances in atmospheric sciences. Beginning with the Chemical Instrument Test and Evaluation (CITE) conducted in 1983-1984, NASA has completed many suborbital campaigns over the past three decades. Since the early 2010s, suborbital missions are typically assigned to a NASA Distributed Active Archive Center (DAAC) prior to the mission for long-term archival and distribution. Efforts are being made by NASA’s Earth Science Data and Information System (ESDIS) Project and the Airborne Data Management Group (ADMG) to assign legacy missions to DAACs for permanent archival and distribution, so that these valuable datasets remain to be available to the scientific community. NASA’s Atmospheric Science Data Center (ASDC) has been named the assigned DAAC for nearly 20 atmospheric composition legacy missions, including missions conducted as part of the Global Tropospheric Experiment (GTE) and expects to be named the assigned DAAC for more of these missions over the next few years. The primary goal of the ASDC is to provide access to the datasets as they are currently formatted to the broad user community and enhance their findability and accessibility. However, data reporting standards have evolved significantly since 1983 and the datasets span a wide variety of file formats, including text, Ames, GTE, and ICARTT (International Consortium for Atmospheric Research on Transport and Transformation), and the amount of metadata and relevant information included in the files also varies greatly and can not be readily extracted without subject matter knowledge. This has caused challenges for the ASDC’s suborbital metadata extraction pipeline in ensuring that accurate and necessary metadata is being provided for the missions by all the ASDC’s existing search mechanisms. To make the data more findable and accessible, the ASDC has begun researching ways to further enhance the datasets, including distributing value-added products (i.e. consistent file format such as ICARTT or netCDF), adding standard names from the ESDIS Standards Coordination Office (ESCO)-approved Atmospheric Composition Variable Standard Names Convention (ACVSNC), and creating outreach materials such as ArcGIS StoryMaps, User Guides, and Micro Articles, providing overviews of the missions and what type of data was collected during the missions. These efforts also help support NASA’s Open-Source Science by enhancing the FAIRness of the legacy data products. This presentation will review the ASDC’s ongoing efforts, progress made, and future plans for legacy missions.

Megan Buzanowicz↗
NASA NTRSDate not supplied

Stewardship Best Practices for Improved Discovery and Reuse of Heterogeneous and Cross-Disciplinary Earth System Data

Some of the Earth system data products such as those from NASA airborne and field investigations (a.k.a. campaigns), are highly heterogeneous and cross-disciplinary, making the data extremely challenging to manage. For example, airborne and field campaign measurements tend to be sporadic over a period of time, with large gaps. Data products generated are of various processing levels and utilized for a wide range of inter- and cross-disciplinary research and applications. Data and derived products have been historically stored in a variety of domain-specific standard (and some non-standard) formats and in various locations such as NASA Distributed Active Archive Centers (DAACs), NASA airborne science facilities, field archives, or even individual scientists’ computer hard drives. As a result, airborne and field campaign data products have often been managed and represented differently, making it onerous for data users to find, access, and utilize campaign data. Some difficulties in discovering and accessing the campaign data originate from the incomplete data product and contextual metadata that may contain details relevant to the campaign (e.g. campaign acronym and instrument deployment locations), but tend to lack other significant information needed to understand conditions surrounding the data. Such details can be burdensome to locate after the conclusion of a campaign. Utilizing consistent terminology, essential for improved discovery and reuse, is also challenging due to the variety of involved disciplines. To help address the aforementioned challenges faced by many repositories and data managers handling airborne and field data, this presentation will describe stewardship practices developed by the Airborne Data Management Group (ADMG) within the Interagency Implementation and Advanced Concepts Team (IMPACT) under the NASA’s Earth Science Data systems (ESDS) Program.

best practices↗
NASA NTRS2010

The Kepler Science Operations Center Pipeline Framework Extensions

The Kepler Science Operations Center (SOC) is responsible for several aspects of the Kepler Mission, including managing targets, generating on-board data compression tables, monitoring photometer health and status, processing the science data, and exporting the pipeline products to the mission archive. We describe how the generic pipeline framework software developed for Kepler is extended to achieve these goals, including pipeline configurations for processing science data and other support roles, and custom unit of work generators that control how the Kepler data are partitioned and distributed across the computing cluster. We describe the interface between the Java software that manages the retrieval and storage of the data for a given unit of work and the MATLAB algorithms that process these data. The data for each unit of work are packaged into a single file that contains everything needed by the science algorithms, allowing these files to be used to debug and evolve the algorithms offline.

Klaus, Todd C.↗
NASA NTRSDate not supplied

TPSAS-NF1676L-17867-DND

In response to the exponential growth in science data analysis and visualization capabilities, data centers have been developing new processes to package and deliver large volumes of aggregated subsets of archived data. New standards are evolving to help data providers and application programmers manage the growing needs of the science community. These standards evolve from the best practices gleaned from new products and capabilities. The NASA Atmospheric Sciences Data Center (ASDC) has developed and deployed production provider-specific search and subset web applications for the CALIPSO, CERES, TES, and MOPITT missions. This presentation explores a CERES CCCM (CALIPSO, CloudSat, CERES, MODIS) data validation use case that leverages aggregated subset results from CERES CCCM (Level2), CERES SSF (Level2), and CALIPSO LIDAR (Level) datasets. Additionally, it examines the standards and formats that ASDC developers have applied to the delivered files as well as the implementation strategies for subsetting and processing the aggregated products.

Walter E Baskin↗
DOE OSTI2022

An open science exploration of global 1-km simulations of the earth's atmosphere

We are making available a multi-petabyte data set for open science via a data hackathon. The participants bring their science to the computing and data facility, alleviating the need to transfer and manage the large volume of data. This accelerates the process of discovery science while increasing scientific throughput.

Anantharaj, Valentine↗
NASA NTRS2016

Stewardship of NASA's Earth Science Data and Ensuring Long-Term Active Archives

Program, NASA has followed an open data policy, with non-discriminatory access to data with no period of exclusive access. NASA has well-established processes for assigning and or accepting datasets into one of 12 Distributed Active Archive Centers (DAACs) that are parts of EOSDIS. EOSDIS has been evolving through several information technology cycles, adapting to hardware and software changes in the commercial sector. NASA is responsible for maintaining Earth science data as long as users are interested in using them for research and applications, which is well beyond the life of the data gathering missions. For science data to remain useful over long periods of time, steps must be taken to preserve: (1) Data bits with no corruption, (2) Discoverability and access, (3) Readability, (4) Understandability, (5) Usability' and (6). Reproducibility of results. NASAs Earth Science data and Information System (ESDIS) Project, along with the 12 EOSDIS Distributed Active Archive Centers (DAACs), has made significant progress in each of these areas over the last decade, and continues to evolve its active archive capabilities. Particular attention is being paid in recent years to ensure that the datasets are published in an easily accessible and citable manner through a unified metadata model, a common metadata repository (CMR), a coherent view through the earthdata.gov website, and assignment of Digital Object Identifiers (DOI) with well-designed landing product information pages.

Data Management↗
NASA NTRS2004

Value-added Data Services at the Goddard Earth Sciences Data and Information Services Center

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), in addition to serving the Earth Science community as one of the major Distributed Active Archives Centers (DAACs), provides much more than just data. Among the value-added services available to general users are subsetting data spatially and/or by parameter, online analysis (to avoid downloading unnecessarily all the data), and assistance in obtaining data from other centers. Services available to data producers and high-volume users include consulting on building new products with standard formats and metadata and construction of data management systems. A particularly useful service is data processing at the DISC (i.e., close to the input data) with the users algorithm. This can take a number of different forms: as a configuration-managed algorithm within the main processing stream; as a stand-alone program next to the on-line data storage; as build-it-yourself code within the Near-Archive Data Mining (NADM) system; or as an on-the-fly analysis with simple algorithms embedded into the web-based tools. Partnerships between the GES DISC and scientists, both producers and users, allow the scientists to concentrate on science, while the GES DISC handles the data management, e.g., formats, integration, and data processing. The existing data management infrastructure at the GES DISC supports a wide spectrum of options: from simple data support to sophisticated on-line analysis tools, producing economies of scale and rapid time-to-deploy. At the same time, such partnerships allow the GES DISC to serve the user community more efficiently and to better prioritize on-line holdings. Several examples of successful partnerships are described in the presentation.

Leptoukh, Gregory G.↗
NASA NTRS2017

Cumulus: NASA's Cloud Based Distributed Active Archive Center Prototype

NASAs Earth Science Data System (ESDS) Program serves as a central cog in order to facilitate the implementation of NASA's Earth Science strategic plan. Since 1994, the ESDS Program has committed to the full and open sharing of Earth science data obtained from NASA instruments to all users. One of the key responsibilities of the ESDS Program is to continuously evolve the entire data and information system to maximize returns on the collected NASA data. An independent review was conducted in 2015 to holistically review the EOSDIS in order to identify gaps. The review recommendations were to investigate two areas: one, whether commercial cloud providers offer potential for storage, processing, and operational efficiencies, and two, the potential development of new data access and analysis paradigms. In response, ESDS has initiated several prototypes investigating the advantages and risks of leveraging cloud computing. This paper describes one such prototyping activity named Cumulus. Cumulus is being designed and developed as a "native" cloud-based data ingest, archive and management system that can be used for all future NASA Earth science data streams. Cumulus will foster design of new analysisvisualization tools that can leverage collocated data from all of the distributed DAACs as well as elastic cloud computing resources.

Earth Science Informatics↗
NASA NTRS2011

Application of ESE Data and Tools to Air Quality Management: Services for Helping the Air Quality Community use ESE Data (SHAirED)

The goal of this REASoN applications and technology project is to deliver and use Earth Science Enterprise (ESE) data and tools in support of air quality management. Its scope falls within the domain of air quality management and aims to develop a federated air quality information sharing network that includes data from NASA, EPA, US States and others. Project goals were achieved through a access of satellite and ground observation data, web services information technology, interoperability standards, and air quality community collaboration. In contributing to a network of NASA ESE data in support of particulate air quality management, the project will develop access to distributed data, build Web infrastructure, and create tools for data processing and analysis. The key technologies used in the project include emerging web services for developing self describing and modular data access and processing tools, and service oriented architecture for chaining web services together to assemble customized air quality management applications. The technology and tools required for this project were developed within DataFed.net, a shared infrastructure that supports collaborative atmospheric data sharing and processing web services. Much of the collaboration was facilitated through community interactions through the Federation of Earth Science Information Partners (ESIP) Air Quality Workgroup. The main activities during the project that successfully advanced DataFed, enabled air quality applications and established community-oriented infrastructures were: develop access to distributed data (surface and satellite), build Web infrastructure to support data access, processing and analysis create tools for data processing and analysis foster air quality community collaboration and interoperability.

Falke, Stefan↗
DOE OSTI2023

FAIR for AI: An interdisciplinary and international community building perspective

A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to include the software, tools, algorithms, and workflows that produce data. FAIR principles are now being adapted in the context of AI models and datasets. Here, we present the perspectives, vision, and experiences of researchers from different countries, disciplines, and backgrounds who are leading the definition and adoption of FAIR principles in their communities of practice, and discuss outcomes that may result from pursuing and incentivizing FAIR AI research. The material for this report builds on the FAIR for AI Workshop held at Argonne National Laboratory on June 7, 2022.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2013

Global Data Assembly Center (GDAC) Report to the GHRSST Science Team

In 2012-2013 the Global Data Assembly Center (GDAC) at NASA's Physical Oceanography Distributed Active Archive Center (PO.DAAC) continued its role as the primary clearinghouse and access node for operational GHRSST data streams, as well as its collaborative role with the NOAA Long Term Stewardship and Reanalysis Facility (LTSRF) for archiving. Our presentation reported on our data management activities and infrastructure improvements since the last science team meeting in 2012.

user reports↗
NASA NTRSDate not supplied

NASA's Earthdata Forum: Bringing the Science Community Together

The Earthdata Forum is an actively maintained and operational user forum for all participating NASA EOSDIS Distributed Active Archive Centers (DAACs) and GCMD. The Forum serves as a cross-DAAC platform from which user communities can obtain authoritative information relating to NASA Earth Science. The Atmospheric Science Data Center (ASDC) DAAC takes on the role of both Project Manager and Hosting Organization. As Project Manager, the ASDC leads and facilitates the project's efforts. The Forum launched February 2020 with five participating DAACs: Atmospheric Science Data Center (ASDC), Crustal Dynamics Data Information System (CDDIS), Goddard Earth Science Data and Information Services Center (GES DISC), Global Hydrology Resource Center (GHRC), and Land Processes DAAC (LP DAAC). Since February 2020, the of Level 1 and Atmosphere Archive and Distribution System DAAC (LAADS), the Ocean Biology DAAC (OBDAAC) and Global Change Master Directory (GCMD) have also joined the Forum. The Forum assumes both a proactive and reactive approach to NASA products scientific queries, allowing the scientific community to ask questions about NASA's products, tools, and services and receive announcements and news regarding products, new data releases, and such. In this presentation, we will describe the Earthdata Forum approach to support the scientific community in detail and discuss the successes and challenges of this approach.

Ingrid Garcia-Solera↗
NASA NTRS2008

The NASA Real Time Mission Monitor - A Situational Awareness Tool for Conducting Tropical Cyclone Field Experiments

The NASA Real Time Mission Monitor (RTMM) is a situational awareness tool that integrates satellite, aircraft state information, airborne and surface instruments, and weather state data in to a single visualization package for real time field experiment management. RTMM optimizes science and logistic decision-making during field experiments by presenting timely data and graphics to the users to improve real time situational awareness of the experiment's assets. The RTMM is proven in the field as it supported program managers, scientists, and aircraft personnel during the NASA African Monsoon Multidisciplinary Analyses (investigated African easterly waves and Tropical Storm Debby and Helene) during August-September 2006 in Cape Verde, the Tropical Composition, Cloud and Climate Coupling experiment during July-August 2007 in Costa Rica, and the Hurricane Aerosonde mission into Hurricane Noel in 2-3 November 2007. The integration and delivery of this information is made possible through data acquisition systems, network communication links, and network server resources built and managed by collaborators at NASA Marshall Space Flight Center (MSFC) and Dryden Flight Research Center (DFRC). RTMM is evolving towards a more flexible and dynamic combination of sensor ingest, network computing, and decision-making activities through the use of a service oriented architecture based on community standards and protocols. Each field experiment presents unique challenges and opportunities for advancing the functionality of RTMM. A description of RTMM, the missions it has supported, and its new features that are under development will be presented.

Goodman, Michael↗
NASA NTRS2002

Global Change Data Center: Mission, Organization, Major Activities, and 2001 Highlights

Rapid efficient access to Earth sciences data is fundamental to the Nation's efforts to understand the effects of global environmental changes and their implications for public policy. It becomes a bigger challenge in the future when data volumes increase further and missions with constellations of satellites start to appear. Demands on data storage, data access, network throughput, processing power, and database and information management are increased by orders of magnitude, while budgets remain constant and even shrink. The Global Change Data Center's (GCDC) mission is to provide systems, data products, and information management services to maximize the availability and utility of NASA's Earth science data. The specific objectives are (1) support Earth science missions be developing and operating systems to generate, archive, and distribute data products and information; (2) develop innovative information systems for processing, archiving, accessing, visualizing, and communicating Earth science data; and (3) develop value-added products and services to promote broader utilization of NASA Earth Sciences Enterprise (ESE) data and information. The ultimate product of GCDC activities is access to data and information to support research, education, and public policy.

Wharton, Stephen W.↗
DOE OSTI2023

Not just for programmers: How GitHub can accelerate collaborative and reproducible research in ecology and evolution

Abstract Researchers in ecology and evolutionary biology are increasingly dependent on computational code to conduct research. Hence, the use of efficient methods to share, reproduce, and collaborate on code as well as document research is fundamental. GitHub is an online, cloud‐based service that can help researchers track, organize, discuss, share, and collaborate on software and other materials related to research production, including data, code for analyses, and protocols. Despite these benefits, the use of GitHub in ecology and evolution is not widespread. To help researchers in ecology and evolution adopt useful features from GitHub to improve their research workflows, we review 12 practical ways to use the platform. We outline features ranging from low to high technical difficulty, including storing code, managing projects, coding collaboratively, conducting peer review, writing a manuscript, and using automated and continuous integration to streamline analyses. Given that members of a research team may have different technical skills and responsibilities, we describe how the optimal use of GitHub features may vary among members of a research collaboration. As more ecologists and evolutionary biologists establish their workflows using GitHub, the field can continue to push the boundaries of collaborative, transparent, and open research.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Laboratory evaluation of open source and commercial electrical conductivity sensor precision and accuracy: How do they compare?

Variation in the electrical conductivity (EC) of water can reveal environmental disturbance and natural dynamics, including factors such as anthropogenic salinization. Broader application of open source (OS) EC sensors could provide an inexpensive method to measure water quality. While studies show that other water quality parameters can be robustly measured with sensors, a similar effort is needed to evaluate the performance of OS EC sensors. To address this need, we evaluated the accuracy (mean error, %) and precision (sample standard deviation) of OS EC sensors in the laboratory via comparison to EC calibration standards using three different OS and OS/commercial-hybrid (OS/C) EC sensors and data logger configurations and two commercial (C) EC sensors and data logger configurations. We also evaluated the effect of cable length (7.5 m and 30 m) and sensor calibration on OS sensor accuracy and precision. We found a significant difference between OS sensor mean accuracy (3.08%) and all other sensors combined (9.23%). Our study also found that EC sensor precision decreased across all sensor configurations with increasing calibration standard EC. There was also a significant difference between OS sensor mean precision (2.85 μS/cm) and the mean precision of all other sensors combined (9.12 μS/cm). Cable length did not affect OS sensor precision. Furthermore, our results suggest that future research should include evaluating how performance is impacted by combining OS sensors with commercial data loggers as this study found significantly decreased performance in OS/commercial-hybrid sensor configurations. To increase confidence in the reliability of OS sensor data, more studies such as ours are needed to further quantify OS sensor performance in terms of accuracy and precision across different settings and OS sensor and data collection platform configurations.

54 ENVIRONMENTAL SCIENCES↗