Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,765 records · Page 98
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

FAIR AI models in high energy physics

Abstract The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning models—algorithms that have been trained on data without being explicitly programmed—and more generally, artificial intelligence (AI) models, are an important target for this because of the ever-increasing pace with which AI is transforming scientific domains, such as experimental high energy physics (HEP). In this paper, we propose a practical definition of FAIR principles for AI models in HEP and describe a template for the application of these principles. We demonstrate the template’s use with an example AI model applied to HEP, in which a graph neural network is used to identify Higgs bosons decaying to two bottom quarks. We report on the robustness of this FAIR AI model, its portability across hardware architectures and software frameworks, and its interpretability.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Journey over Destination: Dynamic Sensor Placement Enhances Generalization

Reconstructing complex, high-dimensional global fields from limited data points is a challenge across various scientific and industrial domains. This is particularly important for recovering spatio-temporal fields using sensor data from, for example, laboratory-based scientific experiments, weather forecasting, or drone surveys. Given the prohibitive costs of specialized sensors and the inaccessibility of
certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of machine learning algorithms trained to reconstruct fields given a limited dataset is of critical importance. In this study, we introduce a general
approach that employs moving sensors to enhance data exploitation during the training of an attention based neural network, thereby improving field reconstruction. The training of sensor locations is accomplished using an end-to-end workflow, ensuring
differentiability in the interpolation of field values associated to the sensors, and is simple to implement using differentiable programming. Additionally, we have incorporated a correction mechanism to prevent sensors from entering invalid regions within the domain. We evaluated our method using two distinct datasets; the results show that our approach enhances learning, as evidenced by improved test scores.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

BASIN-3D: A brokering framework to integrate diverse environmental data

Diverse observational and simulation datasets are needed to understand and predict complex ecosystem behavior over seasonal to decadal and century time-scales. Integration of these datasets poses a major barrier towards advancing environmental science, particularly due to differences in the structure and formats of data provided by various sources. Here, we describe BASIN-3D (Broker for Assimilation, Synthesis and Integration of eNvironmental Diverse, Distributed Datasets), a data integration framework designed to dynamically retrieve and transform heterogeneous data from different sources into a common format to provide an integrated view. BASIN-3D enables users to adopt a standardized approach for data retrieval and avoid customizations for the data type or source. We demonstrate the value of BASIN-3D with two use cases that require integration of data from regional to watershed spatial scales. The first application uses the BASIN-3D Python library to integrate time-series hydrological and meteorological data to provide standardized inputs to analytical and machine learning codes in order to predict the impacts of hydrological disturbances on large river corridors of the United States. The second application uses the BASIN-3D Django framework to integrate diverse time-series data in a mountainous watershed in East River, Colorado, United States to enable scientific researchers to explore and download data through an interactive web portal. Thus, BASIN-3D can be used to support data integration for both web-based tools, as well as data analytics using Python scripting and extensions like Jupyter notebooks. The framework is expected to be transferable to and useful for many other field and modeling studies.

Varadharajan, C↗
DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗
NASA NTRS1984

The pilot climate data system

The Pilot Climate Data System (PCDS) is an interactive scientific information management system for locating, obtaining, manipulating, and displaying climate-research data. The PCDS was developed to manage a large collection of data of interest to the National Aeronautics and Space Administration's (NASA) research community and currently provides such support for approximately twenty data sets. In order to provide the PCDS capabilities, NASA's Goddard Space Flight Center (NASA/GSFC) has integrated the capabilities of several general-purpose software packages with specialized software for reading and reformatting the supported data sets. These capabilities were integrated in a manner which allows the PCDS to be easily expanded, either to provide support for additional data sets or to provide additional functional capabilities. This also allows the PCDS to take advantage of new technology as it becomes available, since parts of the system can be replaced with more powerful components without significantly affecting the user interface.

Reph, M. G.↗
NASA NTRS1992

EMASS (tm): An expandable solution for NASA space data storage needs

The data acquisition, distribution, processing, and archiving requirements of NASA and other U.S. Government data centers present significant data management challenges that must be met in the 1990's. The Earth Observing System (EOS) project alone is expected to generate daily data volumes greater than 2 Terabytes (2(10)(exp 12) Bytes). As the scientific community makes use of this data their work product will result in larger, increasingly complex data sets to be further exploited and managed. The challenge for data storage systems is to satisfy the initial data management requirements with cost effective solutions that provide for planned growth. This paper describes the expandable architecture of the E-Systems Modular Automated Storage System (EMASS (TM)), a mass storage system which is designed to support NASA's data capture, storage, distribution, and management requirements into the 21st century.

Peterson, Anthony L.↗
NASA NTRS2017

Towards a Preservation Content Standard for Earth Observation Data

Information from Earth observing missions (remote sensing with airborne and spaceborne instruments, and in situ measurements such as those from field campaigns) is proliferating in the world. Many agencies across the globe are generating important datasets by collecting measurements from instruments on board aircraft and spacecraft, globally and constantly. The data resulting from such measurements are a valuable resource that needs to be preserved for the benefit of future generations. These observations are the primary record of the Earths environment and therefore are the key to understanding how conditions in the future will compare to conditions today. Earth science observational data, derived products and models are used to answer key questions of global significance. In the near-term, as long as the missions data are being used actively for scientific research, it continues to be important to provide easy access to the data and services commensurate with current information technology. For the longer term, when the focus of the research community shifts toward new missions and observations, it is essential to preserve the previous mission data and associated information. This will enable a new user in the future to understand how the data were used for deriving information, knowledge and policy recommendations and to repeat the experiment to ascertain the validity and possible limitations of conclusions reached in the past and to provide confidence in long term trends that depended on data from multiple missions. Organizations that collect, process, and utilize Earth observation data today have a responsibility to ensure that the data and associated content continue to be preserved by them or are gathered and handed off to other organizations for preservation for the benefit of future generations. In order to ensure preservation of complete content necessary for understanding and reusing the data and derived digital products from todays missions, it is necessary to develop a specification of such preservation content. While there are existing standards that address archival and preservation in general, there are no existing international standards or specifications today to address what content should be preserved. The purpose of this paper is to outline briefly the existing standards that apply to preservation, describe a recent effort in getting an international standard in place for specifying preservation content for Earth observation data and derived digital data products and the remaining work needed to arrive at a standard.

ISO Standards↗
NASA NTRS1978

Accessing numeric data via flags and tags: A final report on a real world experiment

An experiment is reported which: extended the concepts of data flagging and tagging to the aerospace scientific and technical literature; generated experience with the assignment of data summaries and data terms by documentation specialists; and obtained real world assessments of data summaries and data terms in information products and services. Inclusion of data summaries and data terms improved users' understanding of referenced documents from a subject perspective as well as from a data perspective; furthermore, a radical shift in document ordering behavior occurred during the experiment toward proportionately more requests for data-summarized items.

Kottenstette, J. P.↗
NASA NTRS1985

The status of the Nimbus-7 earth-radiation-budget data set

Over six years of data from the earth-radiation-budget (ERB) experiment on board the Nimbus-7 spacecraft have been received to date. Five years of data have been archived and are readily available to the scientific community, while the sixth year's data will be archived by the fall of 1985. There are three complementary, but independent, data sets: earth-radiation-budget data, solar-irradiance data, and shortwave- and longwave-radiance data suitable for bidirectional-reflectance studies. The solar and the wide-field-of-view earth-flux channels are still recording high-quality data, but the narrow-field-of-view scanner failed after 20 months. The condition of the three data sets is described. In addition, plans for data-set improvement are discussed and a simple algorithm to improve the wide-field-of-view data is presented.

Kyle, H. L.↗
NASA NTRS1997

Parallelization and Visual Analysis of Multidimensional Fields: Application to Ozone Production, Destruction, and Transport in Three Dimensions

This final report has four sections. We first describe the actual scientific results attained by our research team, followed by a description of the high performance computing research enhancing those results and prompted by the scientific tasks being undertaken. Next, we describe our research in data and program visualization motivated by the scientific research and also enabling it. Last, we comment on the indirect effects this research effort has had on our work, in terms of follow up or additional funding, student training, etc.

Schwan, Karsten↗
NASA NTRS2008

The NASA NEESPI Data Portal to Support Studies of Climate and Environmental Changes in Non-Boreal Europe

NASA NEESPI (Northern Eurasia Earth Science Partnership Initiative) data portal is a NASA funded project that focuses on collecting satellite remote sensing data, providing tools, information, and services in support of NEESPI scientific objectives (Leptoukh, et al., 2007). The data can be accessed online through anonymous ftp, through an advanced data searching and ordering system Mirador that uses keywords to find data quickly in a Google-like interface, and through the Goddard Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni). The portal provides preprocessed data from different satellite sensors and numerical models to the same spatial and temporal resolution and the same projection so that the data can be used easily to perform inter-comparison or relationship studies. In addition, it provides parameter and spatially subsetted data for regional studies. Studies of regional carbon, hydrology, aerosols in non-boreal Europe and their interactions with global climate are very challenging research topics. The NASA NEESPI data portal makes many satellite data available for such studies, including information on land cover types, fire, vegetation index, aerosols, land surface temperature, soil moisture, precipitation, snow/ice, and other parameters. This paper will introduce the features and products available in the system, focusing on the online data 1 tool, Giovanni NEESPI. An example that explores different data through Giovanni NEESPI in temperate region of non-boreal Europe will be presented.

Shen, Suhung↗
NASA NTRS2017

Giovanni - The Bridge Between Data and Science

This article describes new features in the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni), a user-friendly online tool that enables visualization, analysis, and assessment of NASA Earth science data sets without downloading data and software. Since the satellite era began, data collected from Earth-observing satellites have been widely used in research and applications; however, using satellite-based data sets can still be a challenge to many. To facilitate data access and evaluation, as well as scientific exploration and discovery, the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) has developed Giovanni for a wide range of users around the world. This article describes the latest capabilities of Giovanni with examples, and discusses future plans for this innovative system.

GPM↗
DOE OSTI2023

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Database of Stress-Strain Properties Auto-generated from the Scientific Literature using ChemDataExtractor

Abstract There has been an ongoing need for information-rich databases in the mechanical-engineering domain to aid in data-driven materials science. To address the lack of suitable property databases, this study employs the latest version of the chemistry-aware natural-language-processing (NLP) toolkit, ChemDataExtractor, to automatically curate a comprehensive materials database of key stress-strain properties. The database contains information about materials and their cognate properties: ultimate tensile strength, yield strength, fracture strength, Young’s modulus, and ductility values. 720,308 data records were extracted from the scientific literature and organized into machine-readable databases formats. The extracted data have an overall precision, recall and F-score of 82.03%, 92.13% and 86.79%, respectively. The resulting database has been made publicly available, aiming to facilitate data-driven research and accelerate advancements within the mechanical-engineering domain.

Kumar, Pankaj↗
NASA NTRS2020

NASA ESDS Citizen Science Data Working Group

This document provides guidelines for legal, policy, and ethical issues; standards for citizen science data collection and management; information on ensuring usability of citizen science data and communication regarding its use; and best practices for long-term archival of citizen science data.Section1 contains a detailed discussion of policy, ethical, and legal considerations influencing citizen science data collection. Section 2 considers standards for documentation, including documentation of instrumentation, procedures, and the data itself. It concludes with a discussion of how citizen science data should be attributed. Section 3 provides guidance about how to ensure citizen science data are collected and stored in a useable way. It also considers how NASA and data producers should notify the scientific community, including citizen scientists and the public, about citizen science datasets and the scientific conclusions reached using them. Finally, Section 4 provides detailed information regarding what should be archived from projects using a citizen science approach, including data and code. It provides guidance about archive location, process, and timeframe, as well as information about data access and distribution services provided by NASA that may be relevant to data producers working with citizen scientists.

Citizen Science↗
NASA NTRS1994

A New Generation of Intelligent Trainable Tools for Analyzing Large Scientific Image Databases

In a variety of scientific disciplines two-dimensional digital image data is now relied on as a basic component of routine scientific investigation. The proliferation of image acquisition hardware such as multi-spectral remote-sensing platforms, medical imaging sensors, and high-resolution cameras have led to the widespread use of image data in fields such as atmospheric studies, planetary geology, ecology, agriculture, glacielogy, forestry, astronomy, diagnostic medicine, to name but a few.

machine↗
NASA NTRS2012

Global Long-Term SeaWiFS Deep Blue Aerosol Products available at NASA GES DISC

Long-term climate data records about aerosols are needed in order to improve understanding of air quality, radiative forcing, and for many other applications. The Sea-viewing Wide Field-of-view Sensor (SeaWiFS) provides a global well-calibrated 13- year (1997-2010) record of top-of-atmosphere radiance, suitable for use in retrieval of atmospheric aerosol optical depth (AOD). Recently, global aerosol products derived from SeaWiFS with Deep Blue algorithm (SWDB) have become available for the entire mission, as part of the NASA Making Earth Science data records for Use in Research for Earth Science (MEaSUREs) program. The latest Deep Blue algorithm retrieves aerosol properties not only over bright desert surfaces, but also vegetated surfaces, oceans, and inland water bodies. Comparisons with AERONET observations have shown that the data are suitable for quantitative scientific use [1],[2]. The resolution of Level 2 pixels is 13.5x13.5 km2 at the center of the swath. Level 3 daily and monthly data are composed by using best quality level 2 pixels at resolution of both 0.5ox0.5o and 1.0ox1.0o. Focusing on the southwest Asia region, this presentation shows seasonal variations of AOD, and the result of comparisons of 5-years (2003- 2007) of AOD from SWDB (Version 3) and MODIS Aqua (Version 5.1) for Dark Target (MYD-DT) and Deep Blue (MYD-DB) algorithms.

Shen, Suhung↗