Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,225 records · Page 68

Automated Indexing of Structured Scientific Metadata Using Apache Solr

Scientific datasets are continuously growing with the amount of raw data being collected worldwide. This amount of data poses the biggest challenge to web search engines on how to retrieve them efficiently. This paper discusses how major scientific data centers are using popular open-source search platforms such as Solr [1] to retrieve structured data stored in data sources such as relational database management systems using its import handler mechanisms [2]. Additionally, we will also focus on how we can configure Solr to serve advanced full-text, faceted search capabilities, along with its key features, which simplify representing and delivering better performance to the scientific search interfaces.

Guntupally, Kavya↗

Earth-Based Planetary Data

NASA sponsors Earth-based planetary astronomy to provide data needed for: planning future missions; supporting missions already in progress; and enhanching the scientific analysis of data returned by past missions. Such data complement spacecraft data by providing observational coverage of important planetary phenomena at epochs before and after spacecraft encounters, with time scales longer than a spacecraft encounter, and at wavelengths and spectral resolutions not covered by spacecraft instruments. Earth-based planetary data are defined here as: observations made with ground based telescopes, balloon-borne instruments, rocket-borne instruments, and Earth-orbiting spacecraft. Data in the last category differ somewhat in nature and availability from the other three. In all categories, planetary data are only a subset of the total astronomical observations on record. As a result part of the task is to identify and extract (or separately catalog) those observations which are planetary. The data cover wavelengths in the ultraviolet, the visual, and the infrared. The instrumental techniques addressed are: imaging, polarimetry, photometry, spectrophotometry, and spectroscopy.

Source record↗

A Cloud Mask for AIRS

With the preliminary release of AIRS Level 1 and 2 data to the scientific community, there is a growing need for an accurate AIRS cloud mask for data assimilation studies and in producing products derived from cloud free radiances. Current cloud information provided with the AIRS data are limited or based on simplified threshold tests. A multispectral cloud detection approach has been developed for AIRS that utilizes the hyper-spectral capabilities to detect clouds based on specific cloud signatures across the short wave and long wave infrared window regions. This new AIRS cloud mask has been validated against the existing AIRS Level 2 cloud product and cloud information derived from MODIS. Preliminary results for both day and night applications over the continental U.S. are encouraging. Details of the cloud detection approach and validation results will be presented at the conference.

Brubaker, N.↗

Programming Abstractions for Managing Workflows on Tiered Storage Systems

Scientific workflows in High Performance Computing (HPC) environments are processing large amounts of data. The storage hierarchy on HPC systems is getting deeper, driven by new technologies (NVRAMs, SSDs, etc.) There is a need for new programming abstractions that allow users to seamlessly manage data at the workflow level on multi-tiered storage systems, and provide optimal workflow performance and use of storage resources. In previous work, we introduced a software architecture Managing Data on Tiered Storage for Scientific Workflows (MaDaTS) that used a Virtual Data Space (VDS) abstraction to hide the complexities of the underlying storage system while allowing users to control data management strategies. In this article, we detail the data-centric programming abstractions that allow users to manage a workflow around its data on the storage layer. The programming abstractions simplify data management for scientific workflows on multi-tiered storage systems, without affecting workflow performance or storage capacity. We measure the overheads and effectiveness introduced by the programming abstractions of MaDaTS. Our results show that these abstractions can optimally use the storage capacity in lesser capacity storage tiers, and simplify data management without adding any performance overheads.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Materials Data Science Ontology(MDS-Onto): Unifying Domain Knowledge in Materials and Applied Data Science

Ontologies have gained popularity in the scientific community as a way to standardize terminologies in organizations’ data. Although certain cohorts have created frameworks with rules and guidelines on creating ontologies, there exist significant variations in how Materials Science ontologies are currently developed. We seek to provide guidance in the form of a unified automated framework for developing interoperable and modular ontologies for Materials Data Science that simplifies the ontology terms matching by establishing a semantic bridge up to the Basic Formal Ontology(BFO). This framework provides key recommendations on how ontologies should be positioned within the semantic web, what knowledge representation language is recommended, and where ontologies should be published online to boost their findability and interoperability. Two fundamental components of the MDS-Onto framework are the bilingual package called FAIRmaterials for ontology creation and FAIRLinked, for FAIR data creation. To showcase the practical capabilities of FAIRmaterials, we present two exemplar domain ontologies of MDS-Onto: Synchrotron X-Ray Diffraction and Photovoltaics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

A New Default Colormap for ParaView

ParaView is one of the most prominent software tools for scientific visualization used by scientists around the world. Color is a primary conduit to visually map data to its representation and, thus, enable investigation and interpretation of the data. Colormap selection has a significant impact on the data revealed; its design and selection is a critical aspect of scientific data visualization. A common choice for a user is the program’s default colormap, so careful consideration of this default is consequential. Although the current default colormap in ParaView, a succession of hues from cool blue to warm red, has served the community well, research shows that more nuanced colormap configurations increase discriminability while maintaining other critical metrics. These findings inspire us to revisit and update the default colors in ParaView. Here, in this study, we present a new ParaView default colormap, the criteria and methods of development, and example visualizations and analytic metrics.

97 MATHEMATICS AND COMPUTING↗

Combustion machine learning: Principles, progress and prospects

Progress in combustion science and engineering has led to the generation of large amounts of data from large-scale simulations, high-resolution experiments, and sensors. This corpus of data offers enormous opportunities for extracting new knowledge and insights—if harnessed effectively. Machine learning (ML) techniques have demonstrated remarkable success in data analytics, thus offering a new paradigm for data-intense analyses and scientific investigations through combustion machine learning (CombML). While data-driven methods are utilized in various combustion areas, recent advances in algorithmic developments, the accessibility of open-source software libraries, the availability of computational resources, and the abundance of data have together rendered ML techniques ubiquitous in scientific analysis and engineering. This article examines ML techniques for applications in combustion science and engineering. Starting with a review of sources of data, data-driven techniques, and concepts, we examine supervised, unsupervised, and semi-supervised ML methods. Various combustion examples are considered to illustrate and to evaluate these methods. Next, we review past and recent applications of ML approaches to problems in combustion, spanning fundamental combustion investigations, propulsion and energy-conversion systems, and fire and explosion hazards. Challenges unique to CombML are discussed and further opportunities are identified, focusing on interpretability, uncertainty quantification, robustness, consistency, creation and curation of benchmark data, and the augmentation of ML methods with prior combustion-domain knowledge.

33 ADVANCED PROPULSION SYSTEMS↗

Real-time remote scientific model validation

This paper describes flight results from the use of a CLIPS-based validation facility to compare analyzed data from a space life sciences (SLS) experiment to an investigator's preflight model. The comparison, performed in real-time, either confirms or refutes the model and its predictions. This result then becomes the basis for continuing or modifying the investigator's experiment protocol. Typically, neither the astronaut crew in Spacelab nor the ground-based investigator team are able to react to their experiment data in real time. This facility, part of a larger science advisor system called Principal Investigator in a Box, was flown on the space shuttle in October, 1993. The software system aided the conduct of a human vestibular physiology experiment and was able to outperform humans in the tasks of data integrity assurance, data analysis, and scientific model validation. Of twelve preflight hypotheses associated with investigator's model, seven were confirmed and five were rejected or compromised.

Frainier, Richard↗

The NASA AfriSAR Campaign: Airborne SAR and Lidar Measurements of Tropical Forest Structure and Biomass in Support of Current and Future Space Missions

In 2015 and 2016, the AfriSAR campaign was carried out as a collaborative effort among international space and National Park agencies (ESA, NASA, ONERA, DLR, ANPN and AGEOS) in support of the upcoming ESA BIOMASS, NASA-ISRO Synthetic Aperture Radar (NISAR) and NASA Global Ecosystem Dynamics Initiative (GEDI) missions. The NASA contribution to the campaign was conducted in 2016 with the NASA LVIS (Land Vegetation and Ice Sensor) Lidar, the NASA L-band UAVSAR (Uninhabited Aerial Vehicle Synthetic Aperture Radar). A central motivation for the AfriSAR deployment was the common AGBD estimation requirement for the three future spaceborne missions, the lack of sufficient airborne and ground calibration data covering the full range of ABGD in tropical forest systems, and the intercomparison and fusion of the technologies. During the campaign, over 7000 km2 of waveform Lidar data from LVIS and 30000 km2 of UAVSAR data were collected over 10 key sites and transects. In addition, field measurements of forest structure and biomass were collected in sixteen 1 hectare sized plots. The campaign produced gridded Lidar canopy structure products, gridded aboveground biomass and associated uncertainties, Lidar based vegetation canopy cover profile products, Polarimetric Interferometric SAR and Tomographic SAR products and field measurements. Our results showcase the types of data products and scientific results expected from the spaceborne Lidar and SAR missions; we also expect that the AfriSAR campaign data will facilitate further analysis and use of waveform Lidar and multiple baseline polarimetric SAR datasets for carbon cycle, biodiversity, water resources and more applications by the greater scientific community.

AfriSAR↗

Modeling Astrophysical Explosions and Nucleosynthesis (Final Technical Report)

The goal of this project was to further our theoretical understanding of the extreme physics and heavy element nucleosynthesis occurring in stellar explosions, in particular supernovae and neutron star mergers. To study these systems, we further developed and applied numerical simulations (radiation hydrodynamical models and nuclear reaction networks) run on high performance computers. The numerical calculations and companion theoretical analysis helped to better define the sites of astrophysical nucleosynthesis, and make predictions of the associated observables (nucleosynthetic yields and electromagnetic signals) that can be used to anticipate and interpret experimental data. This project enhances the scientific value of the nuclear data obtained from current and future experimental facilities (e.g., rare isotope beams) by improving our understanding of the astrophysical context and specific physical conditions under which we expect such reactions to take place in Nature.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Continuous Baseline Microphysical Retrieval (MICROBASE) Value-Added Product Report

This technical report describes the Continuous Baseline Microphysical Retrieval (MICROBASE) Value-Added Product (VAP) produced operationally by the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) User Facility. MICROBASE provides a continuous estimate of cloud microphysical properties at ARM fixed observatories and ARM Mobile Facility (AMF) sites. It is designed to run operationally and provide data to the ARM Data Center for scientific distribution. This technical report presents an overview of the VAP as a resource for data users and ongoing records for major updates to these products.

54 ENVIRONMENTAL SCIENCES↗

ERTS-1 DCS technical support provided by Wallops Station

Wallops Station accepted the tasks of providing ground truth to several ERTS investigators, operating a DCP repair depot, designing and building an airborne DCP Data Acquisition System, and providing aircraft underflight support for several other investigators. Additionally, the data bank is generally available for use by ERTS and other investigators that have a scientific interest in data pertaining to the Chesapeake Bay area. Working with DCS has provided a means of evaluating the system as a data collection device possibly applicable to ongoing Earth Resources Program activities in the Chesapeake Bay area as well as providing useful data and services to other ERTS investigators. The two areas of technical support provided by Wallops, ground truth stations and repair for DCPs, are briefly discussed.

Smith, R.↗

An interactive, discipline-independent data visualization system

The data visualization techniques used by National Space Science Data Center Graphics System are discussed. The system includes a self-describing data abstraction for the storage and manipulation of multidimensional data for discipline-independent scientific applications. The system's data format and graphics display are described. Consideration is given to techniques developed for the rapid display and manipulation of large, complex data sets and for manipulating algorithms with portable implementations which can operate on any data object or geometry. The processes of operating the system are given and examples are presented from applying the system to various types of data, including TOMS data, data from the International Satellite Cloud Climatology Project, and IRAS data.

Treinish, Lloyd A.↗

SPASE: Current Uses, Tools, and Plans

The Space Physics Archive Search and Extract (SPASE) project is an international collaboration among Heliophysics (solar and space physics) groups concerned with data acquisition and archiving. Within this community there are a variety of old and new data centers, resident archives, "virtual observatories", etc. acquiring, holding, and distributing data. The main product of the SPASE group is an XML-based SPASE Data Model now in operational use to enable searches for and ultimate acquisition of data of interest to a researcher. The SPASE Data Model defines the content of resource descriptions (metadata). The intent is to describe all SCientifically usable Heliophysics data sets using the Data Model. Another product of the SPASE group, in collaboration with NASA's Virtual Observatories, is a set of tools and services which work with SPASE meta data. This includes Registry Services which can retrieve and render metadata using resource identifiers and facilitate the downloading of the data referenced by the meta data. The SPASE Data Model has also been used as a vocabulary in specialized data models. One example is the Heliophysics Event List Manager (HELM) model. The SPASE Data Model is also being expanded to provide the means for more detailed description of data sets with the aim of enabling more automated ingestion and use of the data through detailed format descriptions. The evolution is based on a number of lessons learned and feedback from our community. Some of the lessons learned are unique to Heliophysics, and some are common to the various data diSCiplines. We will discuss the present state of SPASE usage, the role the SPASE Data Model can play in speCialized data models and how we foresee the development direction in the future.

Thieman, J. R.↗

Ionosphere Plasma State Determination in Low Earth Orbit from International Space Station Plasma Monitor

A plasma diagnostic package is deployed on the International Space Station (ISS). The system - a Floating Potential Measurement Unit (FPMU) - is used by NASA to monitor the electrical floating potential of the vehicle to assure astronaut safety during extravehicular activity. However, data from the unit also reflects the ionosphere state and seems to represent an unutilized scientific resource in the form of an archive of scientific plasma state data. The unit comprises a Floating Potential probe and two Langmuir probes. There is also an unused but active plasma impedance probe. The data, at one second cadence, are collected, typically for a two week period surrounding extravehicular activity events. Data is also collected any time a visiting vehicle docks with ISS and also when any large solar events occur. The telemetry system is unusual because the package is mounted on a television camera stanchion and its data is impressed on a video signal that is transmitted to the ground and streamed by internet to two off center laboratory locations. The data quality has in the past been challenged by weaknesses in the integrated ground station and distribution systems. These issues, since mid‐2010, have been largely resolved and the ground stations have been upgraded. Downstream data reduction has been developed using physics based modeling of the electron and ion collecting character in the plasma. Recursive algorithms determine plasma density and temperature from the raw Langmuir probe current voltage sweeps and this is made available in real time for situational awareness. The purpose of this paper is to describe and record the algorithm for data reduction and to show that the Floating probe and Langmuir probes are capable of providing long term plasma state measurement in the ionosphere. Geophysical features such as the Appleton anomaly and high latitude modulation at the edge of the Auroral zones are regularly observed in the nearly circular, 51 deg inclined, 400 km altitude ISS orbit. Evidence of waves in the ion collection current data is seen in geographic zones known to exhibit the spread‐F phenomenon. An anomaly in the current collection characteristic of the cylindrical probe appears also too be organized by the geomagnetic field.

Kramer, Leonard↗

SLR Data: from Station to User

The ILRS provides laser ranging data and products on an operational basis to geodesy analysts as well as a broader scientific community. ILRS is one of four services within the International Association for Geodesy (IAG) supporting space geodesy. IAG established these services to facilitate international cooperation and scientific research: Networks, Data centers, Analysis groups. Services perform successful operations through cooperation of many international organizations, leveraging their respective resources to all levels of service functionality.

Noll, Carey↗

Distributed Augmentation, Hypersweeps, and Branch Decomposition of Contour Trees for Scientific Exploration

Contour trees describe the topology of level sets in scalar fields and are widely used in topological data analysis and visualization. A main challenge of utilizing contour trees for large-scale scientific data is their computation at scale using highperformance computing. To address this challenge, recent work has introduced distributed hierarchical contour trees for distributed computation and storage of contour trees. However, effective use of these distributed structures in analysis and visualization requires subsequent computation of geometric properties and branch decomposition to support contour extraction and exploration. In this work, we introduce distributed algorithms for augmentation, hypersweeps, and branch decomposition that enable parallel computation of geometric properties, and support the use of distributed contour trees as query structures for scientific exploration. Finally, we evaluate the parallel performance of these algorithms and apply them to identify and extract important contours for scientific visualization.

97 MATHEMATICS AND COMPUTING↗

Reliable and Efficient Machine Learning (Final Technical Report)

Modern scientific experiments generate massive amounts of data at a pace much faster than humans can manually analyze. While machine learning has revolutionized commercial data analysis (such as recommending movies or recognizing faces), applying these tools to complex scientific discovery is challenging because scientific answers must be precise, interpretable, and adhere to physical laws. The research under this project aims to develop new mathematical tools and computer algorithms specifically designed for scientific applications. Major progress has been made in automatically cleaning and deconstructing messy experimental data, analyzing the visual information of physical phenomena, determining the underlying physical variables, and providing rig orous mathematical analysis of interesting algorithms and concepts widely used in machine learning. This project addressed the critical gap between our ability to generate massive scientific data and our ability to extract interpretable information from it. We established mathematical foundations for Scientific Machine Learning (SciML) aimed at effective data analytics and automated discovery. Our work focused on three core objectives: (1) developing reliable feature extraction methods for dynamic high-dimensional data, (2) establishing mathematical foundations for discovering dynamics via neural networks, and (3) creating rigorous optimization techniques for these models. Key outcomes come from two fronts. On the practical side, they include the development of algorithms that significantly enhance the extraction of signals from field data, as well as the capability to handle situations that exhibit smooth variations or physical stretching due to temperature changes. They also include the creation of an automated framework for discovering fundamental state variables from raw experimental data, demonstrating the ability to identify intrinsic physical dimensions without prior knowledge of the governing laws. On the theoretical front, the research results in theoretical advances in Optimal Transport, a widely used notion in SciML, specifically regarding functions with fixed-size nodal sets, provide sharp bounds relevant to uncertainty quantification. Meanwhile, the outcomes also include the establishment of convergence theories for nonlocal gradient descent methods, enabling robust optimization with noisy data in high-dimensional settings commonly encountered in scientific modeling. The project also helps creating opportunities to train the next generation of researchers, equipping them with the necessary technical skills for today’s workplace and preparing them for future advances.

97 MATHEMATICS AND COMPUTING↗