Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Vast volumes of data are produced by today's scientific simulations and advanced instruments. These data cannot be stored and transferred efficiently because of limited I/O bandwidth, network speed, and storage capacity. Error-bounded lossy compression can be an effective method for addressing these issues: not only can it significantly reduce data size, but it can also control the data distortion based on user-defined error bounds. In practice, many scientific applications have specific requirements or constraints for lossy compression, in order to guarantee that the reconstructed data are valid for post hoc analysis. For example, some datasets contain irrelevant data that should be isolated in particular and users often have intuition regarding value ranges, geospatial regions, and other data subsets that are crucial for subsequent analysis. Existing state-of-the-art error-bounded lossy compressors, however, do not consider these constraints during compression, resulting in inferior compression ratios with respect to user's post hoc analysis, due to the fact that the data itself provides little or no value for post hoc analysis. In this work we address this issue by proposing an optimized framework that can preserve diverse constraints during the error-bounded lossy compression, e.g., cleaning the irrelevant data, efficiently preserving different precision for multiple value intervals, and allowing users to set diverse precision over both regular and irregular regions. We perform our evaluation on a supercomputer with up to 2,100 cores. Experiments with six real-world applications show that our proposed diverse constraints based error-bounded lossy compressor can obtain a higher visual quality or data fidelity on reconstructed data with the same or even higher compression ratios compared with the traditional state-of-the-art compressor SZ. Furthermore, our experiments also demonstrate very good scalability in compression performance compared with the I/O throughput of the parallel file system.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

User Scientific Data Systems: Experience Report

This paper presents an abbreviated history of NASA science data management system development over the past ten years by selecting two case studies, each representative of a distinct era of science data management systems.

Scientific Data↗
DOE OSTI2024

A General Framework for Error-controlled Unstructured Scientific Data Compression

Data compression plays a key role in reducing storage and I/O costs. Traditional lossy methods primarily target data on rectilinear grids and cannot leverage the spatial coherence in unstructured mesh data, leading to suboptimal compression ratios. We present a multi-component, error-bounded compression framework designed to enhance the compression of floating-point unstructured mesh data, which is common in scientific applications. Our approach involves interpolating mesh data onto a rectilinear grid and then separately compressing the grid interpolation and the interpolation residuals. This method is general, independent of mesh types and typologies, and can be seamlessly integrated with existing lossy compressors for improved performance. We evaluated our framework across twelve variables from two synthetic datasets and two real-world simulation datasets. The results indicate that the multi-component framework consistently outperforms state-of-the-art lossy compressors on unstructured data, achieving, on average, a 2.3 − 3.5× improvement in compression ratios, with error bounds ranging from 1 × 10 the −6 to 1×10−2. We further investigate impact of hyperparameters, such as grid spacing and error allocation, to deliver optimal compression ratios in diverse datasets.

Gong, Qian↗
NASA NTRS2022

Scientific Data From Precipitation Driver Response Model Intercomparison Project

This data descriptor reports the main scientific values from General Circulation Models (GCMs) in the Precipitation Driver and Response Model Intercomparison Project (PDRMIP). The purpose of the GCM simulations has been to enhance the scientific understanding of how changes in greenhouse gases, aerosols, and incoming solar radiation perturb the Earth’s radiation balance and its climate response in terms of changes in temperature and precipitation. Here we provide global and annual mean results for a large set of coupled atmospheric-ocean GCM simulations and a description of how to easily extract files from the dataset. The simulations consist of single idealized perturbations to the climate system and have been shown to achieve important insight in complex climate simulations. We therefore expect this data set to be valuable and highly used to understand simulations from complex GCMs and Earth System Models for various phases of the Coupled Model Intercomparison Project

Precipitation Driver Response Model Intercompariso↗
NASA NTRS2000

A Language for Probabilistic Modeling of Scientific Data

We describe a portable container which allows specification of the probabilistic relations between several variables. The applications motivating this work are mainly scientific inference problems.

probabilistic modeling scientific data↗
NASA NTRS1978

SDMS: A scientific data management system

SDMS is a data base management system developed specifically to support scientific programming applications. It consists of a data definition program to define the forms of data bases, and FORTRAN-compatible subroutine calls to create and access data within them. Each SDMS data base contains one or more data sets. A data set has the form of a relation. Each column of a data set is defined to be either a key or data element. Key elements must be scalar. Data elements may also be vectors or matrices. The data elements in each row of the relation form an element set. SDMS permits direct storage and retrieval of an element set by specifying the corresponding key element values. To support the scientific environment, SDMS allows the dynamic creation of data bases via subroutine calls. It also allows intermediate or scratch data to be stored in temporary data bases which vanish at job end.

Massena, W. A.↗
NASA NTRS2013

Researcher-driven Campaigns Engage Nature's Notebook Participants in Scientific Data Collection

One of the many benefits of citizen science projects is the capacity they hold for facilitating data collection on a grand scale and thereby enabling scientists to answer questions they would otherwise not been able to address. Nature's Notebook, the plant and animal phenology observing program of the USA National Phenology Network (USA-NPN) suitable for scientists and non-scientists alike, offers scientifically-vetted data collection protocols and infrastructure and mechanisms to quickly reach out to hundreds to thousands of potential contributors. The USA-NPN has recently partnered with several research teams to engage participants in contributing to specific studies. In one example, a team of scientists from NASA, the New Mexico Department of Health, and universities in Arizona, New Mexico, Oklahoma, and California are using juniper phenology observations submitted by Nature's Notebookparticipants to improve predictions of pollen release and inform asthma and allergy alerts. In a second effort, researchers from the University of Maryland Center for Environmental Science are engaging Nature's Notebookparticipants in tracking leafing phenophases of poplars across the U.S. These observations will be compared to information acquired via satellite imagery and used to determine geographic areas where the tree species are most and least adapted to predicted climate change. Researchers in these partnerships receive benefits primarily in the form of ground observations. Launched in 2010, the juniper pollen effort has engaged participants in several western states and has yielded thousands of observations that can play a role in model ground validation. Periodic evaluation of these observations has prompted the team to improve and enhance the materials that participants receive, in an effort to boost data quality. The poplar project is formally launching in spring of 2013 and will run for three years; preliminary findings from 2013 will be presented. Participants in these special campaigns benefit through direct engagement in science. This form of researcher partnership has now been successfully pilot-tested and implemented in several instances, and provides a template for future research project campaigns.

Crimmins, Theresa M.↗
NASA NTRS2013

Researcher-driven Campaigns Engage Nature's Notebook Participants in Scientific Data Collection

One of the many benefits of citizen science projects is the capacity they hold for facilitating data collection on a grand scale and thereby enabling scientists to answer questions they would otherwise not been able to address. Nature's Notebook, the plant and animal phenology observing program of the USA National Phenology Network (USA-NPN) suitable for scientists and non-scientists alike, offers scientifically-vetted data collection protocols and infrastructure and mechanisms to quickly reach out to hundreds to thousands of potential contributors. The USA-NPN has recently partnered with several research teams to engage participants in contributing to specific studies. In one example, a team of scientists from NASA, the New Mexico Department of Health, and universities in Arizona, New Mexico, Oklahoma, and California are using juniper phenology observations submitted by Nature's Notebookparticipants to improve predictions of pollen release and inform asthma and allergy alerts. In a second effort, researchers from the University of Maryland Center for Environmental Science are engaging Nature's Notebookparticipants in tracking leafing phenophases of poplars across the U.S. These observations will be compared to information acquired via satellite imagery and used to determine geographic areas where the tree species are most and least adapted to predicted climate change. Results/Conclusions Researchers in these partnerships receive benefits primarily in the form of ground observations. Launched in 2010, the juniper pollen effort has engaged participants in several western states and has yielded thousands of observations that can play a role in model ground validation. Periodic evaluation of these observations has prompted the team to improve and enhance the materials that participants receive, in an effort to boost data quality. The poplar project is formally launching in spring of 2013 and will run for three years; preliminary findings from 2013 will be presented. Participants in these special campaigns benefit through direct engagement in science. This form of researcher partnership has now been successfully pilot-tested and implemented in several instances, and provides a template for future research project campaigns.

Crimmins, Theresa M.↗
NASA NTRS2004

Scientific Data Collection/Analysis: 1994-2004

This custom bibliography from the NASA Scientific and Technical Information Program lists a sampling of records found in the NASA Aeronautics and Space Database. The scope of this topic includes technologies for lightweight, temperature-tolerant, radiation-hard sensors. This area of focus is one of the enabling technologies as defined by NASA s Report of the President s Commission on Implementation of United States Space Exploration Policy, published in June 2004.

Source record↗
NASA NTRSDate not supplied

Standardizing Algorithm Documentation For Improved Scientific Data Understanding: The Algorithm Publication Tool Prototype

Algorithm Theoretical Basis Documents (ATBDs) are documents which accompany Earth observation data products generated from algorithms. While ATBDs are essential to scientific reproducibility, these key documents are not standardized and are often difficult to find. In this paper, we present the prototype Algorithm Publication Tool (APT), a cloud-based ATBD authoring and editing tool for NASA’s Earth science data systems. A standardized ATBD information model is also described as well as lessons learned from developing the prototype tool.

Kaylin Bugbee↗
NASA NTRS1971

Scientific data reduction and analysis plan: PI services

This plan comprises two parts. The first concerns the real-time data display to be provided by MSC during the mission. The prime goal is to assess the operation of the UVS and to identify any problem areas that could be corrected during the mission. It is desirable to identify any possible observations of unusual scientific interest in order to repeat these observations at a later point in the mission, or to modify the time line with respect to the operating modes of the UVS. The second part of the plan discusses the more extensive postflight analysis of the data in terms of the scientific objectives of this experiment.

Feldman, P. D.↗
DOE OSTI2024

Immersive Visualization for Scientific Data Analysis

We will present the use of immersive visualization at the National Renewable Energy Laboratory (NREL), showcasing how immersive visualization is advancing scientific research and engineering practices and transforming our day-to-day operations. We are leveraging immersive visualization to support scientific discovery and engineering in various domains, including material design, computational fluid dynamics, immersive analytics, grid modernization, digital twins, and situated visualization. We have observed several benefits across four key areas: enhanced spatial judgments, improved understanding through interaction, increased capacity to embed high-dimensional data, and improved collaboration.

immersive analytics↗
DOE OSTI2023

Advanced Visualization for Scientific Data Analysis and Insight [Slides]

This talk will explore how we have used advanced visualization technologies to support analytical reasoning and knowledge discovery. Specifically, we will present several examples detailing some recent scientific successes using state-of-the-art immersive and high-resolution visualization at the National Renewable Energy Laboratory's Computational Science Center. On multiple occasions, we have observed scientists and engineers discover features in their data using advanced visualization technologies that they had not seen in prior investigations of their data on traditional desktop displays. We have embedded more information into our analytics tools, allowing engineers to explore complex multivariate spaces. We have observed how interactions seem to catalyze understanding.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Hypothesis testing via AI: Generating physically interpretable models of scientific data with machine learning (Full Technical Report)

Deep learning has demonstrated an exceptional ability to solve complex tasks (an engineering success); however, it has done so at the expense of the ability to generate new knowledge (a scientific failure). We propose an alternative framework—entitled Deep Symbolic Regression (DSR)—in which artificial neural networks (NNs) rapidly generate hypotheses about physical relationships among inputs. This framework bypasses the need to interpret an NN altogether, while still leveraging the representational power of deep learning. The resulting models are tractable mathematical expressions, which are inherently and readily human interpretable and can provide insights into underlying physical phenomena. Further, we fold this methodology into the scientific process by allowing the scientist to directly integrate a priori knowledge and beliefs to accelerate learning. We demonstrate this methodology on symbolic regression—the problem of rediscovering underlying expressions describing a dataset—and achieve state-of-the-art performance across a wide variety of symbolic regression problems. Further, we generalize our DSR framework to apply to the more general class of symbolic optimization problems, in which one seeks to optimize a sequence of symbols or “tokens” under a black-box reward function. Examples of other symbolic optimization problems include neural architecture search and computational antibody design. Our generalized tool, Deep Symbolic Optimization (DSO), has been demonstrated on the task of learning symbolic control policies for reinforcement learning environments, and has been adopted as an enabling capability for computational antibody design.

97 MATHEMATICS AND COMPUTING↗