Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data diversity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

The Quickest, Lowest-cost Lunar Resource Assessment Program: Integrated High-tech Earth-based Astronomy

Science and technology applications for the Moon have not fully kept pace with technical advancements in sensor development and analytical information extraction capabilities. Appropriate unanswered questions for the Moon abound, but until recently there has been little motivation to link sophisticated technical capabilities with specific measurement and analysis projects. Over the last decade enormous technical progress has been made in the development of (1) CCD photometric array detectors; (2) visible to near-infrared imaging spectrometers; (3)infrared spectroscopy; (4) high-resolution dual-polarization radar imaging at 3.5, 12, and 70 cm; and equally important (5) data analysis and information extraction techniques using compact powerful computers. Parts of each of these have been tested separately, but there has been no programmatic effort to develop and optimize instruments to meet lunar science and resource assessment needs (e.g., specific wavelength range, resolution, etc.) nor to coordinate activities so that the symbiotic relation between different kinds of data can be fully realized. No single type of remotely acquired data completely characterizes the lunar environment, but there has been little opportunity for integration of diverse advanced sensor data for the Moon. Two examples of technology concepts for lunar measurements are given. Using VIS/near-IR spectroscopy, the mineral composition of surface material can be derived from visible and near-infrared radiation reflected from the surface. The surface and subsurface scattering properties of the Moon can be analyzed using radar backscattering imaging.

Pieters, Carle M.↗
NASA NTRSDate not supplied

Supporting Global Air Quality Management Needs With A Flexible Data Fusion Tool for Estimation and Forecasting in Google Earth Engine

High spatial and temporal resolution air quality estimation and forecasting can be enhanced by combining global data sources, like chemical transport models and satellite remote sensing, with local information from regulatory and low-cost air quality monitors. Successful integration of data from these diverse sources is complicated by many factors, however, including differences in spatial and temporal resolution, data availability and latency issues, varying data quality, and large computational and data storage requirements. This presentation will provide an overview of a NASA-funded effort to develop the foundation for future operationalization of air quality forecasting for world-wide end-users and integration into their air quality management decision processes, which will be achieved in future phases of this multi-year project. We will summarize our progress in developing a data fusion system using the Google Earth Engine platform which can integrate model, satellite, and surface-level monitoring datasets to enhance estimation and forecasting of air-quality-relevant pollutants at sub-daily and sub-city scales. The tool is being developed in close cooperation with several city- and regional-level air quality managers in the USA and around the world. Our end-goal is to provide these air quality managers with the information they need to assess and anticipate the impacts of poor air quality, track changes in air quality due to ongoing mitigation efforts and land use changes, and identify ways to improve their air quality monitoring strategies. This presentation will focus on recent advances achieved through the project, including integration of multiple air quality datasets in a prototype data fusion system in Google Earth Engine, the quantification of uncertainties associated with our data fusion approach, and the development of user interfaces and visualization tools to convey air quality information in a way which best meets end-user needs.

Carl Malings↗
NASA NTRSDate not supplied

Supporting Global Air Quality Management Needs With A Flexible Data Fusion Tool for Estimation and Forecasting in Google Earth Engine

High spatial and temporal resolution air quality estimation and forecasting can be enhanced by combining global data sources, like chemical transport models and satellite remote sensing, with local information from regulatory and low-cost air quality monitors. Successful integration of data from these diverse sources is complicated by many factors, however, including differences in spatial and temporal resolution, data availability and latency issues, varying data quality, and large computational and data storage requirements. This presentation will provide an overview of a NASA-funded effort to develop the foundation for future operationalization of air quality forecasting for world-wide end-users and integration into their air quality management decision processes, which will be achieved in future phases of this multi-year project. We will summarize our progress in developing a data fusion system using the Google Earth Engine platform which can integrate model, satellite, and surface-level monitoring datasets to enhance estimation and forecasting of air-quality-relevant pollutants at sub-daily and sub-city scales. The tool is being developed in close cooperation with several city- and regional-level air quality managers in the USA and around the world. Our end-goal is to provide these air quality managers with the information they need to assess and anticipate the impacts of poor air quality, track changes in air quality due to ongoing mitigation efforts and land use changes, and identify ways to improve their air quality monitoring strategies. This presentation will focus on recent advances achieved through the project, including integration of multiple air quality datasets in a prototype data fusion system in Google Earth Engine, the quantification of uncertainties associated with our data fusion approach, and the development of user interfaces and visualization tools to convey air quality information in a way which best meets end-user needs.

Carl Malings↗
NASA NTRS2017

Collaborative Data Publication Utilizing the Open Data Repository's (ODR) Data Publisher

Introduction: For small communities in diverse fields such as astrobiology, publishing and sharing data can be a difficult challenge. While large, homogenous fields often have repositories and existing data standards, small groups of independent researchers have few options for publishing standards and data that can be utilized within their community. In conjunction with teams at NASA Ames and the University of Arizona, the Open Data Repository's (ODR) Data Publisher has been conducting ongoing pilots to assess the needs of diverse research groups and to develop software to allow them to publish and share their data collaboratively. Objectives: The ODR's Data Publisher aims to provide an easy-to-use and implement software tool that will allow researchers to create and publish database templates and related data. The end product will facilitate both human-readable interfaces (web-based with embedded images, files, and charts) and machine-readable interfaces utilizing semantic standards. Characteristics: The Data Publisher software runs on the standard LAMP (Linux, Apache, MySQL, PHP) stack to provide the widest server base available. The software is based on Symfony (www.symfony.com) which provides a robust framework for creating extensible, object-oriented software in PHP. The software interface consists of a template designer where individual or master database templates can be created. A master database template can be shared by many researchers to provide a common metadata standard that will set a compatibility standard for all derivative databases. Individual researchers can then extend their instance of the template with custom fields, file storage, or visualizations that may be unique to their studies. This allows groups to create compatible databases for data discovery and sharing purposes while still providing the flexibility needed to meet the needs of scientists in rapidly evolving areas of research. Research: As part of this effort, a number of ongoing pilot and test projects are currently in progress. The Astrobiology Habitable Environments Database Working Group is developing a shared database standard using the ODR's Data Publisher and has a number of example databases where astrobiology data are shared. Soon these databases will be integrated via the template-based standard. Work with this group helps determine what data researchers in these diverse fields need to share and archive. Additionally, this pilot helps determine what standards are viable for sharing these types of data from internally developed standards to existing open standards such as the Dublin Core (http://dublincore.org) and Darwin Core (http://rs.twdg.org) metadata standards. Further studies are ongoing with the University of Arizona Department of Geosciences where a number of mineralogy databases are being constructed within the ODR Data Publisher system. Conclusions: Through the ongoing pilots and discussions with individual researchers and small research teams, a definition of the tools desired by these groups is coming into focus. As the software development moves forward, the goal is to meet the publication and collaboration needs of these scientists in an unobtrusive and functional way.

easy to use and implement software tool↗
DOE OSTI2025

Data for "Which plant traits increase soil carbon sequestration? Empirical evidence from a long-term poplar genetic diversity trial"

This archive contains all data and code used by the following publication: Field, J. L., Sloan, B. P., Craig, M. E., Calloway, P., Ottinger, S. L., Mead, T., Abramoff, R. Z., Venegas, M. P., Chhetri, H. B., Haiby, K., Kalluri, U. C., Muchero, W., Schadt, C. W., & Mayes, M. A. (2025). Which plant traits increase soil carbon sequestration? Empirical evidence from a long-term poplar genetic diversity trial (p. 2025.02.17.638464). bioRxiv. https://doi.org/10.1101/2025.02.17.638464 Our analysis combined several soil and root data sets collected by Oak Ridge National Laboratory (ORNL) researchers/collaborators from the Clatskanie Poplar Common Garden in Clatskanie, OR by from 2009-2024. The raw data data files are located */02-data/01-raw/* which we harmonized using the codes in */01-codes/01-harmonize-clatskanie-data-pub.qmd*. The final processed data set used in the paper is found at */02-data/02-processed/clatskanie-c-fit-data.csv* and its columns are described in the table below.

Sloan, Brandon [ORNL] (ORCID:0000000316304271)↗
DOE OSTI2024

Fungal community composition predicts forest carbon storage at a continental scale

Forest soils harbor hyper-diverse microbial communities which fundamentally regulate carbon and nutrient cycling across the globe. Directly testing hypotheses on how microbiome diversity is linked to forest carbon storage has been difficult, due to a lack of paired data on microbiome diversity and in situ observations of forest carbon accumulation and storage. Here, we investigated the relationship between soil microbiomes and forest carbon across 238 forest inventory plots spanning 15 European countries. We show that the composition and diversity of fungal, but not bacterial, species is tightly coupled to both forest biotic conditions and a seven-fold variation in tree growth rates and biomass carbon stocks when controlling for the effects of dominant tree type, climate, and other environmental factors. This linkage is particularly strong for symbiotic endophytic and ectomycorrhizal fungi known to directly facilitate tree growth. Since tree growth rates in this system are closely and positively correlated with belowground soil carbon stocks, we conclude that fungal composition is a strong predictor of overall forest carbon storage across the European continent.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Ontology-Enriched Specifications Enabling Findable, Accessible, Interoperable, and Reusable Marine Metagenomic Datasets in Cyberinfrastructure Systems

Marine microbial ecology requires the systematic comparison of biogeochemical and sequence data to analyze environmental influences on the distribution and variability of microbial communities. With ever-increasing quantities of metagenomic data, there is a growing need to make datasets Findable, Accessible, Interoperable, and Reusable (FAIR) across diverse ecosystems. FAIR data is essential to developing analytical frameworks that integrate microbiological, genomic, ecological, oceanographic, and computational methods. Although community standards defining the minimal metadata required to accompany sequence data exist, they haven’t been consistently used across projects, precluding interoperability. Moreover, these data are not machine-actionable or discoverable by cyberinfrastructure systems. By making ‘omic and physicochemical datasets FAIR to machine systems, we can enable sequence data discovery and reuse based on machine-readable descriptions of environments or physicochemical gradients. In this work, we developed a novel technical specification for dataset encapsulation for the FAIR reuse of marine metagenomic and physicochemical datasets within cyberinfrastructure systems. This includes using Frictionless Data Packages enriched with terminology from environmental and life-science ontologies to annotate measured variables, their units, and the measurement devices used. This approach was implemented in Planet Microbe, a cyberinfrastructure platform and marine metagenomic web-portal. Here, we discuss the data properties built into the specification to make global ocean datasets FAIR within the Planet Microbe portal. We additionally discuss the selection of, and contributions to marine-science ontologies used within the specification. Finally, we use the system to discover data by which to answer various biological questions about environments, physicochemical gradients, and microbial communities in meta-analyses. This work represents a future direction in marine metagenomic research by proposing a specification for FAIR dataset encapsulation that, if adopted within cyberinfrastructure systems, would automate the discovery, exchange, and re-use of data needed to answer broader reaching questions than originally intended.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

The Persistent Challenge of Data Locality in the Post-Exascale Era

The era of exascale computing, exemplified by systems like Frontier achieving exaflop-level performance, marks a milestone. However, the quest for sheer compute power leads to strong imbalance in system design. Hence, scaling advancements in memory, network bandwidth, and storage are also necessary and pose challenges, with a crucial need to address data locality issues. This article underscores the fundamental importance of data locality as a key abstraction for optimizing application performance. Despite notable software solutions, the growing complexity of parallelism and memory hierarchy demands performance-portable data locality solutions across diverse computing platforms. Additionally, the article revisits data locality aspects, covering hardware considerations, application perspectives, software stack abstractions, and tool support. It concludes with insights into data locality challenges and opportunities, emphasizing the ongoing significance of collaborative research for progress in this critical issue.

Unat, Didem [Koc University, Istanbul (Turkey)] (O↗
NASA NTRS1995

A data reduction, management, and analysis system for a 10-terabyte data set

Within 12 months a 5-year space-based research investigation with an estimated daily data volume of 10 to 15 gigabytes will be launched. Our instrument/analysis team will analyze 2 to 8 gigabytes per day from this mission. Most of these data will be spatial and multispectral collected from nine sensors covering the UV/Visible/NlR spectrum. The volume and diversity of these data and the nature of its analysis require a very robust reduction and management system. This paper is a summary of the systems requirements and a high-level description of a solution. The paper is intended as a case study of the problems and potential solutions faced by the new generation of Earth observation data support systems.

DeMajistre, R.↗
DOE OSTI2014

InterMine Webservices for Phytozome (Rev2)

A datawarehousing framework for information provides a useful infrastructure for providers and users of genomic data. For providers, the infrastructure give them a consistent mechanism for extracting raw data. While for the users, the web services supported by the software allows them to make complex, and often unique, queries of the data. Previously, phytozome.net used BioMart to provide the infrastructure. As the complexity, scale and diversity of the dataset as grown, we decided to implement an InterMine web service on our servers. This change was largely motivated by the ability to have a more complex table structure and richer web reporting mechanism than BioMart. For InterMine to achieve its more complex database schema it requires an XML description of the data and an appropriate loader. Unlimited one-to-many and many-to-many relationship between the tables can be enabled in the schema. We have implemented support for:1.) Genomes and annotations for the data in Phytozome. This set is the 48 organisms currently stored in a back end CHADO datastore. The data loaders are modified versions of the CHADO data adapters from FlyMine. 2.) Interproscan results from all proteins in the Phytozome database. 3.) Clusters of proteins into a grouped heirarchically by similarity. 4.) Cufflinks results from tissue-specific RNA-Seq data of Phytozome organisms. 5.) Diversity data (GATK and SnpEFF results) from a set of individual organism. The last two datatypes are new in this implementation of our web services. We anticipate that the scale of these data will increase considerably in the near future.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2019

Tracking materials science data lineage to manage millions of materials experiments and analyses

Abstract In an era of rapid advancement of algorithms that extract knowledge from data, data and metadata management are increasingly critical to research success. In materials science, there are few examples of experimental databases that contain many different types of information, and compared with other disciplines, the database sizes are relatively small. Underlying these issues are the challenges in managing and linking data across disparate synthesis and characterization experiments, which we address with the development of a lightweight data management framework that is generally applicable for experimental science and beyond. Five years of managing experiments with this system has yielded the Materials Experiment and Analysis Database (MEAD) that contains raw data and metadata from millions of materials synthesis and characterization experiments, as well as the analysis and distillation of that data into property and performance metrics via software in an accompanying open source repository. The unprecedented quantity and diversity of experimental data are searchable by experiment and analysis attributes generated by both researchers and data processing software. The search web interface allows users to visualize their search results and download zipped packages of data with full annotations of their lineage. The enormity of the data provides substantial challenges and opportunities for incorporating data science in the physical sciences, and MEAD’s data and algorithm management framework will foster increased incorporation of automation and autonomous discovery in materials and chemistry research.

Soedarmadji, Edwin↗
DOE OSTI2022

The case for data science in experimental chemistry: examples and recommendations

The physical sciences community is increasingly taking advantage of the possibilities offered by modern data science to solve problems in experimental chemistry and potentially to change the way we design, conduct and understand results from experiments. Successfully exploiting these opportunities involves considerable challenges. In this Expert Recommendation, we focus on experimental co-design and its importance to experimental chemistry. We provide examples of how data science is changing the way we conduct experiments, and we outline opportunities for further integration of data science and experimental chemistry to advance these fields. Our recommendations include establishing stronger links between chemists and data scientists; developing chemistry-specific data science methods; integrating algorithms, software and hardware to ‘co-design’ chemistry experiments from inception; and combining diverse and disparate data sources into a data network for chemistry research.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Time series methods for the analysis of soundscapes and other cyclical ecological data

Biodiversity monitoring has entered an era of ‘big data’, exemplified by a near-continuous collection of sounds, images, chemical and other signals from organisms in diverse ecosystems. Such data streams have the potential to help identify new threats, assess the effectiveness of conservation interventions, as well as generate new ecological insights. However, appropriate analytical methods are often still missing, particularly with respect to characterizing cyclical temporal patterns. Here, we present a framework for characterizing and analysing ecological responses that represent nonstationary, complex temporal patterns and demonstrate the value of using Fourier transforms to decorrelate continuous data points. In our example, we use a framework based on three approaches (spectral analysis, magnitude squared coherence, and principal component analysis) to characterize differences in tropical forest soundscapes within and across sites and seasons in Gabon. By reconstructing the underlying, cyclic behaviour of the soundscape for each site, we show how one can identify circadian patterns in acoustic activity. Soundscapes in the dry season had a complex diel cycle, requiring multiple harmonics to represent daily variation, while in the wet season there was less variance attributable to the daily cyclic patterns. Our framework can be applied to most continuous, or near-continuous ecological data collected at a fine temporal resolution, allowing ecologists to explore patterns of temporal autocorrelation at multiple levels for biologically meaningful trends. Such methods will become indispensable as biological big data are used to understand the impact of anthropogenic pressures on biodiversity and to inform efforts to mitigate them.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Combining Molecular Simulations and Machine Learning to Comprehensively Explore Adsorption Space (Final Report)

This is the final report for the project "Combining Molecular Simulations and Machine Learning to Comprehensively Explore Adsorption Space". The central aim of this project was to develop modeling tools that vastly increase the diversity of molecules for which reliable adsorption isotherms in nanoporous materials are available. Within the project, molecular simulation data for a diverse set of examples was combined with machine-learning (ML) methods to rapidly predict adsorption equilibria in crystalline nanoporous materials (primarily MOFs).

36 MATERIALS SCIENCE↗
NASA NTRS1992

SAVS: A Space Analysis and Visualization System

We propose to develop, test, demonstrate, and deliver to NASA a powerful and versatile data acquisition, manipulation, analysis and visualization system which will enhance scientific capabilities in the display and interpretation of diverse and distributed data within an integrated user-friendly environment. Our approach exploits existing technologies and combines three major elements into an easy-to-use interactive package: (1) innovative visualization software; (2) advanced database techniques; and (3) a rich set of mathematical and image processing tools. Visualization capabilities will include one-, two-, and three-dimensional displays, along with animation, compression, warping and slicing functions. Analysis tools will include generic mathematical and statistical techniques along with the ability to use large scale models for interactive interpretation of large volume data sets. Our system will be implemented on Sun and DEC UNIX workstations and on the Stardent Graphics Super computer. Our final deliverable will include complete documentation and a NASA/NSF-CDAW/SUNDIAL campaign demonstration.

Szuszczewicz, Edward P.↗
NASA NTRS1993

SAVS: A Space Analysis and Visualization System

We propose to develop, test, demonstrate, and deliver to NASA a powerful and versatile data acquisition, manipulation, analysis and visualization system which will enhance scientific capabilities in the display and interpretation of diverse and distributed data within an integrated user-friendly environment. Our approach exploits existing technologies and combines three major elements into an easy-to-use interactive package: (1) innovative visualization software; (2) advanced database techniques; and (3) a rich set of mathematical and image processing tools. Visualization capabilities will include one-, two-, and three-dimensinal displays, along with animation, compression, warping and slicing functions. Analysis tools will include generic mathematical and statistical techniques along with the ability to use large scale models for interactive interpretation of large volume data sets. Our system will be implemented on Sun and EC UNIX workstations and on the Stardent Graphics Supercomputer. Our final deliverable will include complete documentation and a NASA/NSF-CDAW/SUNDIAL campaign demonstration.

Szuszczewicz, Edward P.↗
DOE OSTI2013

From data to analysis: linking NWChem and Avogadro with the syntax and semantics of Chemical Markup Language

Background Multidisciplinary integrated research requires the ability to couple the diverse sets of data obtained from a range of complex experiments and computer simulations. Integrating data requires semantically rich information. In this paper the generation of semantically rich data from the NWChem computational chemistry software is discussed within the Chemical Markup Language (CML) framework. Results The NWChem computational chemistry software has been modified and coupled to the FoX library to write CML compliant XML data files. The FoX library was expanded to represent the lexical input files used by the computational chemistry software. Conclusions The production of CML compliant XML files for the computational chemistry software NWChem can be relatively easily accomplished using the FoX library. A unified computational chemistry or CompChem convention and dictionary needs to be developed through a community-based effort. The long-term goal is to enable a researcher to do Google-style chemistry and physics searches.

Chemical Markup Language, FoX, NWChem, computation↗
DOE OSTI2012

Assessing health impacts in complex eco-epidemiological settings in the humid tropics: Modular baseline health surveys

The quantitative assessment of health impacts has been identified as a crucial feature for realising the full potential of health impact assessment (HIA). In settings where demographic and health data are notoriously scarce, but there is a broad range of ascertainable ecological, environmental, epidemiological and socioeconomic information, a diverse toolkit of data collection strategies becomes relevant for the mainly small-area impacts of interest. We present a modular, cross-sectional baseline health survey study design, which has been developed for HIA of industrial development projects in the humid tropics. The modular nature of our toolkit allows our methodology to be readily adapted to the prevailing eco-epidemiological characteristics of a given project setting. Central to our design is a broad set of key performance indicators, covering a multiplicity of health outcomes and determinants at different levels and scales. We present experience and key findings from our modular baseline health survey methodology employed in 14 selected sentinel sites within an iron ore mining project in the Republic of Guinea. We argue that our methodology is a generic example of rapid evidence assembly in difficult-to-reach localities, where improvement of the predictive validity of the assessment and establishment of a benchmark for longitudinal monitoring of project impacts and mitigation efforts is needed.

54 ENVIRONMENTAL SCIENCES↗