Enabling AI in Reactive Transport Modeling with Community Data Mining
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Metabolic network modeling of microbial communities provides an in-depth understanding of community-wide metabolic and regulatory processes. Compared to single organism analyses, community metabolic network modeling is more complex because it needs to account for interspecies interactions. To date, most approaches focus on reconstruction of high-quality individual networks so that, when combined, they can predict community behaviors as a result of interspecies interactions. However, this conventional method becomes ineffective for communities whose members are not well characterized and cannot be experimentally interrogated in isolation. Here, we tested a new approach that uses community-level data as a critical input for the network reconstruction process. This method focuses on directly predicting interspecies metabolic interactions in a community, when axenic information is insufficient. We validated our method through the case study of a bacterial photoautotroph-heterotroph consortium that was used to provide data needed for a community-level metabolic network reconstruction. Resulting simulations provided experimentally validated predictions of how a photoautotrophic cyanobacterium supports the growth of an obligate heterotrophic species by providing organic carbon and nitrogen sources.
Data sharing benefits the researcher, the scientific community, and the public by allowing the impact of data to be generalized beyond one project and by making science more transparent. However, many scientific communities have not developed protocols or standards for publishing, citing, and versioning datasets. One community that lags in data management is that of low-temperature geochemistry (LTG). This paper resulted from an initiative from 2018 through 2020 to convene LTG and data scientists in the U.S. to strategize future management of LTG data. Through webinars, a workshop, a preprint, a townhall, and a community survey, the group of U.S. scientists discussed the landscape of data management for LTG – the data-scape. Currently this data-scape includes a “street bazaar” of data repositories. This was deemed appropriate in the same way that LTG scientists publish articles in many journals. The variety of data repositories and journals reflect that LTG scientists target many different scientific questions, produce data with extremely different structures and volumes, and utilize copious and complex metadata. Nonetheless, the group agreed that publication of LTG science must be accompanied by sharing of data in publicly accessible repositories, and, for sample-based data, registration of samples with globally unique persistent identifiers. LTG scientists should use certified data repositories that are either highly structured databases designed for specialized types of data, or unstructured generalized data systems. Recognizing the need for tools to enable search and cross-referencing across the proliferating data repositories, the group proposed that the overall data informatics paradigm in LTG should shift from “build data repository, data will come” to “publish data online, cybertools will find”. Funding agencies could also provide portals for LTG scientists to register funded projects and datasets, and forge approaches that cross national boundaries. Finally, the needed transformation of the LTG data culture requires emphasis in student education on science and management of data.
The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented a series of new features designed to connect people to data. These features, which are based on feedback from the GDR user community and surveys of the greater geothermal research community, are designed to improve data quality and empower members of all communities to better engage with geothermal data resources by providing universal access to data and by improving the connections between data providers, subject matter experts, and the communities of people using GDR data. This paper will explore some of the recent enhancements made to the GDR to improve data discoverability, reduce submission time, and result in better quality data submissions. These improvements include the ability for users to save a list of their favorite datasets, search for insight into geothermal datasets or data availability, or sign up to receive notifications of future updates to specific datasets. These improvements aim to enhance the overall user experience of the GDR while further connecting communities to the data they need to inform decisions, advance geothermal research, and develop innovative solutions to local energy problems.
The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented a series of new features designed to connect people to data. These features, which are based on feedback from the GDR user community and surveys of the greater geothermal research community, are designed to improve data quality and empower members of all communities to better engage with geothermal data resources by providing universal access to data and by improving the connections between data providers, subject matter experts, and the communities of people using GDR data. This paper will explore some of the recent enhancements made to the GDR to improve data discoverability, reduce submission time, and result in better quality data submissions. These improvements include the ability for users to save a list of their favorite datasets, search for insight into geothermal datasets or data availability, or sign up to receive notifications of future updates to specific datasets. These improvements aim to enhance the overall user experience of the GDR while further connecting communities to the data they need to inform decisions, advance geothermal research, and develop innovative solutions to local energy problems.
This data release includes experimental data of potentiometric titration for iron oxides. The data in the provided .csv files is not our own experimental data but have been compiled from the multiple literature sources. The master database is L-SCIE (LLNL Surface Complexation/Ion Exchange) database, and the provided .csv files are extracted data from L-SCIE. The .csv files were obtained by using the Lawrence Livermore National Laboratory Surface Complexation Database Converter (SCDC) code written in the R programming language (free licensing available at https://ipo.llnl.gov/technologies/software/llnl-surface-complexation-database-converter-scdc).The released data was used for developing a comprehensive community data-driven surface complexation modeling (SCM) framework for simulating potentiometric titration of mineral surfaces. Compiled community data for ferrihydrite, goethite, hematite, and magnetite are fit to produce representative protolysis constants that can reproduce potentiometric titration data collected from multiple literature sources.
This data package is associated with the publication “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning’’ submitted to the Journal of Geophysical Research: Machine Learning and Computation (Scheibe et al. 2024). River sediment respiration observations are expensive and labor intensive to obtain and there is no physical model for predicting this quantity. The Worldwide Hydrobiogeochemisty Observation Network for Dynamic River Systems (WHONDRS) observational data set (Goldman et al.; 2020) is used to train machine learning (ML) models to predict respiration rates at unsampled sites. This repository archives training data, ML models, predictions, and model evaluation results for the purposes of reproducibility of the results in the associated manuscript and community reuse of the ML models trained in this project. One of the key challenges in this work was to find an optimum configuration for machine learning models to work with this feature-rich (i.e. 100+ possible input variables) data set. Here, we used a two-tiered approach to managing the analysis of this complex data set: 1) a stacked ensemble of ML models that can automatically optimize hyperparameters to accelerate the process of model selection and tuning and 2) feature permutation importance to iteratively select the most important features (i.e. inputs) to the ML models. The major elements of this ML workflow are modular, portable, open, and cloud-based, thus making this implementation a potential template for other applications. This data package is associated with the GitHub repository found at Please see the file level metadata (flmd; “sl-archive-whondrs_flmd.csv”) for a list of all files contained in this data package and descriptions for each. Please see the data dictionary (dd; “sl-archive-whondrs_dd.csv”) for a list of all column headers contained within comma separated value (csv) files in this data package and descriptions for each. The GitHub repository is organized into five top-level directories: (1) “input_data” holds the training data for the ML models; (2) “ml_models” holds machine learning models trained on the data in “input_data”; (3) “scripts” contains data preprocessing and postprocessing scripts and intermediate results specific to this data set that bookend the ML workflow; (4) “examples” contains the visualization of the results in this repository including plotting scripts for the manuscript (e.g., model evaluation, FPI results) and scripts for running predictions with the ML models (i.e., reusing the trained ML models); (5) “output_data” holds the overall results of the ML model on that branch. Each trained ML model resides on its own branch in the repository; this means that inputs and outputs can be different branch-to-branch. Furthermore, depending on the number of features used to train the ML models, the preprocessing and postprocessing scripts, and their intermediate results, can also be different branch-to-branch. The “main-*” branches are meant to be starting points (i.e. trunks) for each model branch (i.e. sprouts). Please see the Branch Navigation section in the top-level README.md in the GitHub repository for more details. There is also one hidden directory “.github/workflows”. This hidden directory contains information for how to run the ML workflow as an end-to-end automated GitHub Action but it is not needed for reusing the ML models archived here. Please the top-level README.md in the GitHub repository for more details on the automation.
This dataset provides vegetation survey data collected across vegetation community plots within the 10 chambered treatment plots (6, 19, 11, 20, 4, 13, 8, 16, 10, 17) and 3 unchambered ambient plots (7, 14, 21) from the Spruce and Peatland Responses Under Changing Environments (SPRUCE) experimental site within the Marcell Experimental Forest in northern Minnesota, USA. Collections occurred from 2014-2020 (2014-04-27 to 2020-08-06). The vegetation community plots, three in each experimental plot, were established in 2014 and designated to be undisturbed and reserved for long-term sampling. Vegetation survey data were collected at the peak of each growing season by placing a 1x2 meter rectangular grid over each community plot. The 1x2 meter grid is subset into 50 20x20 cm grid cells. Observations are of the presence of each species in each grid cell for each subplot. That’s a total of 39 subplots across the 13 experimental plots. From 2018 to 2020, shrub stem density was counted and percent shrub cover was estimated in five selected grid cells in each vegetation community plot. Also included are photographs of each community plot taken at the time of data collection for each measurement year. These data provide insight into the responses of plant community composition to warming and increased carbon dioxide. These data were use in analyses in McPartland et al (2018; 2020). This dataset contains three data files in comma-separate values (*.csv) format and seven compressed (*.zip) files containing 877 JPEG (*.jpg) files. Additional metadata are provided: a data dictionary and a file-level metadata file in comma-separate values (.csv) format and a user guide in PDF (*.pdf) format.
River sediment microbial respiration is a key indicator of ecosystem functioning and the biogeochemical fluxes across this critical zone link surface and subsurface waters. As such, there is tremendous interest in measuring and mapping these respiration rates. Respiration observations are expensive and labor intensive; there is limited data available to the community. An open science, collaborative initiative is collecting samples for respiration rate analysis and multi-scale metadata; this evolving data set is being used for making machine learning (ML) predictions at unsampled sites to help inform continued community engagement. However, it is a challenge to find an optimum configuration for ML models to work with this feature-rich (i.e., 100+ possible input variables) data set. Here, we present results from a two-tiered approach to managing the analysis of this complex data set: (a) a stacked ensemble of models that automatically optimizes hyperparameters and manages the training of many models and (b) feature permutation importance to detect the most important features in the models. The major elements of this workflow are modular, portable, open, and cloud-based thus making this implementation a potential template for other applications. The models developed here predict that sediment organic matter chemistry is one of the most important features for predicting sediment respiration rate. Other larger-scale, important features fall into the categories of climatic, ecological, geological, and fluvial settings. Leveraging these larger-scale features to generate data-driven estimates of river sediment respiration rates reveals spatially consistent but heterogeneous patterns across the river network of the Columbia River Basin.
NASA Goddard Earth Sciences Data and Information Services Center (GESDISC) is one of the 12 NASA Science Mission Directorate Data Centers. The main goal of GESDISC is to provide earth science data, information, and services to the earth science data community. Consequently, data discovery is at the center of our mission and our search engine is the primary tool for our users to interact, find, and access our data. Existing search approaches are largely focused on hard-matching of keywords in the search query with dataset metadata. Here we propose to expand the search by introducing a complementary natural language processing (NLP) search. At the heart of our proposed NLP search, we trained a joint embedding using scientific text corpus and a curated set of dataset metadata. The embedding learns the association between words in our dataset metadata and those of the scientific text corpus. This enables us to go beyond simple hard-matching of a query and data set metadata and have a notion of “similarity” between the search query and the datasets. We further integrated our NLP search into the Elastic Search (ES) framework leveraging similarity search capabilities offered through the “dense_vector” field type. Our preliminary evaluations show that our proposed NLP search has the potential to be utilized to complement the existing search engine and serve as a base for a dataset recommendation system.
This dataset contains supplementary information for a manuscript describing the ESS-DIVE (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) data repository's community data and metadata reporting formats. The purpose of creating the ESS-DIVE reporting formats was to provide guidelines for formatting some of the diverse data types that can be found in the ESS-DIVE repository. The 6 teams of community partners who developed the reporting formats included scientists and engineers from across the Department of Energy National Lab network. Additionally, during the development process, 247 individuals representing 128 institutions provided input on the formats. The primary files in this dataset are 10 data and metadata crosswalk for ESS-DIVE’s reporting formats (all files ending in _crosswalk.csv). The crosswalks compare elements used in each of the reporting formats to other related standards and data resources (e.g., repositories, datasets, data systems). This dataset also contains additional files recommended by ESS-DIVE’s file-level metadata reporting format. Each data file has an associated dictionary (files ending in _dd.csv) which provide a brief description of each standard or data resource consulted in the data reporting format development process. The flmd.csv file describes each file contained within the dataset.
Communities worldwide have used vaccines and facemasks to mitigate the COVID-19 pandemic. When an individual opts to vaccinate or wear a mask, they may lower their own risk of becoming infected as well as the risk that they pose to others while infected. The first benefit–reducing susceptibility–has been established across multiple studies, while the second–reducing infectivity–is less well understood. Using a new statistical method, we estimate the efficacy of vaccines and facemasks at reducing both types of risks from contact tracing data collected in an urban setting. We find that vaccination reduced the risk of onward transmission by 40.7% [95% CI 25.8–53.2%] during the Delta wave and 31.0% [95% CI 19.4–40.9%] during the Omicron wave and that mask wearing reduced the risk of infection by 64.2% [95% CI 5.8–77.3%] during the Omicron wave. By harnessing commonly-collected contact tracing data, the approach can broadly provide timely and actionable estimates of intervention efficacy against a rapidly evolving pathogen.
Data assimilation is the process by which observations are combined with short-range NWP model output to produce an analysis of the state of the atmosphere at a specified time. Since its inception in the late 1990s, the multiagency Weather Research and Forecasting (WRF) model effort has had a strong data assimilation component, dedicating two working groups to the subject. This article documents the history of the WRF data assimilation effort, and discusses the challenges associated with balancing academic, research, and operational data assimilation requirements in the context of the WRF effort to date. The WRF Model's Community Variational/Ensemble Data Assimilation System (WRFDA) has evolved over the past 10 years, and has resulted in over 30 refereed publications to date, as well as implementation in a wide range of real-time and operational NWP systems.
The Alpine Treeline Warming Experiment (ATWE) was a common garden-climate manipulation experiment set up across an elevation gradient in Niwot Ridge, in the Front Range of the Colorado Rocky Mountains, USA. The project sought to learn more about the effects of climate change on alpine and subalpine ecosystems, namely tree species ranges and alpine plant communities. Plots were experimentally manipulated using infrared heaters set up to warm plots to temperatures comparable to those projected for the year 2100. Other treatments include watering, and a combination of watering and heating. Three sites were set up at different elevations to study three tree species, with the highest-elevation “Alpine” site ( ~3540 m) containing twenty additional plots to study alpine plant communities. Data in this package originate from these unseeded alpine plots. To evaluate soil nitrogen availability given site treatments, resin bags were deployed, removed, and extracted annually to produce ammonium and nitrate/nitrite readings.--------------------------------------------------Data files within this archive are in comma-separated-values (.csv) and Microsoft Excel (.xlsx) formats. .csvs can be read and opened by Microsoft Excel, R, or any other simple text-editing software, and .xlsx files can be opened using Microsoft Excel. Geospatial data associated with this package are in .kml and ESRI shapefile (.shp) formats. .kml files can be read using Google Earth or Google Maps, and shapefiles can be read with any software compatible with the file type, such as QGIS or ESRI’s ArcMap suite.Data files in this package - excluding “Winkler_2019_ALPO_inorganic_N_allyears.csv” - are provided in both Microsoft Excel and .csv formats, for added accessibility and flexibility in workflows. File contents are identical.
Here, we present a new data set of annual historical (1750–2014) anthropogenic chemically reactive gases (CO, CH 4 , NH 3 , NO x , SO 2 , NMVOCs), carbonaceous aerosols (black carbon – BC, and organic carbon – OC), and CO 2 developed with the Community Emissions Data System (CEDS). We improve upon existing inventories with a more consistent and reproducible methodology applied to all emission species, updated emission factors, and recent estimates through 2014. The data system relies on existing energy consumption data sets and regional and country-specific inventories to produce trends over recent decades. All emission species are consistently estimated using the same activity data over all time periods. Emissions are provided on an annual basis at the level of country and sector and gridded with monthly seasonality. These estimates are comparable to, but generally slightly higher than, existing global inventories. Emissions over the most recent years are more uncertain, particularly in low- and middle-income regions where country-specific emission inventories are less available. Future work will involve refining and updating these emission estimates, estimating emissions' uncertainty, and publication of the system as open-source software.
This paper demonstrates the use of two Bayesian statistical models to analyze single-event sonic boom exposure and human annoyance data from community response surveys. Each model is fit to data from a NASA pilot study.Unlike many community noise surveys, this study used a panel sample to collect multiple observations per participant instead of a single observation. Thus, a multilevel (also known as hierarchical or mixed-effects) model is used to account for the within-subject correlation in the panel sample data. This paper describes a multilevel logistic regression model and a multilevel ordinal regression model. The paper also proposes a method for calculating a summary dose-response curve from the multilevel models that represents the population. The two models’ summary dose-response curves are visually similar. However, their estimates differ when calculating the noise dose at a fixed percent highly annoyed.
Global anthropogenic emission inventories remain vital for understanding the sources of atmospheric pollution and the associated impacts on the environment, human health, and society. Rapid changes in today's society require that these inventories provide contemporary estimates of multiple atmospheric pollutants with both source sector and fuel type information to understand and effectively mitigate future impacts. To fill this need, we have updated the open-source Community Emissions Data System (CEDS) (Hoesly et al., 2019) to develop a new global emission inventory, CEDSGBD-MAPS. This inventory includes emissions of seven key atmospheric pollutants (NOx; CO; SO2; NH3; non-methane volatile organic compounds, NMVOCs; black carbon, BC; organic carbon, OC) over the time period from 1970–2017 and reports annual country-total emissions as a function of 11 anthropogenic sectors (agriculture; energy generation; industrial processes; on-road and non-road transportation; separate residential, commercial, and other sectors (RCO); waste; solvent use; and international shipping) and four fuel categories (total coal, solid biofuel, the sum of liquid-fuel and natural-gas combustion, and remaining process-level emissions). The CEDSGBD-MAPS inventory additionally includes monthly global gridded (0.5°?×?0.5°) emission fluxes for each compound, sector, and fuel type to facilitate their use in earth system models. CEDSGBD-MAPS utilizes updated activity data, updates to the core CEDS default scaling procedure, and modifications to the final procedures for emissions gridding and aggregation. Relative to the previous CEDS inventory (Hoesly et al., 2018), these updates extend the emission estimates from 2014 to 2017 and improve the overall agreement between CEDS and two widely used global bottom-up emission inventories. The CEDSGBD-MAPS inventory provides the most contemporary global emission estimates to date for these key atmospheric pollutants and is the first to provide global estimates for these species as a function of multiple fuel types and source sectors. Dominant sources of global NOx and SO2 emissions in 2017 include the combustion of oil, gas, and coal in the energy and industry sectors as well as on-road transportation and international shipping for NOx. Dominant sources of global CO emissions in 2017 include on-road transportation and residential biofuel combustion. Dominant global sources of carbonaceous aerosol in 2017 include residential biofuel combustion, on-road transportation (BC only), and emissions from the waste sector. Global emissions of NOx, SO2, CO, BC, and OC all peak in 2012 or earlier, with more recent emission reductions driven by large changes in emissions from China, North America, and Europe. In contrast, global emissions of NH3 and NMVOCs continuously increase between 1970 and 2017, with agriculture as a major source of global NH3 emissions and solvent use, energy, residential, and the on-road transport sectors as major sources of global NMVOCs. Due to similar development methods and underlying datasets, the CEDSGBD-MAPS emissions are expected to have consistent sources of uncertainty as other bottom-up inventories. The CEDSGBD-MAPS source code is publicly available online through GitHub: https://github.com/emcduffie/CEDS/tree/CEDS_GBD-MAPS (last access: 1 December 2020). The CEDSGBD-MAPS emission inventory dataset (both annual country-total and monthly global gridded files) is publicly available under https://doi.org/10.5281/zenodo.3754964 (McDuffie et al., 2020c).