Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “csv”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

2022 Annual Technology Baseline (ATB) Cost and Performance Data for Electricity Generation Technologies

These data provide the 2022 update of the Electricity Annual Technology Baseline (ATB). Starting in 2015 NREL has presented the ATB, consisting of detailed cost and performance data, both current and projected, for electricity generation and storage technologies. The ATB products now include data (Excel workbook, Tableau workbooks, and structured summary csv files), as well as documentation and user engagement via a website, presentation, and webinar. Starting in 2021, the data are cloud optimized and provided in the OEDI data lake. The data for 2015 - 2020 are can be found on the NREL Data Search Page. The website documentation can be found on the ATB Website.

Array↗

LBNL Fault Detection and Diagnostics Datasets

These datasets can be used to evaluate and benchmark the performance accuracy of Fault Detection and Diagnostics (FDD) algorithms or tools. It contains operational data from simulation, laboratory experiments, and field measurements from real buildings for seven HVAC systems/equipment (rooftop unit, single-duct air handler unit, dual-duct air handler unit, variable air volume box, fan coil unit, chiller plant, and boiler plant). Each dataset includes a .pdf file to document key information necessary to understand the content and scope, multiple csv files containing all the time-series data for faults at different severity levels and one fault-free case, and a ttl file to visualize the data according to BRICK schema. The dataset was created by LBNL, PNNL, NREL, ORNL and Drexel University.

AC↗

BuildingsBench: A Large-Scale Dataset of 900K Buildings and Benchmark for Short-Term Load Forecasting

The BuildingsBench datasets consist of: - Buildings-900K: A large-scale dataset of 900K buildings for pretraining models on the task of short-term load forecasting (STLF). Buildings-900K is statistically representative of the entire U.S. building stock. - 7 real residential and commercial building datasets for benchmarking two downstream tasks evaluating generalization: zero-shot STLF and transfer learning for STLF. Buildings-900K can be used for pretraining models on day-ahead STLF for residential and commercial buildings. The specific gap it fills is the lack of large-scale and diverse time series datasets of sufficient size for studying pretraining and finetuning with scalable machine learning models. Buildings-900K consists of synthetically generated energy consumption time series. It is derived from the NREL End-Use Load Profiles (EULP) dataset (see link to this database in the links further below). However, the EULP was not originally developed for the purpose of STLF. Rather, it was developed to "...help electric utilities, grid operators, manufacturers, government entities, and research organizations make critical decisions about prioritizing research and development, utility resource and distribution system planning, and state and local energy planning and regulation." Similar to the EULP, Buildings-900K is a collection of Parquet files and it follows nearly the same Parquet dataset organization as the EULP. As it only contains a single energy consumption time series per building, it is much smaller (~110 GB). BuildingsBench also provides an evaluation benchmark that is a collection of various open source residential and commercial real building energy consumption datasets. The evaluation datasets, which are provided alongside Buildings-900K below, are collections of CSV files which contain annual energy consumption. The size of the evaluation datasets altogether is less than 1GB, and they are listed out below: 1. ElectricityLoadDiagrams20112014 2. Building Data Genome Project-2 3. Individual household electric power consumption (Sceaux) 4. Borealis 5. SMART 6. IDEAL 7. Low Carbon London A README file providing details about how the data is stored and describing the organization of the datasets can be found within each data lake version under BuildingsBench.

Array↗

2023 Annual Technology Baseline (ATB) Cost and Performance Data for Electricity Generation Technologies

These data provide the 2023 update of the Electricity Annual Technology Baseline (ATB). Starting in 2015 NREL has presented the ATB, consisting of detailed cost and performance data, both current and projected, for electricity generation and storage technologies. The ATB products now include data (Excel workbook, Tableau workbooks, and structured summary csv files), as well as documentation and user engagement via a website, presentation, and webinar. Starting in 2021, the data are cloud optimized and provided in the OEDI data lake. The data for 2015 - 2020 are can be found on the NREL Data Search Page. The website documentation can be found on the ATB Website.

Array↗

2024 Annual Technology Baseline (ATB) Cost and Performance Data for Electricity Generation Technologies

These data provide the 2024 update of the Electricity Annual Technology Baseline (ATB). Starting in 2015 NREL has presented the ATB, consisting of detailed cost and performance data, both current and projected, for electricity generation and storage technologies. The ATB products now include data (Excel workbook, Tableau workbooks, and structured summary csv files), as well as documentation and user engagement via a website, presentation, and webinar. Starting in 2021, the data are cloud optimized and provided in the OEDI data lake. The data for 2015 - 2020 are can be found on the NREL Data Search Page. The website documentation can be found on the ATB Website.

Array↗

Walker Branch Watershed: Temperature Response of Organic-Matter Decomposition in Headwater Stream

This data set reports the results of a field study investigating the effect of temperature on organic-matter decomposition in the West Fork of Walker Branch, a headwater stream on the Oak Ridge Reservation in east Tennessee. The goal of this project was to evaluate the effect of temperature on organic-matter decomposition in a stream ecosystem to better understand how carbon dynamics across terrestrial-aquatic carbon interfaces may change with warming. The West Fork is a spring-fed headwater stream and the spring inputs create a natural longitudinal temperature gradient. In winter, the upstream sites near the springs are warmer due to the spring input. In summer, the temperature gradient reverses and the upstream sites are cooler. One experiment examined the decomposition of senesced leaves from 3 tree species (red maple [Acer rubrum], tulip poplar [Liriodendron tulipifera], and white oak [Quercus alba]) along the temperature gradient in winter (December 2011 - March 2012). The second experiment examined the breakdown of cotton strips, a substrate of consistent quality (95% cellulose), along the temperature gradient with approximately month-long incubations over ~2 years (August 2011 to October 2013). This dataset contains six data files in comma separate (*.csv) format.

54 ENVIRONMENTAL SCIENCES↗

Soil Respiration and Microbial Biomass from Soil Incubations with 13C Labeled Additions

This data set provides the respiration and soil microbial biomass data from a series of short and long-term laboratory incubation experiments with 13C labeled substrates to examine how plant communities (forest vs. grassland), edaphic properties, and microbial communities influence C cycling and the long-term fate of C in soil systems. Paired forest and grassland soils from four locations were incubated with 13C labeled glucose in a short-term (144 hours) study while 13C labeled cellulose was added to soils used in the long-term (729 days) study. Soil respiration, microbial biomass, and soil 13C values were monitored throughout the study. Soils were collected from paired forest and grassland locations in the following states in the United States of America: Tennessee (TN), Ohio (OH), Iowa (IA), and Missouri (MO). This dataset contains one data file in comma-separate (*.csv) format.

54 ENVIRONMENTAL SCIENCES↗

SPRUCE S1 Bog Vegetation Survey and Peat Depth Data: 2009

This data set reports the results of a field survey of the S1 Bog to characterize the vegetation and to determine peat depth. The survey was conducted on September 21 and 22, 2009. This initial survey of vegetation and peat depth characteristics of the target bog was conducted to evaluate the logical locations for installing replicated experimental blocks for SPRUCE. The goal was to identify multiple locations of uniform aboveground vegetation and belowground peat depth for positioning experimental units within the bog. The experimental work was conducted in a Picea mariana [black spruce] - Sphagnum spp. bog forest in northern Minnesota, 40 km north of Grand Rapids, in the USDA Forest Service Marcell Experimental Forest (MEF). There is one (*.csv) data file containing the vegetation survey data and observations, and the peat depth measurements. Also included are figures of the spatial distributions of vegetation survey and peat depth results (Vegetation_survey_2009_figures.pdf). Photos of the surveyed plots are included in several *.zip files.photos.

54 ENVIRONMENTAL SCIENCES↗

SPRUCE Peat Mercury, Methylmercury and Sulfur Concentrations from Experimental Plot Cores, 2012

This data set reports the results of analyses of peat core samples for total mercury and methylmercury, as well as carbon, nitrogen and sulfur. Samples were collected as part of a larger sampling campaign from the SPRUCE experimental study plots located in the S1-Bog. From 2012-08-13 to 2012-10-15, a team of SPRUCE investigators and collaborators collected core samples of peat in the SPRUCE experimental plots. The goal was to characterize the biological, physical, and chemical characteristics of peat, and how those characteristics changed throughout the depth profile of the bog, prior to the initialization of the SPRUCE experimental warming and CO2 treatments. Cores were collected from 16 experimental plots; peat cores were collected from the hummock and hollow surfaces to depths of 300 cm in 10 cm, 25, and 50 cm increments (Pierce et al, 2022). Three replicate cores were collected from both hummock and hollow locations in each plot. The coring locations within each plot were mapped. This dataset contains one file in comma separated (*.csv) format. User Note: The coring descriptions and some reported data are the same as in the related data set, Iverson et al. (2014). Samples for mercury and carbon, nitrogen and sulfur analyses by the University of Minnesota were collected as part of this sampling campaign.

54 ENVIRONMENTAL SCIENCES↗

SPRUCE Deep Peat Heating (DPH) Peat Water Content and Temperature for Experimental Plot Cores, June 2014 through June 2015

This data set provides the peat water content and peat temperature at time of sampling for peat cores collected before and during the SPRUCE Deep Peat Heating (DPH) study. Cores were collected during three sampling events: 03 June 2014, 09 September 2014, and 16 June 2015. Two cores were extracted from hollow locations in each of the 10 experimental plots (4, 6, 8, 10, 11, 13, 16, 17, 19, and 20). The five ambient plots (5, 7, 14, and 21) were not included in the DPH sampling efforts. See Table 1. Cores were partitioned into samples at 11 depth increments: 0-10, 10-20, 20-30, 30-40, 40-50, 50-75, 75-100, 100-125, 125-150, 150-175, and 175-200 cm below surface of the hollow. The DPH study was undertaken to evaluate possible rapid responses of deep peat carbon (C) stocks, microbial communities, and biogeochemical cycling processes to unprecedented temperatures. In June 2014, deep soil heating was initiated at 5 warming levels (+0, +2.25, +4.5, +6.75 and +9 °C) with 2 replicate plots per level. Warming levels at depth were achieved over a 25 (+ 2.25 °C) to 60-day (+9 °C) period depending on the target treatment temperatures. Data file – There is one comma-separated (*.csv) file with peat core water content and temperature at time of sampling, plus the amount of dried archived material retained after analyses. User Note: In this third release, four variables related to extrapolations of peat core temperature were removed. See the Data Set History table for additional details. 25-150, 150-175, and 175-200 cm below surface of the hollow.

54 ENVIRONMENTAL SCIENCES↗

SPRUCE Whole Ecosystems Warming (WEW) Environmental Data Beginning August 2015

This data set provides a record of the half-hourly averages of automated environmental data collected for 12 SPRUCE plots (4, 6, 7, 8, 10, 11, 13, 16, 17, 19, 20, and 21) beginning during deep peat heating (DPH) and continuing throughout the whole ecosystem warming (WEW) manipulations for the SPRUCE Project (Hanson et al. 2017). In August 2015, WEW was initiated at 5 warming levels (+0, +2.25 +4.5, +6.75 and +9 °C) with 2 plots per warming level. DPH measurements were underway before the initiation of WEW heating treatments and both are expected to operate through 2025. This current version includes data from 2014 through 2024. This data set includes 15 data files provided in comma separated (*.csv) format: 12 individual data files for each of the monitored SPRUCE plots, two data files corresponding to figures from Hanson et al. (2017) that support key analyses of the performance of the WEW systems, and one data file containing snow and ice depths.

54 ENVIRONMENTAL SCIENCES↗

SPRUCE Geochemical Changes in Porewater from Northern Peatlands at Multiple Depths in Field Samples and over Time in Peat Incubations

This data set reports the results of the analysis of dissolved organic matter (DOM) in porewater samples collected from multiple depths in northern peatlands and during laboratory peat incubations to characterize the gain or loss of specific chemical moieties (e.g. CH2, H2, CH2O, etc.) that may be attributed to microbial degradation. During incubations, CO2 and CH4 concentrations and stable isotopes were determined by isotope ratio mass spectrometry. Porewater samples were collected from the S1 Bog (April 2015), a Sphagnum-dominated site at northern Minnesota peatland in the Marcell Experimental Forest, using permanently-installed piezometers, which provided profiles from the watertable surface to 200 cm depth. This is the site of the SPRUCE experiment. Porewater and peat samples were collected in July 2015 from an additional Sphagnum-dominated site within Stordalen Mire, Sweden, a palsa mire underlain by thawing permafrost, using perforated stainless steel tubes that were inserted into the peat to the desired depths (13 cm and 32 cm). Porewater samples were analyzed by Fourier transform ion cyclotron resonance mass spectrometry (FTICRMS) followed by Kendrick mass transform analysis. The Kendrick mass (i.e., Kendrick 1963) transform analysis identifies the gain or loss of specific chemical moieties (e.g. CH2, H2, CH2O, etc.) via mass differences, and was used to infer potential microbial degradation pathways by which compounds were consumed in these samples. Increasing depth would suggest more degradation of DOM with the greater length of time for peat accumulation and similarly, with increasing length of peat incubation. During incubations, CO2 and CH4 concentrations and stable isotopes were determined by isotope ratio mass spectrometry. There are four files provided in comma separated value (*.csv) format.

54 ENVIRONMENTAL SCIENCES↗

SIF and Vegetation Indices in the US Midwestern Agroecosystems, 2016-2021, Carbon Monitoring System (CMS)

This dataset contains half-hourly ground solar-induced chlorophyll fluorescence (SIF) and vegetation indices including NDVI, EVI, Red edge chlorophyll index, green chlorophyll index, and photochemical reflectance index at seven crop sites in Nebraska and Illinois for the period 2016-2021. Four sites were located at Eddy Covariance (EC) tower sites (sites US-Ne2, US-Ne3, US-UiB, and US-UiC), and three sites were located on private farms (sites Reifsteck, Rund, and Reinhart). The sites were either miscanthus, corn-soybean rotation or corn-corn-soybean rotation. The spectral data for SIF retrieval and hyperspectral reflectance for vegetation index calculation were collected by the FluoSpec2 system, installed near planting, and uninstalled after harvest to collect whole growing-season data. Raw nadir SIF at 760 nm from different algorithms (sFLD, 3FLD, iFLD, SFM) are included. SFM_nonlinear and SFM_linear represent the Spectral fitting method (SFM) with the assumption that fluorescence and reflectance change with wavelength non-linearly and linearly, respectively. Additional data include two SIF correction factors including calibration coefficient adjustment factor (f_cal_corr_QEPRO) and upscaling nadir SIF to eddy covariance footprint factor (ratio_EC footprint, SIF pixel), and measured FPAR from quantum sensors and Rededge NDVI calculated FPAR. The data are provided in comma-separated values (CSV) format.

Ameriflux↗

The fortedata R package: open-science datasets from a manipulative experiment testing forest resilience

The fortedata R package is an open data notebook from the Forest Resilience Threshold Experiment (FoRTE) – a modeling and manipulative field experiment that tests the effects of disturbance severity and disturbance type on carbon cycling dynamics in a temperate forest. Package data consist of measurements of carbon pools and fluxes and ancillary measurements to help analyze and interpret carbon cycling over time. Currently the package includes data and metadata from the first three FoRTE field seasons, serves as a central, updatable resource for the FoRTE project team, and is intended as a resource for external users over the course of the experiment and in perpetuity. Further, it supports all associated FoRTE publications, analyses, and modeling efforts. This increases efficiency, consistency, compatibility, and productivity while minimizing duplicated effort and error propagation that can arise as a function of a large, distributed and collaborative effort. More broadly, fortedata represents an innovative, collaborative way of approaching science that unites and expedites the delivery of complementary datasets to the broader scientific community, increasing transparency and reproducibility of taxpayer-funded science. The fortedata package is available via GitHub: https://github.com/FoRTExperiment/fortedata (last access: 19 February 2021), and detailed documentation on the access, used, and applications of fortedata are available at https://fortexperiment.github.io/fortedata/ (last access: 19 February 2021). The first public release, version 1.0.1 is also archived at https://doi.org/10.5281/zenodo.4399601 (Atkins et al., 2020b). All data products are also available outside of the package as .csv files: https://doi.org/10.6084/m9.figshare.13499148.v1 (Atkins et al., 2020c).

97 MATHEMATICS AND COMPUTING↗

Epi Archive: Automated Synthesis of Global Notifiable Disease Data

ObjectiveLANL has built software that automatically collects global notifiable disease data, synthesizes the data, and makes it available to humans and computers within the Biosurveillance Ecosystem (BSVE) as a novel data stream. These data have many applications including improving the prediction and early warning of disease events.IntroductionMost countries do not report national notifiable disease data in a machine-readable format. Data are often in the form of a file that contains text, tables and graphs summarizing weekly or monthly disease counts. This presents a problem when information is needed for more data intensive approaches to epidemiology, biosurveillance and public health.While most nations likely store incident data in a machine-readable format, governments are often hesitant to share data openly for a variety of reasons that include technical, political, economic, and motivational issues1.A survey conducted by LANL of notifiable disease data reporting in over fifty countries identified only a few websites that report data in a machine-readable format. The majority (>70%) produce reports as PDF files on a regular basis. The bulk of the PDF reports present data in a structured tabular format, while some report in natural language.The structure and format of PDF reports change often; this adds to the complexity of identifying and parsing the desired data. Not all websites publish in English, and it is common to find typos and clerical errors.LANL has developed a tool, Epi Archive, to collect global notifiable disease data automatically and continuously and make it uniform and readily accessible.MethodsWe conducted a survey of the national notifiable disease reporting systems notating how the data are reported and in what formats. We determined the minimal metadata that is required to contextualize incident counts properly, as well as optional metadata that is commonly found.The development of software to regularly ingest notifiable disease data and make it available involves three or four main steps: scraping, detecting, parsing and persisting.Scraping: we examine website design and determine reporting mechanisms for each country/website, as well as what varies across the reporting mechanisms. We then designed and wrote code to automate the downloading of the data for each country. We store all artifacts presented as files (PDF, XLSX, etc.) in their original form, along with appropriate metadata for parsing and data provenance.Detecting: This step is required when parsing structured non-machine-readable data such as tabular data in PDF files. We combined the Nurminen methodology of PDF table detection with in-house heuristics to find the desired data within PDF reports2.Parsing: We determined what to extract from each dataset and parsed these data into uniform data structures, correctly accommodating the variations in metadata (e.g., time interval definitions) and the various human languages.Persisting: We store the data in the Epi Archive database and make it available on the internet and through the BSVE. The data is persisted into a structured and normalized SQL database.ResultsThe Epi Archive tool currently contains national and/or subnational notifiable disease data from twenty nations. When a user accesses the Epi Archive site, they are prompted with four fields: country, subregion, disease of interest, and date duration. Upon form submission, a time series is generated from the users’ specifications. The generated graph can then be downloaded into a CSV file if a user is interested in performing personal analysis. Additionally, the data from Epi Archive can be reached through a REST API (Representational State Transfer Application Programming Interface).ConclusionsLANL, as part of a currently funded DTRA effort, is automatically and continually collecting global notifiable disease data. While 20 nations are in production, more are being brought online in the near future. These data are already being utilized and will have many applications including improving the prediction and early warning of disease events.References[1] van Panhuis WG, Paul P, Emerson C, et al. A systematic review of barriers to data sharing in public health. BMC Public Health. 2014. 14:1144. doi:10.1186/1471-2458-14-1144[2] Nurminen, Anssi. "Algorithmic extraction of data in tables in PDF documents." (2013).

Khalsa, Hari S.↗

Epi Archive: automated data collection of notifiable disease data

LANL has built a software program that automatically collects global notifiable disease data—particularly data stored in files—and makes it available and shareable within the Biosurveillance Ecosystem(BSVE) as a new data source. This will improve the prediction and early warning of disease events and other applications. Most countries do not report national notifiable disease data in a machine-readable format. Data are often in the form of a file that contains text, tables and graphs summarizing weekly or monthly disease counts. This presents a problem when information is needed for more data intensive approaches to epidemiology, biosurveillance and public health as exemplified by the Biosurveillance Ecosystem(BSVE).While most nations do likely store their data in a machine-readable format, the governments are often hesitant to share data openly for a variety of reasons that include technical, political, economic, and motivational issues. For example, an attempt by LANL to obtain a weekly version of openly available monthly data, reported by the Australian government, resulted in an onerous bureaucratic reply. The obstacles to obtaining data included: paperwork to request data from each of the Australian states and territories, a long delay to obtain data (up to 3 months) and extensive limitations on the data’s use that prohibit collaboration and sharing. This type of experience when attempting to contact public health departments or ministries of health for data is not uncommon. A survey conducted by LANL of notifiable disease data reporting in 52 countries identified only 10 as being machine-readable and 42 being reported in pdf files on a regular basis. Within the 42 nations that report in pdf files, 32 report in a structured, tabular format and10 in a non-structured way. As a result, LANL has developed a tool-Epi Archive (formerly known as EPIC)-to automatically and continuously collect global notifiable disease data and make it readily accessible. We conducted a survey of the national notifiable disease reporting systems notating how the data is reported in two important dimensions: date standards and case definitions. The development of software to regularly ingests notifiable disease data frand makes this data available involved four main steps scraping, extracting, parsing and persisting. For scraping: we would examine website designs and determine reporting mechanisms for each country/website as well as what varies across the reporting mechanisms. We then designed and wrote code to automate the downloading of report pdf files, for each country. We stored report pdfs along with appropriate metadata for extracting and parsing. For extracting: we developed software that can extract notifiable disease data presented in tabular form from a pdf file. We combined the methodology of figure placement detection with the in-house developed table extraction and annotation heuristics. For parsing: we determined what to extract from each pdf dataset from the survey conducted. We then parsed the extracted data into uniform data structures correctly accommodating the dimensions surveyed and the various human languages. This task involved ingesting notifiable disease data in many disparate formats extracted from pdf files and coalescing the data into a standardized format. For persisting: We then store the data in the Epi Archive PostgreSQL database and make it available through the BSVE. The EpiArchive tool currently contains subnational notifiable disease data from 10 nations. When a user accesses the EpiArchive site, they are prompted with four fields: country, region, disease, and date duration. These fields allow the user to specify the location(down to the state level), the disease of interest, and the duration of interest. Upon form submission, a time series is generated from the users’ specifications. The generated time series can then be downloaded into a csv file if a user is interested in performing personal analysis. Additionally, the data from EpiArchive can be reached through an API. LANL as part of a currently funded DTRA effort so that it will automatically and continuously collect global notifiable disease data—particularly data stored in pdf files—and make it available and shareable within the Biosurveillance Ecosystem (BSVE) as a new data source. This will provide data to analytics and users that will improve the prediction and early warning of disease events and other applications.

59 BASIC BIOLOGICAL SCIENCES↗

Thermochemical Data for Furan-based Monomer Candidates for Frontal Ring-Opening Metathesis Polymerization (FROMP)

This dataset includes 471 furan-based monomer candidates for frontal ring-opening metathesis polymerization (FROMP) and relevant thermochemistry as calculated with density functional theory (DFT). The monomer candidates were combinatorically enumerated using Diels-Alder reactions of furan derivatives as dienes and four types of dienophiles (alkenes, alkynes, allenes, and benzynes). Common substituents were enumerated for the dienophile classes, and methyl substitution on the diene was explored. We used the SMILES arbitrary target specification (SMARTS) language to produce monomers and ring-opened structures from diene and dienophile precursor SMILES, and we studied the ring-opening reaction using a homodesmotic equation with ethene. RDKit conformers were initially generated from SMILES, then optimized with GFN2-xTB. The two conformers lowest in energy were then optimized with DFT using the wb97x-D3 functional, def2-TZVP basis set, and def2/J auxiliary basis set. Gibbs free energy corrections were obtained through frequency calculations. Structures with imaginary frequencies below -50 cm^{-1} were excluded from this work, and smaller imaginary modes were flipped to be positive for free energy calculations. Modes below 50 cm^{-1} were treated with the modified rigid rotor approximation, and all thermochemical values were calculated at T=200C. The CSV file contains the monomer SMILES, the free energy of reaction for Diels-Alder addition (G_DA_200), and the enthalpy of the ring-opening reaction (H_RO_200). All energies are given in kcal/mol. An interactive HTML is also included to visualize the monomers in this dataset.

Chua, Lauren↗