Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data processing and standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

36 MATERIALS SCIENCE↗
DOE OSTI2025

sourcePy

Pollutant source identification techniques (of which there are many variations) are either locked behind researchers writing their own code for each use case or GUI platforms that are easy to use but inflexible and opaque. The Python package sourcePy brings together many of the pollutant source identification algorithms, giving the user full control out of the box. It aims to create a platform for source identification experiments where the full analysis from beginning to end can be done in Python, with a level of specificity in design that isn't available in the GUI options. sourcePy provides users with a few key features: -A Python interface with HYSPLIT, which can be used to generate trajectories and concentration plumes -Several Python classes which standardize the preparation and processing of data related to source identification experiments -Example scripts and notebooks that allow even new python users to get started with their own experiments quickly -Visualization methods

Arseneau, Isaac [Oak Ridge National Laboratory (OR↗
DOE OSTI2022

PLA Renewable Bio Polymer Based Solid-State Gamma Radiation Detector-Dosimeter for Biomedical and Nuclear Industry Applications

Polylactic acid (PLA) as a “green,” renewable corn-soy based polymer resin was assessed as a novel solid-state detector for rapid-turnaround gamma radiation dosimetry in the 1–100 kGy range–of significant interest in biomedical and general nuclear industry applications. Co-60 was used as the source of gamma photons. It was found that PLA resin responds well in terms of rheology and porosity metrics with an absorbed gamma dose (Dg). In this work, rheological changes were ascertained via measuring the differential mass loss ratio (MLR) of irradiated PLA placed within PTFE-framed (40 mm × 20 mm × 0.77 mm) cavities bearing ~0.9 g of PLA resin and pressed for 12–16 min in a controlled force hot press under ~6.6 kN loading and platens heated to 227 °C for the low Dg range: 0–11 kGy; and to 193 °C for the extended Dg range: 11–120 kGy. MLR varied quadratically from 0.05 to ~0.2 (1σ ~ 0.007) in the 0–11 kGy experiments, and from 0.05 to ~0.5 (1σ ~0.01) in the 0–120 kGy experiments. Rheological changes from gamma irradiation were modeled and simultaneously correlated with void-pocket formations, which increase with Dg. A single PLA resin bead (~0.04 g) was compressed 5 min at 216 °C in 0–16 kGy experiments, and compressed 2 min at 232 °C in the 16–110 kGy experiments, to form sturdy ~100 µm thick wafers in the same press. Aggregate coupon porosity was then readily measurable with conventional optical microscope imaging and analyzed with standard image processing; this provided complementary data to MLR. Average porosity vs. dose varied quadratically from ~0 to ~15% in the 0–16 kGy range and from ~0 to ~18% over the 16–114 kGy range. These results provide evidence for utilizing “green”/renewable (under $0.01) PLA resin beads for rapid and accurate (+/-5–10%) gamma dosimetry over a wide 0–120 kGy range, using simple to deploy mass and void measuring techniques using common laboratory equipment.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same domain, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications, which rely on large quantities of data from multiple sources. To address these challenges, the open-source Time-Series Data Pipelines (Tsdat) Python framework was developed by Pacific Northwest National Laboratory, with strategic guidance and direction provided by the National Renewable Energy Laboratory and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the marine energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat framework and the data standards within which it operates. A beta version of Tsdat has been released and is being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented data standards and automated data pipelines for the following data types: 1) drilling data, 2) geospatial datasets, and 3) DAS data. An additional data pipeline is proposed for stimulation data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how we can improve this process.

cloud-optimized↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented or is currently implementing data standards and automated data pipelines for the following geothermal data types: 1) drilling data, 2) geospatial datasets, and 3) Distributed Acoustic Sensing (DAS) data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how the GDR team can improve this process.

cloud-optimized↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented data standards and automated data pipelines for the following data types: 1) drilling data, 2) geospatial datasets, and 3) DAS data. An additional data pipeline is proposed for stimulation data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how we can improve this process.

cloud-optimized↗
DOE OSTI2024

DOE BSSD Performance Management Metrics Report Q1

Microbes play key roles in our biosphere, from driving global nutrient cycling to impacting plant, animal and human health and disease. Complex data from microbial genomes, proteins, and metabolites provide a window into these tiny engines that drive life on our planet. Yet these data are dispersed among researchers’ laboratories and various repositories, making it difficult to access. This calls for new ways of managing data, improving data interoperability, advancing community standards, and creating an infrastructure where data are shared efficiently. We have built the National Microbiome Data Collaborative (NMDC) to advance how scientists create, use, and reuse data to redefine the way we understand and harness the power of microbes. The vision of the National Microbiome Data Collaborative (NMDC) is to drive a microbiome data sharing network connecting data, people, and ideas to advance microbiome innovation and discovery. The NMDC was launched in 2019 and brought together DOE National Laboratories to collaborate across resources, capabilities, and expertise. The NMDC team was strategically assembled to include software developers, microbial researchers, metadata experts, and multi-omics specialists. The diversity of the NMDC team reflects the inherently interdisciplinary nature of microbiome science, and we leverage the strengths of the DOE National Laboratory system. Towards BER’s goal of advancing an iterative systems biology approach to the understanding of microbial genomes, the NMDC serves as a foundation for infrastructure, data standards, and community building. Together with the flagship DOE User Facilities, the Joint Genome Institute (JGI) and the Environmental Molecular Sciences Laboratory (EMSL), we are developing core capabilities in metadata standards for environmental descriptors and sample handling and processing; standardized bioinformatic workflows; an interface for data search and access; and robust community engagement activities. The NMDC production platform supports long-term data infrastructure and community building for BER’s bioenergy and environmental research goals. Our approach leverages lessons learned and an ambitious framework for collaborative, interdisciplinary data infrastructure to support microbiome research. The NMDC supports data, information, and knowledge access through three defined software tools – the Submission Portal, NMDC EDGE, and the Data Portal – driven by community needs. Herein, we describe the value proposition for the microbiome research community, our overarching strategy, and challenges and opportunities for developing the NMDC as both an infrastructure and community engagement program.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Amplitude Analysis of the B 0 → K * 0 μ + μ − Decay

An amplitude analysis of the B 0 → K * 0 μ + μ − decay is presented using a dataset corresponding to an integrated luminosity of 4.7 fb − 1 of p p collision data collected with the LHCb experiment. For the first time, the coefficients associated to short-distance physics effects, sensitive to processes beyond the standard model, are extracted directly from the data through a q 2 -unbinned amplitude analysis, where q 2 is the μ + μ − invariant mass squared. Long-distance contributions, which originate from nonfactorizable QCD processes, are systematically investigated, and the most accurate assessment to date of their impact on the physical observables is obtained. The pattern of measured corrections to the short-distance couplings is found to be consistent with previous analyses of b - to s -quark transitions, with the largest discrepancy from the standard model predictions found to be at the level of 1.8 standard deviations. The global significance of the observed differences in the decay is 1.4 standard deviations. © 2024 CERN, for the LHCb Collaboration 2024 CERN

Aaij, R. (ORCID:0000000305331952)↗
DOE OSTI2024

Cellulose molecular weight of corn stover with cotreatment

Here we present a dataset of cellulose molecular weights from corn stover after a cotreatment process. The corn stover sample was subjected to a combination of mechanical disruption (ball milling) and consolidated bioprocessing (CBP) process using the bacterium C. thermocellum at 60 grams/L solids loadings. Cellulose was isolated from control, no cotreatment control, and cotreatment corn stover. Cellulose was then derivatized using anhydrous pyridine and phenyl isocyanate over 48 h at 70 ºC. The reaction was quenched by anhydrous methanol. Methanol and water mixture (7:3, v/v) was added dropwise to promote precipitation of the cellulose derivative. The solids were collected by filtration and thoroughly washed with the methanol and water mixture followed by water. The cellulose derivative was dried overnight under vacuum at 40°C. The obtained cellulose tricarbanilates was then dissolved in tetrahydrofuran (THF) at 1.0 mg/mL and the solution was filtered through a 0.45 µm PTFE filter and placed in an auto-sampler vial. The molecular weight of cellulose was measured by Agilent GPC SECurity 1200 system equipped with four Waters Styragel columns (i.e., HR0.5, HR2, HR4, and HR6) and a UV detector (270 nm). THF was used as the mobile phase. Data collection and processing was performed by Polymer Standards Service WinGPC Unity software (Build 6807). The molecular weight was calculated relative to the polystyrene calibration curve. The data on molecular weights provided information about the effects of cotreatment on corn stover cellulose molecular changes.

Cellulose, molecular weights, corn stover, cotreat↗
DOE OSTI2025

Bayesian Statistical Analysis for Mass Spectrometric Data Processing

• Thermal ionization mass spectrometry (TIMS) is a “gold standard’ technique for actinide isotope amount ratio and assay measurements. • Ubiquitously used in: • Nuclear Nonproliferation • Nuclear Safeguards • Nuclear Forensics • Basic Science • Savannah River National Laboratory (SRNL) installed a new Thermo Scientific TRITON Plus TIMS in early 2024.

McLarty, Ellis C. [Savannah River National Laborat↗
DOE OSTI2025

Bayesian Statistical Analysis for Mass Spectrometric Data Processing

Thermal ionization mass spectrometry (TIMS) is a “gold standard’ technique for actinide isotope amount ratio and assay measurements. • Ubiquitously used in: • Nuclear Nonproliferation • Nuclear Safeguards • Nuclear Forensics • Basic Science • Savannah River National Laboratory (SRNL) installed a new Thermo Scientific TRITON Plus TIMS in early 2024.

McLarty, Ellis C. [Savannah River National Laborat↗
DOE OSTI2022

2.3.3.404 - National Lab and University Collaboration for MHK Instrumentation and Data Processing Tools

Field and laboratory validation, testing, demonstration, and operation are critical steps for increasing the technology readiness level of marine energy (ME) converters because they provide high-quality testing and performance data that are critical information used to feed all aspects of technology development. This project, in partnership with industry, enables the marine and hydrokinetic energy (MHK) community to reliably and efficiently collect, process, manage, and share quality data by facilitating access to and development of instrumentation, guidelines and data processing/QA tools. Under this project, open-source data processing code (MHKiT) and tools (ME Data Pipeline, MRE Code Hub, PRIMRE Code Catalog), instrumentation (loads measurements), data acquisition systems (miniDAQ), and measurement guidance tools (Telesto, high EMI guidance) were developed to facilitate the collection and processing of quality laboratory and field data. Overall, this project is intended to improve the quality of the data collected during laboratory and field demonstration projects by standardizing the collection and processing techniques, as well as by improving access to instrumentation, code, and measurement guidance. Quality data will, in turn, lead to improved knowledge capture following ME device testing.

data processing↗
DOE OSTI2023

Search for long-lived, massive particles in events with displaced vertices and multiple jets in pp collisions at $\sqrt{s}$ = 13 TeV with the ATLAS detector

A search for long-lived particles decaying into hadrons is presented. The analysis uses 139 fb –1 of $pp$ collision data collected at $\sqrt{s}$ = 13 TeV by the ATLAS detector at the LHC using events that contain multiple energetic jets and a displaced vertex. The search employs dedicated reconstruction techniques that significantly increase the sensitivity to longlived particles decaying in the ATLAS inner detector. Background estimates for Standard Model processes and instrumental effects are extracted from data. The observed event yields are compatible with those expected from background processes. The results are used to set limits at 95% confidence level on model-independent cross sections for processes beyond the Standard Model, and on scenarios with pair-production of supersymmetric particles with long-lived electroweakinos that decay via a small $R$-parity-violating coupling. The pair-production of electroweakinos with masses below 1.5 TeV is excluded for mean proper lifetimes in the range from 0.03 ns to 1 ns. When produced in the decay of $m$($\tilde{g}$) = 2.4 TeV gluinos, electroweakinos with $m$($\tilde{χ}_{1}^{0}$) = 1.5 TeV are excluded with lifetimes in the range of 0.02 ns to 4 ns.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

SPRUCE Vegetation Phenology in Experimental Plots from Phenocam Imagery, 2015-2022

This data set consists of PhenoCam data from the SPRUCE experiment from the beginning of whole ecosystem warming (Hanson et al. 2017) in August 2015 through March 31 of 2023, with start- and end-of-season phenological transition dates derived through the end of autumn 2022. Digital cameras, or phenocams, installed in each SPRUCE enclosure track seasonal variation in vegetation “greenness”, a proxy for vegetation phenology and associated physiological activity. Three separate regions of interest (ROIs) were defined for each camera field of view, corresponding to different vegetation types and demarcating (1) Picea trees (vegetation type EN, for evergreen needleleaf); (2) Larix trees (vegetation type DN, for deciduous needleleaf); and (3) the mixed shrub layer (vegetation type SH). This data set consists of three sets of data files: 3-day summary product files: One file for each camera and each ROI (i.e. vegetation type), characterizing vegetation color at a 3-day time step Contains 35 files in *.csv format inside a compressed (*.zip) file. Transition date file: Estimates “greenness rising” (spring) and “greenness falling” (autumn) transition dates derived from the smoothed daily green chromatic coordinate (GCC) values, for each camera and each ROI (i.e. vegetation type) Contains one file in *.csv format Snow flag files: Indicate days with snow on trees or snow on ground for each experimental enclosure Contains two files in *.csv format, one for snow on trees and one for snow on ground This data set consists of two sets of companion files: Accompanying HTML files show the 90th quantiles of the mean GCC plotted together with transition dates for each vegetation type and plot. Contains three files in HTML format, one for each vegetation type One additional file in HTML format with the transition dates plotted for each vegetation type, by year R files for processing Phenocam files and flags. Contains five files in R file (*.R) format in one compressed (*.zip) file User Note: All imagery is posted in near-real time to the PhenoCam Project web page (http://phenocam.sr.unh.edu/), where it is publicly available. Scroll to “spruce” in the Gallery or link directly to the 29 SPRUCE cameras at https://tinyurl.com/y7z5mau7. The data reported here are based on the complete camera record from SPRUCE and supersedes the previously released phenocam datasets (see Related Data Sets). The estimated transition dates for previously released datasets may differ slightly (in most cases, by ±3 days or less), because following standard PhenoCam processing protocols (Richardson et al. 2018, Scientific Data), smoothing and interpolation, outlier removal, and transition date estimation are always conducted using the full data record.

SPRUCE Experiment, Marcell Experimental Forest, Sp↗
DOE OSTI2024

SPRUCE Vegetation Phenology in Experimental Plots from Phenocam Imagery, 2015-2023

This data set consists of PhenoCam data from the SPRUCE experiment from the beginning of whole ecosystem warming (Hanson et al. 2017) in August 2015 through March 31 of 2024, with start- and end-of-season phenological transition dates derived through the end of autumn 2023. Digital cameras, or phenocams, installed in each SPRUCE enclosure track seasonal variation in vegetation “greenness”, a proxy for vegetation phenology and associated physiological activity. Three separate regions of interest (ROIs) were defined for each camera field of view, corresponding to different vegetation types and demarcating (1) Picea trees (vegetation type EN, for evergreen needleleaf); (2) Larix trees (vegetation type DN, for deciduous needleleaf); and (3) the mixed shrub layer (vegetation type SH). This data set consists of three sets of data files: (1) 3-day summary product files: One file for each camera and each ROI (i.e. vegetation type), characterizing vegetation color at a 3-day time step • Contains 36 files in *.csv format inside a compressed (*.zip) file. (2) Transition date file: Estimates “greenness rising” (spring) and “greenness falling” (autumn) transition dates derived from the smoothed daily green chromatic coordinate (GCC) values, for each camera and each ROI (i.e. vegetation type) • Contains one file in *.csv format (3) Snow flag files: Indicate days with snow on trees or snow on ground for each experimental enclosure • Contains two files in *.csv format, one for snow on trees and one for snow on ground This data set consists of two sets of companion files: (1) Accompanying HTML files show the 90th quantiles of the mean GCC plotted together with transition dates for each vegetation type and plot. • Contains three files in HTML format, one for each vegetation type • One additional file in HTML format with the transition dates plotted for each vegetation type, by year (2) R files for processing Phenocam files and flags. • Contains five files in R file (*.R) format in one compressed (*.zip) file User Note: All imagery is posted in near-real time to the PhenoCam Project web page (http://phenocam.sr.unh.edu/), where it is publicly available. Scroll to “spruce” in the Gallery or link directly to the 29 SPRUCE cameras at https://tinyurl.com/sprucecams. The data reported here are based on the complete camera record from SPRUCE and supersedes the previously released data inclusive of the 2015-2022 data (see Related Data Sets). The estimated transition dates for previously released datasets may differ slightly (in most cases, by ±3 days or less), because following standard PhenoCam processing protocols (Richardson et al. 2018, Scientific Data), smoothing and interpolation, outlier removal, and transition date estimation are always conducted using the full data record.

Spruce and Peatland Responses Under Changing Envir↗
DOE OSTI2022

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond: Preprint

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same field, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications. To address these challenges, the open source Time-Series Data Pipelines (Tsdat) software was developed by a joint collaboration between Pacific Northwest National Laboratory, the National Renewable Energy Laboratory, and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the Marine Energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat software and the data standards within which the framework operates. A beta version of the framework has been released and is currently being used by several projects in marine energy, wind energy, and building energy systems.

big data↗
DOE OSTI2025

SPRUCE Vegetation Phenology in Experimental Plots from PhenoCam Imagery, 2015-2024

This data set consists of PhenoCam data from the SPRUCE experiment from the beginning of whole ecosystem warming (Hanson et al. 2017) in August 2015 through March 31 of 2025 (2015-08-24 to 2025-03-31), with start- and end-of-season phenological transition dates derived through the end of autumn 2024. Digital cameras, or phenocams, installed in each SPRUCE enclosure track seasonal variation in vegetation “greenness”, a proxy for vegetation phenology and associated physiological activity. Three separate regions of interest (ROIs) were defined for each camera field of view, corresponding to different vegetation types and demarcating (1) Picea trees (vegetation type EN, for evergreen needleleaf); (2) Larix trees (vegetation type DN, for deciduous needleleaf); and (3) the mixed shrub layer (vegetation type SH). This data set consists of three sets of data files: (1) 3-day summary product files: One file for each camera and each ROI (i.e. vegetation type), characterizing vegetation color at a 3-day time step. • Contains 36 files in *.csv format inside a compressed (*.zip) file. (2) Transition date file: Estimates “greenness rising” (spring) and “greenness falling” (autumn) transition dates derived from the smoothed daily green chromatic coordinate (GCC) values, for each camera and each ROI (i.e., vegetation type). • Contains one file in *.csv format. (3) Snow flag files: Indicate days with snow on trees or snow on ground for each experimental enclosure. • Contains two files in *.csv format, one for snow on trees and one for snow on ground. This data set consists of two sets of companion files: (1) Accompanying HTML files show the 90th quantiles of the mean GCC plotted together with transition dates for each vegetation type and plot. • Contains three files in HTML format, one for each vegetation type. • One additional file in HTML format with the transition dates plotted for each vegetation type, by year. (2) R files for processing PhenoCam files and flags. • Contains five files in R file(*.R) format and the components of the phenocamr package (Version 1.1.4) used for calculating transition dates for 2015-2024. These are contained in a compressed (*.zip) file. User Note: All imagery is posted in near-real time to the PhenoCam Project web page (https://phenocam.nau.edu), where it is publicly available. Scroll to “spruce” in the Gallery or link directly to the 29 SPRUCE cameras at https://tinyurl.com/sprucecams. This data set is based on the complete camera record from SPRUCE and supersedes all previously released PhenoCam datasets (see Related Data Sets). The estimated transition dates for previously released datasets may differ slightly (in most cases, by ±3 days or less), because following standard PhenoCam processing protocols (Richardson et al. 2018, Scientific Data), smoothing and interpolation, outlier removal, and transition date estimation are always conducted using the full data record.

54 ENVIRONMENTAL SCIENCES↗