Micro-kinetic modeling of temporal analysis of products data using kinetics-informed neural networks
Kinetics-informed neural networks improve fit quality for multi-pulse and noisy temporal analysis of products datasets.
Nai, Dingqi↗
SEARCH · Engineering Papers
Results for “data product”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
MINERvA open-data product
MINERvA is THE neutrino cross section experiment Scintillator tracker/calorimeter ran in the NuMI beam at Fermilab same beam as the MINOS and NOvA oscillation experiments With our data, we are solving systematic shortcomings in neutrino interaction rate/spectra that are the largest part of the systematic uncertainty in today s (and tomorrow s) measurements. Some aspects have NO equivalent in the neutrino program future. Scientific scope includes both particle and nuclear physics GeV scale cross sections, A dependence, MeV scale effects most published measurements for a neutrino experiments ever (well, tied with T2K) with 25% more papers in the pipeline.
Gran, Rik [Minnesota U., Duluth] (ORCID:0000000216↗
Merged Observatory Data Files (MODFs): an integrated observational data product supporting process-oriented investigations and diagnostics
A large and ever-growing body of geophysical information is measured in campaigns and at specialized observatories as a part of scientific expeditions and experiments. These collections of observed data include many essential climate variables (as defined by the Global Climate Observing System) but are often distinguished by a wide range of additional non-routine measurements that are designed to not only document the state of the environment but also the drivers that contribute to that state. These field data are used not only to further understand environmental processes through observation-based studies but also to provide baseline data to test model performance and to codify understanding to improve predictive capabilities. To address the considerable barriers and difficulty in utilizing these diverse and complex data for observation–model research, the Merged Observatory Data File (MODF) concept has been developed. A MODF combines measurements from multiple instruments into a single file that complies with well-established data format and metadata practices and has been designed to parallel the development of corresponding Merged Model Data Files (MMDFs). Using the MODF and MMDF protocols will facilitate the evolution of model intercomparison projects into model intercomparison and improvement projects by putting observation and model data “on the same page” in a timely manner. The MODF concept was developed especially for weather forecast model studies in the Arctic. The surprisingly complex process of implementing MODFs in that context refined the concept itself. Thus, this article explains the concept of MODFs by providing details on the issues that were revealed and resolved during that first specific implementation. Detailed instructions are provided on how to make MODFs, and this article can be considered a MODF creation manual.
54 ENVIRONMENTAL SCIENCES↗
Atmospheric Radiation Measurement (ARM) airborne field campaign data products between 2013 and 2018
Airborne measurements are pivotal for providing detailed, spatiotemporally resolved information about atmospheric parameters and aerosol and cloud properties, thereby enhancing our understanding of dynamic atmospheric processes. For 30 years, the US Department of Energy (DOE) Office of Science supported an instrumented Gulfstream 1 (G-1) aircraft for atmospheric field campaigns. Data from the final decade of G-1 operations were archived by the Atmospheric Radiation Measurement (ARM) Data Center and made publicly available at no cost to all registered users. To ensure a consistent data format and to improve the accessibility of the ARM airborne data, an integrated dataset was recently developed covering the final 6 years of G-1 operations (2013 to 2018, https://doi.org/10.5439/1999133; Mei and Gaustad, 2024). The integrated dataset includes data collected from 236 flights (766.4 h), which covered the Arctic, the US Southern Great Plains (SGP), the US West Coast, the eastern North Atlantic (ENA), the Amazon Basin in Brazil, and the Sierras de Córdoba range in Argentina. These comprehensive data streams provide much-needed insight into spatiotemporal variability in the thermodynamic quantities and aerosol and cloud properties for addressing essential science questions in Earth system process studies. This paper describes the DOE ARM merged G-1 datasets, including information on the acquisition, data collection challenges and future potentials, and quality control processes. It further illustrates the usage of this merged dataset to evaluate the Energy Exascale Earth System Model (E3SM) with the Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package.
54 ENVIRONMENTAL SCIENCES↗
Image masks of global ship tracks for NASA MODIS data products
Ship tracks, long thin artificial cloud features formed from the pollutants in ship exhaust, are satellite-observable examples of aerosol-cloud interactions (ACI) that can lead to increased cloud albedo and thus increased solar reflectivity, phenomena of interest in solar radiation management. In addition to ship tracks being of interest to meteorologists and policy makers, their observed cloud perturbations provide benchmark evidence of ACI that remain poorly captured by climate models. To broadly analyze the effects of ship tracks, high-resolution satellite imagery data highlighting their presence are required. To support this, we provide a hand labelled dataset to serve as a benchmark for a variety of subsequent analyses. Established from a previous dataset that identified ship track presence using NASA’s MODIS Aqua satellite imager, our first-of-its-kind dataset is comprised of image masks: capturing full ship track regions, including their contours, emission points and dispersive patterns. In total, 300 images, or around 2,500 masked ship tracks, observed under varying conditions are provided, and may facilitate training of machine learning algorithms to automate extraction.
Atmospheric dynamics↗
Advanced Precipitation and Boundary Layer Data Products Derived from ARM Radar Wind Profilers
This research project was successful in delivering on four main objectives. First, software was developed to accurately calculate 915-MHz radar wind profiler (RWP) spectrum moments from the recorded Doppler velocity power spectra. Second, software was developed to calibrate the RWP reflectivity factor using collocated surface disdrometer observations. Third, the Python processing code was documented and given to the ARM Infrastructure to produce ARM ‘b level’ calibrated RWP products. Fourth, calibrated RWP products were uploaded to the ARM Archive as PI Products for 10 years of SGP RWP observations and for GoAmazon and TRACER field campaign RWP observations. In addition to working with RWP observations, this research project also worked with KAZR observations to distinguish insects from boundary layer clouds to help improve the ARSCL cloud mask product. The PI worked with senior and early career ARM funded scientists at BNL exploring how to include calibrated RWP moments into future versions of the ARSCL product.
54 ENVIRONMENTAL SCIENCES↗
Establishing robust correction schemes for improved and reliable ARM-AOS aerosol optical data products
Aerosol light absorption and scattering of solar radiation play an important role in the earth’s atmosphere in terms of direct and semi-direct radiative forcing. Optical parameters of importance to the US Department of Energy (DOE) climate models include absorption and scattering coefficients, single scattering albedo (SSA), absorption Angstrom exponents (AAE), and the asymmetry parameter (g). These parameters depend on aerosol size, shape and composition (refractive index), and are spectrally sensitive in the shortwave region. Additionally, these parameters have a complex dependency on the emission source, especially for carbonaceous aerosols. The DOE Atmospheric Radiation Measurement (ARM) user facility has deployed aerosol observing systems (AOS) containing several filter-based instruments to measure and constrain aerosol optical properties and related parameters at multiple sites worldwide. For measurement of aerosol light absorption, the AOS includes filter-based instruments (particle soot absorption photometer and tricolor absorption photometer) that infer particle-phase aerosol absorption coefficients at nominal red, green, and blue wavelength bands from the attenuation (ATN) of light passing through a particulate filter on which aerosols are deposited. Measurement of aerosol scattering is done in situ using nephelometers. By combining inferred absorption coefficients from filter-based ATN measurements and in situ scattering coefficients, value-added products (VAPs) such as SSA, AAE, and g are derived.
54 ENVIRONMENTAL SCIENCES↗
Obtaining Real Production Data through On-Component Printed SIR Patterns
Explore the source record for details and available documents.
Hartley, Daniel [Kansas City National Security Cam↗
Atmospheric Radiation Measurement (ARM) airborne field campaign data products between 2013 and 2018
Explore the source record for details and available documents.
54 ENVIRONMENTAL SCIENCES↗
Vera C. Rubin Observatory Prompt Products
Data products produced by prompt and daily processing of images obtained in the Legacy Survey of Space and Time. These include realtime alerts sent to community alert brokers, newly-discovered Solar System Objects reported to the Minor Planet Center, processed visit and difference images, and source catalogs. Prompt Products are not a static single data release but continually grow throughout the ten-year LSST survey.
79 ASTRONOMY AND ASTROPHYSICS↗
Vegetation Warming Experiment: Environmental Conditions, Utqiagvik (Barrow), Alaska, 2018
Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 16 June - 24 September, 2018. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).
54 ENVIRONMENTAL SCIENCES↗
Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2019
Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June – 25 September, 2019. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017–2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).
54 ENVIRONMENTAL SCIENCES↗
Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2021
Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June - 17 September, 2021. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).
54 ENVIRONMENTAL SCIENCES↗
Raw ERCOT 60-Day SCED Disclosure Reports
This dataset contains ERCOT plant-level wind power production data at a 15-minute time resolution. Data at ERCOT are called 60-Day SCED Disclosure Report; data product number NP3-965-ER. Raw data are in local time. DST is handled as "skip hour" (12a, 1a, 3a,...) in spring, "extra hour" (12a, 1a, 2a, 2a, 3a,...) in fall.
17 WIND ENERGY↗
Vera C. Rubin Observatory Prompt Products: alert packets data
Data products produced by prompt and daily processing of images obtained in the Legacy Survey of Space and Time. These include realtime alerts sent to community alert brokers, newly-discovered Solar System Objects reported to the Minor Planet Center, processed visit and difference images, and source catalogs. Prompt Products are not a static single data release but continually grow throughout the ten-year LSST survey. This dataset is a subset of the full data release consisting of a dataset named alert packets. This dataset contains measurements for 5-sigma sources detected in difference images that were issued to the community brokers.
79 ASTRONOMY AND ASTROPHYSICS↗
Machine Learned Empirical Numerical Integrator from Simulated Data
Recently, a number of state-of-the-art surrogate machine learning (ML) models have been designed for global weather and climate prediction, which have been trained using reanalysis data products. Reanalysis data products are constructed using numerical model simulations that combine numerical integration of partial differential equations and parameterization schemes. These products are typically only archived and made available using coarsened spatial and temporal resolutions. This study explores the impact of the numerical generation methods used to produce the training datasets and the temporal resolution of those datasets on machine learning surrogate models. Using the nonlinear vector autoregression (NVAR) machine as an explainable ML technique, simple dynamical systems are emulated with ML models trained on data produced by three classical numerical integration schemes. NVAR is validated as a skillful ML method, capable of producing accurate predictions and, more importantly, reconstructing both the underlying dynamics and the numerical integration scheme used to generate the training data. However, the machine fails to generalize predictions on unseen test data generated by different numerical integration schemes, despite the underlying dynamical system being the same. This result provides a word of caution for the growing field of machine learning emulation of weather and climate dynamics. Furthermore, we illustrate using NVAR that training on temporally coarsened data may increase the required complexity of ML models and potentially introduce new numerical challenges. Finally, we discover that empirical integration schemes with arbitrary time-stepping sizes can be constructed directly from the data, which implies a potential for the development of empirical numerical integration schemes.
54 ENVIRONMENTAL SCIENCES↗
Statistical Validation of Multiple Related Data Sets—Case Study Using Interstellar Boundary Explorer Satellite Data
Abstract Space scientists often face the question of whether data collected by different instruments are measurements of the same source population. This paper proposes a statistical validation method for evaluating the agreement between such related data sets. It offers a detailed case study focused on validating a new data set from the Interstellar Boundary Explorer (IBEX) mission, which serves as a practical how-to guide for similar analyses. Since 2008, the IBEX satellite has been gathering data on heliospheric energetic neutral atoms (ENAs) while being exposed to various sources of background noise, such as cosmic rays and solar energetic particles. The IBEX mission initially released only a qualified triple-coincidence (qABC) data product, which was designed to provide observations of ENAs free of background contamination. Further measurements revealed that the qABC data were in fact susceptible to contamination, having relatively low ENA counts and high background rates. To mitigate this issue, the mission team recently considered releasing a certain qualified double-coincidence (qBC) data product, which has roughly twice the detection rate of the qABC data product. This paper presents a simulation-based validation of the new qBC data product against the already-released qABC data product. The results show that the qBCs can plausibly be said to be measuring the same source population as the qABCs up to an average absolute deviation of 3.6%. Visual diagnostics provide additional confirmation of source rate coherence across data products. The framework introduced here is general and can be applied to other validation problems both within and outside the field of space physics.
79 ASTRONOMY AND ASTROPHYSICS↗
Advancing Multiscale Simulation of Plasma-Surface Interfaces
We report the development of an atomistic-informed, surface-state-dependent predictive model for particle exchange in a carbon-tungsten plasma-surface interface. The predictive model uses machine learning (ML) techniques to learn the energy and angular distributions for particle exchange and rate functions for surface state evolution from molecular dynamics simulations of cumulative bombardment of tungsten by energetic carbon ions. Each predictive component is sensitive to the energy and trajectory of incident plasma species and the surface state. The surface state is represented by a set of surface state descriptors, which were derived from the atomistic surface state for each independent carbon bombardment event. These descriptors are representative of the composition and degree of amorphization of the outermost angstrom of surface material and were chosen to optimize predictive performance for particle exchange at the interface. The distributions for particle exchange (reflection/sputtering) are demonstrated to vary with each surface state descriptor, motivating the development of surface-state-dependent particle exchange models for plasma simulations. The performance of various ML methods was compared, including polynomial quantile regression, artificial neural networks, k-nearest neighbors, and random forest algorithms, with polynomial regression performing the best for interpolation and extrapolation of learned relationships. In addition to the particle exchange model, a neutral network was developed and used to identify data sufficiency throughout surface descriptor space, which will enable real-time feedback during future data production to ensure data is produced where it is most needed, and we provide commentary on improvements to the data production workflow for future endeavors.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗