Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Raw Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Predicting weather impacts on corn production in a data-limited region using a transfer learning approach

The stability of food supply and prices may depend more on annual changes in yields from year-to-year variability in weather than on longer-term average changes from changing climatic conditions. However, the absence of high-quality data on crop yields at fine spatial resolutions in many regions of the world makes it challenging to statistically model their response to interannual variability in weather patterns. Therefore, there is a need for empirical methods that can project annual crop yield changes even in limited data regions. Here, we propose a transfer learning algorithm that uses high spatial resolution data from one region to project yields in another region with more limited data. The goal of our work is to understand what data types can be beneficial for transferring learning from a source region to a very different target region with more limited data. We utilize Long Short-Term Memory to develop a transfer learning model that is trained on historical county-level corn yield in the United States and predicts district-level corn yield variations in India. Even using smaller amounts of data in India, simulating a data-scarce region, we achieve an average root mean square error of 0.48 bu acre−1 in predicting interannual yield variations. Using Shapley values to interpret results, we explore the contribution of the different weather parameters to interannual yield variability and find a larger influence of precipitation-related variables. Our study demonstrates the usefulness of this method for transferring models of weather impacts on crop yields trained on a data-rich country to one with more limited data. It suggests the potential of applying the transfer learning model to mitigate the need for extensive raw data globally.

Vishwakarma, Srishti [ORNL] (ORCID:000000031674419↗

Data for reproducing the figures of the paper Multimodal Super-Resolution: Discovering hidden physics and its application to fusion plasmas

This deposit contains the raw data for reproducing research results of the paper Multimodal Super-Resolution: Discovering hidden physics and its application to fusion plasmas. The main contribution of this work is to utilize machine learning techniques to reconstruct and enhance the resolution of a diagnostic measurement from other available diagnostics in a system. The proposed techniques is called Diag2Diag.

diag2diag↗

FL‐ADS: Federated learning anomaly detection system for distributed energy resource networks

Abstract With the ongoing development of Distributed Energy Resources (DER) communication networks, the imperative for strong cybersecurity and data privacy safeguards is increasingly evident. DER networks, which rely on protocols such as Distributed Network Protocol 3 and Modbus, are susceptible to cyberattacks such as data integrity breaches and denial of service due to their inherent security vulnerabilities. This paper introduces an innovative Federated Learning (FL)‐based anomaly detection system designed to enhance the security of DER networks while preserving data privacy. Our models leverage Vertical and Horizontal Federated Learning to enable collaborative learning while preserving data privacy, exchanging only non‐sensitive information, such as model parameters, and maintaining the privacy of DER clients' raw data. The effectiveness of the models is demonstrated through its evaluation on datasets representative of real‐world DER scenarios, showcasing significant improvements in accuracy and F1‐score across all clients compared to the traditional baseline model. Additionally, this work demonstrates a consistent reduction in loss function over multiple FL rounds, further validating its efficacy and offering a robust solution that balances effective anomaly detection with stringent data privacy needs.

Purohit, Shaurya [Iowa State University Ames Iowa ↗

A comprehensive guide to CAN IDS data and introduction of the ROAD dataset

Although ubiquitous in modern vehicles, Controller Area Networks (CANs) lack basic security properties and are easily exploitable. A rapidly growing field of CAN security research has emerged that seeks to detect intrusions or anomalies on CANs. Producing vehicular CAN data with a variety of intrusions is a difficult task for most researchers as it requires expensive assets and deep expertise. To illuminate this task, we introduce the first comprehensive guide to the existing open CAN intrusion detection system (IDS) datasets. We categorize attacks on CANs including fabrication (adding frames, e.g., flooding or targeting and ID), suspension (removing an ID’s frames), and masquerade attacks (spoofed frames sent in lieu of suspended ones). We provide a quality analysis of each dataset; an enumeration of each datasets’ attacks, benefits, and drawbacks; categorization as real vs. simulated CAN data and real vs. simulated attacks; whether the data is raw CAN data or signal-translated; number of vehicles/CANs; quantity in terms of time; and finally a suggested use case of each dataset. State-of-the-art public CAN IDS datasets are limited to real fabrication (simple message injection) attacks and simulated attacks often in synthetic data, lacking fidelity. In general, the physical effects of attacks on the vehicle are not verified in the available datasets. Only one dataset provides signal-translated data but is missing a corresponding “raw” binary version. This issue pigeon-holes CAN IDS research into testing on limited and often inappropriate data (usually with attacks that are too easily detectable to truly test the method). The scarcity of appropriate data has stymied comparability and reproducibility of results for researchers. As our primary contribution, we present the Real ORNL Automotive Dynamometer (ROAD) CAN IDS dataset, consisting of over 3.5 hours of one vehicle’s CAN data. ROAD contains ambient data recorded during a diverse set of activities, and attacks of increasing stealth with multiple variants and instances of real (i.e. non-simulated) fuzzing, fabrication, unique advanced attacks, and simulated masquerade attacks. To facilitate a benchmark for CAN IDS methods that require signal-translated inputs, we also provide the signal time series format for many of the CAN captures. Our contributions aim to facilitate appropriate benchmarking and needed comparability in the CAN IDS research field.

97 MATHEMATICS AND COMPUTING↗

DETAIL Component Scaling and Methodology Comparison

The purpose of this study was to develop a process to convert input signals from one facility into another by reflecting geometric and environmental settings. The Dynamic Energy Transport and Integration Laboratory (DETAIL) is a research facility in development. Its aim is to emulate the daily interactions among power production industry systems and receive real-time data from those systems as inputs. To convert signals and ensure that the temporal sequences and magnitudes reflect laboratory settings, the ability to scale and project data is essential. To demonstrate this ability, Dynamical System Scaling (DSS) and Hierarchical Two-Tiered Scaling (H2TS) (methodologies that enable systems to scale and project or extrapolate data sets to desired environments while conserving the observed behavior based on first principles) were applied to DETAIL’s thermocline thermal storage system in the Thermal Energy Distribution System (TEDS) facility and solid-oxide electrolysis cell in the High Temperature Hydrogen Electrolysis (HTHE) facility. Both thermocline and electrolysis cell systems were successfully scaled, and test cases were conducted to generate a doubly accelerated energy charge and discharge in reference to past experimental data from the facilities. The research results represented a case for the thermocline system that required signals to be accelerated without altering the stored energy. To enhance the quality of the accelerated data, error propagation analyses were conducted on DSS post-processing terms to determine the consequences of raw-data-associated errors.

08 HYDROGEN↗

Neutron Coincidence Measurements of Uranium-233 Oxide

Renewed international interest in thorium-fueled advanced reactors has challenged the safeguards community to address future proliferation concerns. Thorium-based technology presents many benefits but does not eliminate the proliferation risks associated with producing and processing fissile material. A byproduct of thorium-fueled reactors is uranium-233, which is classified as a direct-use material. As a result, the development of new or improved methods to characterize and measure materials containing 233U must mirror the pace of development of reactors and facilities that produce such material. Research is underway to assess, develop, and test approaches for safeguarding nuclear materials within the thorium fuel cycle. Neutron signatures from the nondestructive assay (NDA) of materials containing 233U are being quantified to inform the potential characterization of these materials. Using a traditional neutron coincidence counter and a series of well-documented 233U oxide samples, initial measurements have been made to assess the feasibility of 233U characterization and discrimination from other uranium isotopes, primarily 235U, using a combination of measurement techniques and analysis methods. Data acquisition is performed in list mode, allowing for a variety of analyses to be performed on the raw data that is not available using traditional shift register technology. Measurements were performed in passive and active configurations to quantify the strength of signal and to validate simulations in support of this work. This paper presents and discusses the results of the initial measurements of 233U oxide performed at Oak Ridge National Laboratory.

Lockhart, Madeline↗

Utah FORGE: Well 16A(78)-32 Stimulation Tracer Test Results

This archive contains data from the tracer test performed during the Utah FORGE well 16A(78)-32 stimulation. This includes: 1. the raw data file from Pason in the csv format; 2. a one-page Word document from Pason that explains the headers and units used in their data file; 3. an Excel spreadsheet with concentration data and relevant Pason flow data; 4. a Word document with jpg versions of concentration charts (Figs. 1-3) and a tracer mass recovery chart (Fig. 4).

15 GEOTHERMAL ENERGY↗

Maintaining Trust in Reduction: Preserving the Accuracy of Quantities of Interest for Lossy Compression

As the growth of data sizes continues to outpace computational resources, there is a pressing need for data reduction techniques that can significantly reduce the amount of data and quantify the error incurred in compression. Compressing scientific data presents many challenges for reduction techniques since it is often on non-uniform or unstructured meshes, is from a high-dimensional space, and has many Quantities of Interests (QoIs) that need to be preserved. To illustrate these challenges, we focus on data from a large scale fusion code, XGC. XGC uses a Particle-In-Cell (PIC) technique which generates hundreds of PetaBytes (PBs) of data a day, from thousands of timesteps. XGC uses an unstructured mesh, and needs to compute many QoIs from the raw data, f.One critical aspect of the reduction is that we need to ensure that QoIs derived from the data (density, temperature, flux surface averaged momentums, etc.) maintain a relative high accuracy. We show that by compressing XGC data on the high-dimensional, nonuniform grid on which the data is defined, and adaptively quantizing the decomposed coefficients based on the characteristics of the QoIs, the compression ratios at various error tolerances obtained using a multilevel compressor (MGARD) increases more than ten times. We then present how to mathematically guarantee that the accuracy of the QoIs computed from the reduced f is preserved during the compression. We show that the error in the XGC density can be kept under a user-specified tolerance over 1000 timesteps of simulation using the mathematical QoI error control theory of MGARD, whereas traditional error control on the data to be reduced does not guarantee the accuracy of the QoIs.

Gong, Qian↗

Parallelizing the Unpacking and Clustering of Detector Data for Reconstruction of Charged Particle Tracks on Multi-core CPUs and Many-core GPUs

We present results from parallelizing the unpacking and clustering steps of the raw data from the silicon strip modules for reconstruction of charged particle tracks. Throughput is further improved by concurrently processing multiple events using nested OpenMP parallelism on CPU or CUDA streams on GPU. The new implementation along with earlier work in developing a parallelized and vectorized implementation of the combinatoric Kalman filter algorithm has enabled efficient global reconstruction of the entire event on modern computer architectures. We demonstrate the performance of the new implementation on Intel Xeon and NVIDIA GPU architectures.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation

The arrival of direct electron detectors (DED) with high frame-rates in the field of scanning transmission electron microscopy has enabled many experimental techniques that require collection of a full diffraction pattern at each scan position, a field which is subsumed under the name four dimensional-scanning transmission electron microscopy (4D-STEM). DED frame rates approaching 100 kHz require data transmission rates and data storage capabilities that exceed commonly available computing infrastructure. Current commercial DEDs allow the user to make compromises in pixel bit depth, detector binning or windowing to reduce the per-frame file size and allow higher frame rates. This change in detector specifications requires decisions to be made before data acquisition that may reduce or lose information that could have been advantageous during data analysis. The 4D Camera, a DED with 87 kHz frame-rate developed at Lawrence Berkeley National Laboratory, reduces the raw data to a linear-index encoded electron event representation (EER). Here we show with experimental data from the 4D Camera that linear-index encoded EER and its direct use in 4D-STEM phase contrast imaging methods enables real-time, interactive phase-contrast from large-area 4D-STEM datasets. Furthermore, we detail the computational complexity advantages of the EER and the necessary computational steps to achieve real-time interactive ptychography and center-of-mass differential phase contrast using commonly available hardware accelerators.

4D-STEM↗

Dataset for manuscript "Equipartition and the temperature of maximum density of TIP4P/2005 water"

We simulate TIP4P/2005 water in the temperature range of 257 K to 318 K with time-steps 0.25, 0.50, 1.00, 2.00, and 4.00 fs. The density-temperature behavior obtained using 0.25 or 0.50 fs are in excellent agreement with each other but differ from those obtained using time-steps that have been shown earlier to lead to a breakdown of equipartition. The temperature of maximum density (TMD) is 277.15 K with time-step 0.25 or 0.50 fs, but is shifted to progressively lower values for longer time-steps, a trend that holds for different thermostat/barostat combinations. Enhancing the water-water dispersion interaction, as has been recommended for simulating disordered proteins in TIP4P/2005, degrades the description of the liquid-vapor phase envelope. We present a simple physically transparent reasoning to highlight the separation of the time-scales between translational and rotational motion. We also develop a metric, Chi, that we term the equipartition anomaly, to detect equipartition violations in simulations that include molecules that are treated as rigid objects. Calculating Chi is shown to be straightforward and sensitive to equipartition violations. A key takeaway from this study is that using sufficiently short time-steps (less than or equal to 0.5 fs) to preserve equipartition is essential for obtaining meaningful liquid water properties and for producing reliable simulation data, as correct-ensemble sampling is fundamental to ensure reproducibility across codes and simulation alogrithms. The included dataset provides the raw data used in the preparation of the graphs noted in the manuscript.

36 MATERIALS SCIENCE↗

Carbon dioxide (CO2) flux, amplicon sequencing and liquid chromatography–mass spectrometry (LC-MS) data for pyrogenic organic matter (PYOM) amended soil incubations in lab, Madison, WI, 2021-22

This dataset comprises soil-pyrogenic organic matter (PyOM) incubation experiments conducted in a laboratory setting. The focus was on measuring carbon dioxide (CO2) flux emitted in the headspace of incubation jars over a one-month period. The aim was to estimate carbon mineralization from two different carbon fractions in PyOM, as well as carbon mineralization from soil organic carbon (SOC). Bacterial and fungal community profiles were tracked at various time points during the incubation to observe changes in the overall community and specific responders to PyOM. LC-MS analysis of soil samples was also performed to monitor changes in the soil's chemical composition.The provided dataset includes processed CO2 flux measurements obtained from the Picarro-multiplexer measurement setup for each sample incubation jar. This data is available in the "CO2Flux.csv" file. Additionally, the dataset includes CO2 flux data partitioned to estimate mean carbon mineralization from the two PyOM carbon fractions, bulk PyOM, and SOC. The cumulative carbon mineralization data is provided in "cml_respired.csv," while the rate of carbon mineralization data is in "rate_respired.csv." The file "cml_respired_lastcyc.csv" contains cumulative carbon mineralization data for the PyOM carbon fractions, bulk PyOM, and SOC throughout the entire incubation period.For the LC-MS analysis, the raw data can be found in the "20220721_NZ_PeakHeights_FinalAnalysis.csv" file, while sample metadata is provided in "20220721_NZpilot_Metadata.csv". Processed LC-MS data is available in the file "20220721_NZ_FINAL_msdat_sub3.csv". Lastly, the dataset includes the relative abundance data for bacterial genera that exhibited a significant positive response to the addition of PyOM produced at 350 degrees Celsius, which can be found in "resp_genera_350.csv".

54 ENVIRONMENTAL SCIENCES↗

COMBLE Radar b1 Processing: Corrections, Calibrations, and Processing Report

The U.S. Department of Energy’s (DOE) Atmospheric Radiation Measurement (ARM) user facility recently concluded its Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE), with its campaign emphasis on marine boundary-layer clouds and mixed-phase clouds during cold-air outbreaks. The COMBLE campaign featured the deployment of the first ARM Mobile Facility (AMF1) to northern Scandinavia (Andenes, Norway), including its standard complement of ARM cloud radars. In keeping with user demands stemming from the previous AMF Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina, a post-campaign radar mentor effort was initiated for COMBLE. This activity was intended to improve the usability of the ARM cloud radar data sets in response to overall demands for calibrated, corrected data sets for downstream studies and retrieval applications. In addition to the terrain complexities previously important to CACTI data sets (e.g., clutter designation and/or removal), COMBLE presented new challenges to existing ARM radar mentor capabilities. These included the extension of existing methodologies (e.g., relative calibration adjustment [RCA] target techniques) to frozen environments and the potential issues with their applicability when considering mixed-phase precipitation conditions. As in the previous CACTI documentation (Hardin et al. 2020), the overall calibration and conditioning process in ARM nomenclature is referred to as generating a “b1” datastream. For the radars, these “b1” standards refer to a datastream that has been calibrated (and cross-calibrated), with effort to deliver the highest-quality (well-characterized) data possible. The “b1” radar mentor reporting (this current document) is intended to detail (i) the status/quality of the original “a1” (raw) data sets during the COMBLE AMF campaign, (ii) the corrections and calibrations that are applied to generate the b1 datastreams available on ARM’s Data Discovery, and (iii) the details of the applied methods, e.g., how radar offset/calibration numbers were determined.

54 ENVIRONMENTAL SCIENCES↗

Data for: "Inorganic synthesis-structure maps in zeolites with machine learning..."

Repository containing all the data necessary to reproduce the paper "Inorganic synthesis-structure maps in zeolites with machine learning and crystallographic distances". This content includes: raw data for all results in the article; figures appearing in the main paper; and Jupyter notebooks and scripts containing the code for analyzing/reproducing the results.

Schwalbe Koda, Daniel↗

Full spectrum (350-2500 nm) leaf and canopy spectral reflectance, Seward Peninsula, Alaska, 2019

Full-range (350 - 2500 nm) leaf and canopy reflectance spectra of various Arctic tundra ecosystem endmembers, including species-level leaf reflectance, canopy-scale species endmember spectra, plot-scale spectra, and transect spectra. These data were collected in support of ongoing NASA ABoVE AVIRIS data synthesis work. The datasets were collected at the NGEE-Arctic watershed sites, Kougarok MM64 and MM80, and Teller MM27 on the Seward Peninsula, Alaska. The data were collected in July 2019 using a full-range Spectra Vista Corporation (SVC) HR-1024i spectroradiometer. Leaf-level spectra were collected with the original SVC leaf clip/plant probe connected to the spectrometer through a 1.15 meter long fiber optic cable, while canopy-scale reflectance was collected with an 8-degree field-of-view (FOV) foreoptic lens. All spectral measurements were collected as calibrated surface radiance and converted to surface reflectance using a 99.99% reflective Spectralon white reference standard. For those canopy spectra collected with associated functional trait data, the FOV of the instrument was positioned to include the same leaves harvested for functional trait measurements, including leaf mass per area (LMA) and foliar carbon and nitrogen content (see leaf trait data package for these data). Canopy spectra were obtained for 23 different species, and leaf spectra for 8 species. This data package includes processed canopy and leaf spectra (*.csv), processed dGPS locations (*.csv and *.kmz), digital photographs of spectral targets (*.jpg) and raw data from spectroradiometer and dGPS instruments (compressed as tar.gz). Metadata files include data descriptions (_dd.csv) for tabular data and a key to species symbols used in data files. All included files are listed and described in NGA212_flmd.csv.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Hydropower Biological Evaluation Toolset Best Practice Guide

Field studies using live fish are necessary for the evaluation of turbine biological performance, but they cannot determine the specific hydraulic conditions or physical stresses experienced by the fish, the locations where deleterious conditions occur, or the specific causes of the biological response. Using the Sensor Fish (SF) sensing technology, this deficiency can be overcome because the SF can be released independently or concurrently with live fish directly into operating infrastructure, and it takes high-frequency measurements of hydraulic conditions such as pressure, acceleration, and rotation acting on a body in situ during downstream passage. The Hydropower Biological Evaluation Tools (HBET; Hou et al. 2018) software package, developed by Pacific Northwest National Laboratory (PNNL), is designed to assemble, organize, and process data collected by the PNNL-developed SF and by live fish. HBET was developed specifically to design SF field studies, process the raw data, and analyze the processed data efficiently and scientifically. Its objectives are to facilitate SF studies focused on characterizing hydraulic conditions and to apply SF data for evaluating the impacts on fish from passage through hydro-structures. HBET allows users to design new studies, analyze data, perform statistical analyses, and evaluate predicted biological responses. It can be used by researchers, turbine designers, hydropower operators, and regulators to evaluate hydro-structures to enhance environmental sustainability in a cost-effective manner.

13 HYDRO ENERGY↗

Vanderbilt CMS Heavy-Ion Tier-2 Facility (Final Report)

The CMS experiment at the LHC at CERN in Geneva, Switzerland spends a portion of its running time colliding heavy ions. These heavy ion collisions are a key probe of a state of matter known as a Quark Gluon Plasma, which is a state of matter found in the very early universe 10s of microseconds after the big bang. These collision data are recorded by the CMS experiment and then undergo several stages of refinement to extract physics properties of the collisions from the raw data recorded by the various detector subsystems in CMS. In addition, copies of these datasets are stored on tape archival storage for data preservation. This award funds the deployment, maintenance, and operation of a computing facility at Vanderbilt University which processes, stores, and analyzes these collision data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗