Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Raw Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Cross-Market Price Difference Forecast Using Deep Learning for Electricity Markets

Price forecasting is in the center of decision making in electricity markets. Many researches have been done in forecasting energy prices while little research has been reported on forecasting price difference between day-ahead and realtime markets due to its high volatility, which however plays a critical role in virtual trading. To this end, this paper takes the first attempt to employ novel deep learning architecture with Bidirectional Long-Short Term Memory (LSTM) units to forecast the price difference between day-ahead and real-time markets for the same node. The raw data is collected from PJM market, processed and fed into the proposed network. The Root Mean Squared Error (RMSE) and customized performance metric are used to evaluate the performance of the proposed method. Case studies show that it outperforms the traditional statistical models like ARIMA, and machine learning models like XGBoost and SVR methods in both RMSE and the capability of forecasting the sign of price difference. Additionally, to cross-market price difference forecast, the proposed approach has the potential to be applied to solve other forecasting problems such as price spread forecast in DA market for Financial Transmission Right (FTR) trading purpose.

DA/RT price difference↗
DOE OSTI2021

A Minimal Information Set To Enable Verifiable Theoretical Battery Research

Batteries are an enabling technology for addressing sustainability through the electrification of various forms of transportation (1) and grid storage. (2) Batteries are truly multi-scale, multi-physics devices, and accordingly various theoretical descriptions exist to understand their behavior (3-5) ranging from atomistic details to techno-economic trends. As we explore advanced battery chemistries (6,7) or previously inaccessible aspects of existing ones, (8-10) new theories are required to drive decisions. (11-13) The decisions are influenced by the limitations of the underlying theory. Advanced theories used to understand battery phenomena are complicated and require substantial effort to reproduce. However, such constraints should not limit the insights from these theories. We can strive to make the theoretical research verifiable such that any battery stakeholder can assess the veracity of new theories, sophisticated simulations or elaborate analyses. We distinguish verifiability, which amounts to “Can I trust the results, conclusions and insights and identify the context where they are relevant?”, from reproducibility, which ensures “Would I get the same results if I followed the same steps?” With this motivation, we propose a checklist to guide future reports of theoretical battery research in Table 1. We hereafter discuss our thoughts leading to this and how it helps to consistently document necessary details while allowing complete freedom for creativity of individual researchers. Given the differences between experimental and theoretical studies, the proposed checklist differs from its experimental counterparts. (14,15) This checklist covers all flavors of theoretical battery research, ranging from atomic/molecular calculations (16-19) to mesoscale (20,21) and continuum-scale interactions, (9,22) and techno-economic analysis. (23,24) Finally, as more and more experimental studies analyze raw data, (25) we feel this checklist would be broadly relevant.

25 ENERGY STORAGE↗
DOE OSTI2026

Generative large language models for predictive maintenance planning

Maintenance planning and the generation of necessary components for tasks can prove time-consuming and complex. Automating the creation of recurring or similar tasks by leveraging previous planning packages and data, while uncovering insights to automate planning package generation, presents an opportunity to conserve valuable time and resources. This work aims to harness the textual and probabilistic capabilities of large language models (LLMs) to automate the generation of planning packages. Utilizing diverse data sources ranging from raw data to handwritten text, both singular and collaborative LLMs are trained and tested. Results demonstrate their capability to generate essential planning package components, effectively replicating the statistical patterns in the data. This demonstrates the use of these tools inside a digital asset for automated planning. This work outlines a methodology for constructing datasets, a training suite, and evaluation methods for LLM-based textual and conversational planning tools utilized in an asset digital twin. Results indicate that the fine-tuned models generate estimated planning information within the statistical ranges observed in real maintenance data. The models achieve high accuracy (>90%) in document question-answering and instruction generation tasks. Furthermore, the conversational retrieval-augmented generation (RAG) assistant system achieves 100% document retrieval accuracy, while conversational information capture exceeds 98% across the majority of work-package assistant modules.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Top-down proteomics

Proteoforms arising from posttranslational modifications, genetic polymorphisms, and RNA splice variants, play a pivotal role as the key drivers in biology. Thus, a comprehensive understanding of proteoforms is essential for unraveling the intricacies of biological systems and bridging the gap between genotype and phenotype. By analyzing whole proteins without digestion, top-down proteomics (TDP) provides a holistic view of the proteome and presents a next-generation approach for deciphering protein function, uncovering disease mechanisms, and advancing precision medicine. This Primer embarks on a journey into the world of TDP by encapsulating its historical context, underlying principles, recent advances, and an outlook on the future of TDP. The experimental section navigates instrumentation, sample preparation, intact protein separation, tandem mass spectrometry techniques, and data collection. Results decipher raw data, visualize intact protein spectra, unravel data analysis, and explain proteoform identification, characterization, and quantitation, as well as statistical analysis. Various applications of TDP spanning the human proteoform project, biomedical, biopharmaceutical, and clinical applications are described. These are complemented by discussions on measurement reproducibility, limitations, and a forward-looking perspective outlining uncharted waters where the field can advance, and potential exciting future applications of TDP.

Roberts, David S.↗
DOE OSTI2025

SALSA: a new versatile readout chip for MPGD detectors

The SALSA chip is a future readout ASIC foreseen for the MPGD detectors, developed in the framework of the EIC collider project, to equip the MPGD trackers of the EPIC experiment. It is designed to be versatile, to be adapted to other usages of MPGD detectors like TPC or photon detectors. It integrates a frontend block and an ADC for each of the 64 channels, associated to a configurable DSP processor meant to correct data and reduce the raw data flux to limit the output bandwidth. It will be compatible with the continuous readout foreseen for the EPIC DAQ, but will also work in a triggered environment. Several prototypes are already produced in order to qualify the different blocks of the chip, in particular the frontend, the ADC and the clock generation. The next 32-channel prototype is currently under development and is planed to be produced in 2025. In conclusion, the final prototype will be produced and tested from 2026 for a production of the SALSA chip at the horizon of 2027.

Data processing↗
DOE OSTI2026

DP-TwoLevel: two-stage gradient subspace learning for differentially private federated learning

Federated learning (FL) enables collaborative model training across distributed data sources without sharing raw data, but faces fundamental challenges in communication efficiency and privacy. Differentially private (DP) training mitigates information leakage but introduces noise that degrades model performance, especially in high-dimensional settings. We propose DP-TwoLevel, a hierarchical gradient projection method that improves utility under fixed DP constraints by exploiting low-dimensional structure in model updates. Our approach learns a two-level PCA-based representation of gradients and applies DP noise in a reduced-dimensional subspace, thereby lowering the effective noise magnitude while preserving dominant signal components. We evaluate the method across three datasets (MNIST, Fashion-MNIST, CIFAR-10) and three privacy regimes (ϵ∈0.5, 1.0, 2.0). Across nine experimental settings, DP-TwoLevel consistently outperforms DP-FedAvg, achieving an average accuracy improvement of 9.44%, with larger gains observed in lower ϵ(higher-noise) regimes (up to +22.31%). We further analyze scalability across models ranging from 100K to 1.49M parameters and identify a variance-based success criterion: performance remains strong when the projection preserves more than 75% of gradient variance, degrades in a marginal regime (65–75%), and fails below this threshold. Our results demonstrate that structure-aware dimensionality reduction can significantly improve the privacy–utility tradeoff in FL without modifying formal privacy guarantees. We also provide empirical evidence of scaling limitations for global projections and motivate per-layer extensions for larger models.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗
DOE OSTI2023

NSO Processing Scripts (NSO wind loading data set processing scripts) [SWR-23-86]

NREL conducted comprehensive field measurements of the atmospheric turbulent wind conditions and the resulting structural wind loads on parabolic troughs at the Nevada Solar One plant from October 2021 to June, 2023. The measurement set-up included meteorological masts and structural load sensors on four trough rows. Additionally, NREL commissioned a lidar scanning the horizontal plane over the trough field. The high-resolution data set, characterizing the complex flow field and resulting structural loads, is available at https://data.openei.org/submissions/5938. The processing routines published here were used to create the data from the instruments' raw data.

Egerer, Ulrike↗
DOE OSTI2026

Automated point dendrometer, soil moisture and temperature, and meteorological variables datasets, Oct 2024 – Nov 2025, G.A. Pearson Natural Area, Flagstaff, AZ, USA

This data package includes parsed, cleaned, and calibrated data from 48 TOMST automated point dendrometers, 48 TOMST 15 cm soil moisture sensors, and 12 TOMST 30 cm soil moisture sensors. The point dendrometers were cleaned with the “dendRoAnalyst” package in RStudio. The soil sensors were cleaned and calibrated for volumetric water content (VWC) with the “myClim” package in RStudio using the soil texture of the site (sandy clay loam). Additionally, this data package also includes raw data from 2 METER weather stations. Dendrometers and soil sensors have both their sensor ID, as well as the ID for the specific tree they were instrumented on at the G.A. Pearson Natural Area (GPNA) site and their experimental group. The purpose of these data is to understand how ponderosa pine trees in restored (thinned and burned) vs. unrestored (no treatment) areas are responding to drought and seasonal precipitation. These data use radial growth and soil moisture data to answer the following question: how are active season length, growth on different time scales (weekly, monthly, seasonally, and annually), growth during dry periods and after precipitation events, and environmental and biological drivers of radial growth different between restored versus unrestored areas?

Air temperature↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental Conditions, Utqiagvik (Barrow), Alaska, 2018

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 16 June - 24 September, 2018. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2019

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June – 25 September, 2019. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017–2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental Conditions, Utqiagvik (Barrow), Alaska, 2017

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 22 June - 17 September, 2017. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic Vegetation Warming Experiment data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Environmental conditions, Utqiagvik (Barrow), Alaska, 2021

Environmental conditions measured in five warming chambers and paired ambient control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska from 19 June - 17 September, 2021. These data were recorded in support of the Zero Power Warming (ZPW) vegetation warming experiment, a series of single season vegetation warming treatments conducted over four years from 2017-2021 (no experiment in 2020). Air temperature and humidity, infrared surface (canopy) temperature, soil temperature, soil moisture, NDVI (normalized difference vegetation index), PRI (photochemical reflectance index), solar radiation and chamber venting were recorded in each chamber at 1 minute intervals. Ambient air temperature, humidity, solar radiation and uplooking PRI and NDVI were measured at a centrally located meteorology station. Vapor pressure deficit (VPD) was calculated and included in the final processed data products. Data has undergone full QA/QC and is presented as 1 minute data, and hourly and daily aggregate data products. This data package includes unprocessed raw data (*.dat files), processed data (*.csv) and metadata including a full description of sensors, calculations and processing (*.csv, *.pdf). See related NGEE-Arctic "Vegetation Warming Experiment" data packages for leaf-level gas exchange and other leaf trait data; chamber, plot and landscape phenocamera images; thaw depth, and GPS locations of chambers and ambient plots. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County

The dataset contains hourly Anthropogenic heat (AH) from buildings in Los Angeles County, based on weather data from 2018. The hourly AH is aggregated at three spatial resolutions: 450m x 450m grid, 12km x 12km grid, and census tract. The AH is broken down into three components: building envelope surface convection, heating, ventilation, and air conditioning (HVAC) system heat release, and zone exfiltration and exhaust air heat loss. The dataset is created with the physics-based EnergyPlus building energy models to calculate individual buildings' AH considering WRF-UCM simulated microclimate conditions. Please refer to the paper "A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County" for more information about the data generation workflow and the data validation procedure. The data set contains two folders: the "output_data" folder holds the simulation results (EP_output and EP_output_csv), building metadata (building_metadata.geojson and building_metadata.csv), aggregated heat emission and energy consumption time-series data (hourly_heat_energy), and geographical data (geo_data) associated with the GEOID referenced in heat and energy consumption data. The "input_data" folder contains the raw data used to generate files in the "output_data" folder as well as data sets used in the validation. The code repository (https://github.com/IMMM-SFA/xu_etal_2022_sdata) holds the processing scripts for data curation, validation, and visualization.

Energy↗
NASA NTRS1982

Comparison of polar motion data from the 1980 project MERIT short campaign

The short campaign of MERIT (a program of international collaboration to Monitor Earth Rotation and Intercompare the Techniques of observation and analysis) was held during the three-month period August to October, 1980. Based on the analysis of polar motion behavior, the possibility of predicting polar motion for a long time interval (1-2 years in advance) with sufficient accuracy has been found. The predicted polar motion can be used for geodetic purposes. It appears that the accuracy of the prediction is high enough for any practical purpose which requires real time polar motion up to an accuracy of approximately 50 cm. This would include applications such as control of space probes. Attention is given to a MERIT data intercomparison based on raw data, and a MERIT data intercomparison after smoothing.

Mueller, I. I.↗
NASA NTRS1983

Amplitude calibration of spaceborne synthetic aperture radars

Problems encountered during attempts to calibrate SAR imagery, recent successful experiments conducted with SEASAT SAR data, and a proposed program for the calibration and validation of the radar imagery from the forthcoming SIR-B SAR are discussed. The SEASAT SAR data for 10 passes over Death valley, California, were processed with a modified digital correlator. The procedure included a preliminary screening of the data to check for raw data saturation, compensation of waveforms and estimation of the amplitude of the pilot tone. All data was normalized to this pilot tone signal to reduce the effects of variable gains in the data links and ground receivers. The digital correlation algorithm generated image data. Evaluation of 6 passes results in a maximum pass to pass gain variation of only 1.1 dB and a standard deviation amongst the passes of 0.35 dB.

Held, D. N.↗
NASA NTRS1983

Amplitude calibration of spaceborne synthetic aperture radars

Problems encountered during attempts to calibrate SAR imagery, recent successful experiments conducted with SEASAT SAR data, and a proposed program for the calibration and validation of the radar imagery from the forthcoming SIR-B SAR are discussed. The SEASAT SAR data for 10 passes over Death Valley, California, were processed with a modified digital correlator. The procedure included a preliminary screening of the data to check for raw data saturation, compensation of waveforms and estimation of the amplitude of the pilot tone. All data was normalized to this pilot tone signal to reduce the effects of variable gains in the data links and ground receivers. The digital correlation algorithm generated image data. Evaluation of 6 passes results in a maximum pass to pass gain variation of only 1.1 dB and a standard deviation amongst the passes of 0.35 dB. previously announced in STAR as N83-26215

Held, D. N.↗
NASA NTRS2013

Statistical Analyses of Raw Material Data for MTM45-1/CF7442A-36% RW: CMH Cure Cycle

This report describes statistical characterization of physical properties of the composite material system MTM45-1/CF7442A, which has been tested and is currently being considered for use on spacecraft structures. This composite system is made of 6K plain weave graphite fibers in a highly toughened resin system. This report summarizes the distribution types and statistical details of the tests and the conditions for the experimental data generated. These distributions will be used in multivariate regression analyses to help determine material and design allowables for similar material systems and to establish a procedure for other material systems. Additionally, these distributions will be used in future probabilistic analyses of spacecraft structures. The specific properties that are characterized are the ultimate strength, modulus, and Poisson􀂶s ratio by using a commercially available statistical package. Results are displayed using graphical and semigraphical methods and are included in the accompanying appendixes.

Coroneos, Rula↗