Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “transformer based forecasting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Snowmass 2021 Cosmic Frontier White Paper: Cosmology with Millimeter-Wave Line Intensity Mapping

Next-generation tests of fundamental physics and cosmology using large scale structure require measurements over large volumes of the Universe, including high redshifts inaccessible to present-day surveys. Line intensity mapping, an emerging technique that detects the integrated emission of atomic and molecular lines without resolving sources, can efficiently map cosmic structure over a wide range of redshifts. Observations at millimeter wavelengths detect far-IR emission lines such as CO/[CII], and take advantage of observational and analysis techniques developed by CMB experiments. These measurements can provide constraints with unprecedented precision on the physics of inflation, neutrino masses, light relativistic species, dark energy and modified gravity, and dark matter, among many other science goals. In this white paper we forecast the sensitivity requirements for future ground-based mm-wave intensity mapping experiments to enable transformational cosmological constraints. We outline a staged experimental program to steadily improve sensitivity, and describe the necessary investments in developing detector technology and analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Safe Deep Reinforcement Learning for Active Distribution System Model Predictive Control with EVs and DERs

The temporal and spatial mismatch between PV generation and electric vehicle (EV) charging and discharging may cause voltage violations in active distribution networks. Despite the widespread use of deep reinforcement learning (DRL) in power system optimization and control, it lacks guarantees on constraint satisfaction during both training and deployment. This paper proposes a Lagrangian-based safe DRL approach for model predictive control (MPC) of active distribution systems with large-scale integration of PVs, EVs, and energy storage systems (ESSs). A Transformer-LSTM time-series model is proposed to forecast EV charging demand, which is then formulated as a constraint to ensure charging requirements are met. Using this prediction, a Lagrangian-based safe soft actor-critic (SAC) framework is developed for real-time control in a three-phase unbalanced distribution system, enforcing voltage safety constraints while optimizing the cumulative net reward. By integrating the forecasting model with multi-period constraints, the proposed framework jointly coordinates PV systems, EV charging and discharging, and ESS scheduling within the MPC horizon. Numerical experiments on a modified IEEE 123-bus system with real-world data show that, under a high PV penetration scenario, the proposed method increases the net reward by 30.74% and reduces average voltage violations from 0.0011 p.u. to 0.0002 p.u. compared with standard SAC. Compared with the optimal power flow (OPF) approach, it achieves similar voltage security while yielding lower line losses. It also maintains real-time control capability, reducing operation latency to 53.21 ms per 15-minute control interval. The proposed method remains effective under varying PV/EV penetrations and load conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2005

Contour Error Map Algorithm

The contour error map (CEM) algorithm and the software that implements the algorithm are means of quantifying correlations between sets of time-varying data that are binarized and registered on spatial grids. The present version of the software is intended for use in evaluating numerical weather forecasts against observational sea-breeze data. In cases in which observational data come from off-grid stations, it is necessary to preprocess the observational data to transform them into gridded data. First, the wind direction is gridded and binarized so that D(i,j;n) is the input to CEM based on forecast data and d(i,j;n) is the input to CEM based on gridded observational data. Here, i and j are spatial indices representing 1.25-km intervals along the west-to-east and south-to-north directions, respectively; and n is a time index representing 5-minute intervals. A binary value of D or d = 0 corresponds to an offshore wind, whereas a value of D or d = 1 corresponds to an onshore wind. CEM includes two notable subalgorithms: One identifies and verifies sea-breeze boundaries; the other, which can be invoked optionally, performs an image-erosion function for the purpose of attempting to eliminate river-breeze contributions in the wind fields.

Merceret, Francis↗
DOE OSTI2025

A cross-dimensional analysis of data-driven short-term load forecasting methods with large-scale smart meter data

Electricity load forecasting is essential to utility operation and power grid stability. A wide spectrum of data-driven methods, ranging from linear regression models to more recent deep learning models have been adopted to forecast electric load over the years. However, there still lacks a holistic evaluation of the applicability of conventional statistical and machine learning based algorithms with respect to different temporal and spatial scopes, computational requirements, and sensitivity of model-tuning. Enabled by a large-scale electricity load profile dataset of over 40,000 residential customers in a utility region, we conducted a cross-dimensional analysis of data-driven load forecasting methods. Three regression-based and seven deep learning algorithms with different model configurations were evaluated in terms of their overall and peak load prediction accuracy, and training burdens, across spatial aggregation levels ranging from the transformer, feeder, substation, to neighborhood. We found, first, the load forecasting accuracy is constrained by a predictability boundary, influenced by the forecasting horizon and spatial aggregation level. Specifically, RandomForest, XGBoost, TFT, TSMixer, and TiDE models achieved less than 10 % prediction error for up to 96-h ahead forecasting for district, substation, and feeder levels, while other models struggle at long-horizon predictions; Second, for winter and summer peak load dates, most models were able to predict the peak demand timing within ± 1 h, but the prediction percentage error varied by models, with TFT and TiDE models being the top performers; Third, models with similar prediction accuracy can differ in training burden by an order of magnitude. Therefore, choosing model configurations that balance prediction performance and computational resource is an important practical consideration for large-scale deployment of the machine learning based load forecasting. The outcome of this study can guide researchers and practitioners to choose the proper load forecasting algorithms based on their problem scope, required accuracy, and available resources. The predictability boundary can serve as a benchmark for electricity load forecasting problems with new algorithms and datasets.

Li, Han↗
DOE OSTI2025

Geothermal well testing pressure prediction by using a hybrid transformer model system: FORGE well use case

Geothermal has huge potential to become an indispensable component in achieving the goal of sustainable energy economy, given its capability to provide consistent baseload power to the electric grid. Injection tests are crucial in geothermal energy system as they naturally help to evaluate reservoir properties, understand fluid flow and even enhance reservoir performance. In this research, we developed a hybrid model system that integrates machine learning (ML) regression, a physics-based mathematical model, and transformer deep learning. Trained and validated using FORGE injection test dataset, this system can forecast the pressure variations both upward and downward over time. The pressure prediction achieved prediction accuracy within 3-6% variance of true pressure values. The system can significantly save time and reduce costs by testing only a few cycles and then using model predictions for further analysis, instead of conducting additional real injection cycle tests. The developed model system also holds promise for designing injection test processes and maintaining well production in geothermal energy. Presented at the IMAGE ‘25 Conference led by Shell.

FORGE↗
NASA NTRS2008

Radiometric Modeling and Calibration of the Geostationary Imaging Fourier Transform Spectrometer (GIFTS)Ground Based Measurement Experiment

The ultimate remote sensing benefits of the high resolution Infrared radiance spectrometers will be realized with their geostationary satellite implementation in the form of imaging spectrometers. This will enable dynamic features of the atmosphere s thermodynamic fields and pollutant and greenhouse gas constituents to be observed for revolutionary improvements in weather forecasts and more accurate air quality and climate predictions. As an important step toward realizing this application objective, the Geostationary Imaging Fourier Transform Spectrometer (GIFTS) Engineering Demonstration Unit (EDU) was successfully developed under the NASA New Millennium Program, 2000-2006. The GIFTS-EDU instrument employs three focal plane arrays (FPAs), which gather measurements across the long-wave IR (LWIR), short/mid-wave IR (SMWIR), and visible spectral bands. The GIFTS calibration is achieved using internal blackbody calibration references at ambient (260 K) and hot (286 K) temperatures. In this paper, we introduce a refined calibration technique that utilizes Principle Component (PC) analysis to compensate for instrument distortions and artifacts, therefore, enhancing the absolute calibration accuracy. This method is applied to data collected during the GIFTS Ground Based Measurement (GBM) experiment, together with simultaneous observations by the accurately calibrated AERI (Atmospheric Emitted Radiance Interferometer), both simultaneously zenith viewing the sky through the same external scene mirror at ten-minute intervals throughout a cloudless day at Logan Utah on September 13, 2006. The accurately calibrated GIFTS radiances are produced using the first four PC scores in the GIFTS-AERI regression model. Temperature and moisture profiles retrieved from the PC-calibrated GIFTS radiances are verified against radiosonde measurements collected throughout the GIFTS sky measurement period. Using the GIFTS GBM calibration model, we compute the calibrated radiances from data collected during the moon tracking and viewing experiment events. From which, we derive the lunar surface temperature and emissivity associated with the moon viewing measurements.

Tian, Jialin↗
NASA NTRS2018

Sub-Seasonal Forecasting of the Stratospheric Wave Events, Sudden Stratospheric Warmings, and Their Influence on the Troposphere

Stratospheric wave events and major sudden stratospheric warming (SSW) events are well captured in high-resolution global forecasts out to 10 days. Tropospheric influences of SSW include statistically significant shifts in the storm tracks and associated surface temperature and precipitation pattern changes. Since these events can in turn influence the troposphere on time scales of 30-60 days the ability to predict these events and the subsequent long-term response on time scales beyond 10 days is of interest. Here we examine the prediction of stratospheric wave events and their evolution using the NASA GMAO (Global Modeling and Assimilation Office) Sub-Seasonal to Seasonal (S2S) system. This is a recently released subseasonal to seasonal forecast system, GEOS-S2S version 2.1. Compared to GMAO's previous system, the new version runs at higher atmospheric resolution (approximately 1/2 degree globally), contains a substantially improved model of the cryosphere, includes additional interactive aerosol model components, and the ocean data assimilation system has been replaced with a Local Ensemble Transform Kalman Filter. Results are based on a comprehensive series of hindcasts starting from the year 2000. They show that while the S2S system is not as accurate at 10 days in forecasting major SSW events as the NASA GMAO Forward Processing system, it can usefully predict stratospheric anomalies out to 20 days and the subsequent stratospheric/tropospheric evolution beyond 30 days.

Coy, L.↗
DOE OSTI2025

SPECTER: an instrument concept for CMB spectral distortion measurements with enhanced sensitivity

Deviations of the cosmic microwave background (CMB) energy spectrum from a perfect blackbody uniquely probe a wide range of physics, ranging from fundamental physics in the primordial Universe (μ-distortion) to late-time baryonic feedback processes (y-distortion). While the y-distortion can be detected with a moderate increase in sensitivity over that of COBE/FIRAS, the ΛCDM-predicted μ-distortion is roughly two orders of magnitude smaller and requires substantial improvements, with foregrounds presenting a serious obstacle. Within the standard model, the dominant contribution to μ arises from energy injected via Silk damping, yielding sensitivity to the primordial power spectrum at wavenumbers k ≈ 1-10 4 Mpc -1 . Here, we present a new instrument concept, SPECTER, with the goal of robustly detecting μ. The instrument technology is similar to that of LiteBIRD, but with an absolute temperature calibration system. Using a Fisher approach, we optimize the instrument's configuration to target μ while marginalizing over foreground contaminants. Unlike Fourier-transform-spectrometer-based designs, the specific bands and their individual sensitivities can be independently set in this instrument, allowing significant flexibility. We forecast SPECTER to observe the ΛCDM-predicted μ-distortion at ≈ 5σ (10σ) assuming an observation time of 1 (4) year(s) (corresponding to mission duration of 2 (8) years), after foreground marginalization. Our optimized configuration includes 16 bands spanning 1–2000 GHz with ∼degree-scale angular resolution at ∼ 150 GHz and 1100 total detectors. SPECTER will additionally measure the y-distortion at sub-percent precision and its relativistic correction at percent-level precision, yielding tight constraints on the total thermal energy and mean temperature of ionized gas.

CMBR experiments↗
DOE OSTI2023

Attention for Causal Relationship Discovery from Biological Neural Dynamics

This paper explores the potential of the transformer models for learning Granger causality in networks with complex nonlinear dynamics at every node, as in neurobiological and biophysical networks. Our study primarily focuses on a proof-of-concept investigation based on simulated neural dynamics, for which the ground-truth causality is known through the underlying connectivity matrix. For transformer models trained to forecast neuronal population dynamics, we show that the cross-attention module effectively captures the causal relationship among neurons, with an accuracy equal to or superior to that of the most popular Granger causality discovery method. While we acknowledge that real-world neurobiology data will bring further challenges, including dynamic connectivity and unobserved variability, this research offers an encouraging preliminary glimpse into the utility of the transformer model for causal representation learning in neuroscience.

Lu, Ziyu↗
DOE OSTI2025

A Neural Optimizer With Decision-Focused Learning for Optimal Energy Storage Operation

Here, this article introduces a neural optimizer-based framework for optimizing battery energy storage system (BESS) control for grid services, including demand charge and energy cost reduction. By leveraging decision-focused learning (DFL), the proposed framework ensures seamless integration and adaptation, significantly enhancing control performance. A patch time-series transformer is employed for peak load forecasting, incorporating aleatoric uncertainty quantification to account for forecasting uncertainties within the decision-making process. The framework utilizes a solver-in-the-loop approach to generate optimal BESS actions, which are then used to train the neural optimizer-based agent. By co-optimizing both BESS operational modes and output power within the NN, the system achieves improved performance and robustness. After initial training, the forecasting and control models are jointly fine-tuned to account for forecasting errors, further improving decision precision and efficiency through DFL. Case studies are performed to validate the performance of the framework using multiple real-world datasets, demonstrating superior performance in monthly peak load forecasting compared to state-of-the-art models. In addition, the results are compared against existing decision-making approaches. The results demonstrate a reduction in monthly peak forecasting error by approximately 15% across various performance measures and achieve an optimization gap for BESS operation that is about three times smaller compared to existing methods.

Kim, Hyeonjin [Pacific Northwest National Laborato↗
NASA NTRS2021

Expanding the application of soil moisture monitoring systems through regression-based transformation

Relative to other geophysical variables, soil moisture (SM) estimates derived from land surface models (LSMs) and land data assimilation systems (LDAS) are difficult to transfer between platforms and applications. This difficulty stems from the highly model-dependent nature of LSM SM estimates and differences in the vertical support of discretized SM values. As a result, operational SM estimates generated by one LSM (or LDAS) cannot generally be directly applied to a hydrologic monitoring or forecast system designed around a second LSM. This lack of transferability is particularly problematic for LDAS applications, where the time, expertise, and computational resources required to generate an operational LDAS analysis cannot be practically duplicated for every LSM-specific application. Here, we develop a set of simple regression tools for translating SM estimates between LSMs and multiple LDAS analyses. Results demonstrate that simple multivariate linear regression - utilizing independent variables based on multi-layer and temporally lagged SM estimates - can significantly improve upon baseline transformation approaches using direct percentile matching. The proposed regression approaches are effective for both the LSM-to-LSM and LDAS-to-LDAS transformation of multi-layer SM percentiles. Application of this approach will expand the utility of existing, high-quality (but LSM-specific) operational sources of SM information like the NASA Soil Moisture Active Passive Level-4 Soil Moisture product.

Soil Moisture↗
DOE OSTI2024

Short-Term Probabilistic Solar Forecasting via Reinforcement Learning over ECMWF

In this paper, we present an innovative reinforcement learning approach for short-term solar forecasting, leveraging data from the European Centre for Medium-Range Weather Forecasts (ECMWF). The methodology begins with the application of the System Advisor Model (SAM) to transform various ECMWF numerical weather prediction members into predictive photovoltaic power generation. To enhance the precision of deterministic forecasting, we introduce a dynamic model selection algorithm based on Q-learning. This algorithm dynamically identifies and utilizes the most accurate ensemble member for forecasting purposes. Furthermore, we employ a support vector regression surrogate model with a Gaussian distribution to generate probabilistic forecasts, providing a holistic view of solar energy generation uncertainty. To expedite the training process and make it more practical for real-world applications, we integrate a rolling update workflow. This innovative workflow reduces the training period from months to a mere 19 days, making our method highly efficient. Numerical results of the case study show that in comparison to benchmark models, the proposed method improves the deterministic and probabilistic solar forecasting accuracy by up to 40.84% and 48.42%, respectively.

ensemble forecasting↗
DOE OSTI2026

Autoregressive long-horizon prediction of plasma edge dynamics *

Accurate modeling of scrape-off layer (SOL) and divertor-edge dynamics is vital for designing plasma-facing components in fusion devices. High-fidelity edge fluid/neutral codes such as SOLPS-ITER capture SOL physics with high accuracy, but their computational cost limits broad parameter scans and long transient studies. We present transformer-based, autoregressive surrogates for efficient prediction of 2D, time-dependent plasma edge state fields. Trained on SOLPS-ITER spatiotemporal data for the KSTAR tokamak, the surrogates forecast electron temperature, electron density, and radiated power over extended horizons. We evaluate model variants trained with increasing autoregressive horizons (1–100 steps) on short- and long-horizon prediction tasks. Longer-horizon training systematically improves rollout stability and mitigates error accumulation, enabling stable predictions over hundreds to thousands of steps and reproducing key dynamical features such as the motion of high-radiation regions. Measured end-to-end wall-clock times show the surrogate is orders of magnitude faster than SOLPS-ITER, enabling rapid parameter exploration. Prediction accuracy degrades when the surrogate enters physical regimes not represented in the training dataset, motivating future work on data enrichment and physics-informed constraints. Overall, this approach provides a fast, accurate surrogate for computationally intensive plasma edge simulations, supporting rapid scenario exploration, control-oriented studies, and progress toward real-time applications in fusion devices.

autoregressive deep learning↗
NASA NTRS2006

AIRS Retrieval Validation During the EAQUATE

Atmospheric and surface thermodynamic parameters retrieved with advanced hyperspectral remote sensors of Earth observing satellites are critical for weather prediction and scientific research. The retrieval algorithms and retrieved parameters from satellite sounders must be validated to demonstrate the capability and accuracy of both observation and data processing systems. The European AQUA Thermodynamic Experiment (EAQUATE) was conducted mainly for validation of the Atmospheric InfraRed Sounder (AIRS) on the AQUA satellite, but also for assessment of validation systems of both ground-based and aircraft-based instruments which will be used for other satellite systems such as the Infrared Atmospheric Sounding Interferometer (IASI) on the European MetOp satellite, the Cross-track Infrared Sounder (CrIS) from the NPOESS Preparatory Project and the following NPOESS series of satellites. Detailed inter-comparisons were conducted and presented using different retrieval methodologies: measurements from airborne ultraspectral Fourier transform spectrometers, aircraft in-situ instruments, dedicated dropsondes and radiosondes, and ground based Raman Lidar, as well as from the European Center for Medium range Weather Forecasting (ECMWF) modeled thermal structures. The results of this study not only illustrate the quality of the measurements and retrieval products but also demonstrate the capability of these validation systems which are put in place to validate current and future hyperspectral sounding instruments and their scientific products.

Zhou, Daniel K.↗
NASA NTRS2020

Using a Simple Water Balance Framework to Quantify the Impact of Soil Moisture Initialization on Subseasonal Evapotranspiration and Air Temperature Forecasts

Past studies have shown that accurate soil moisture initialization can contribute significant skill to near-surface air temperature (T2M) forecasts at subseasonal leads. The mechanisms by which soil moisture contributes such skill are examined here with a simple water balance-based model that captures the essence of soil moisture behavior in a state-of-the-art subseasonal-to-seasonal (S2S) forecasting system. The simple model successfully transforms initial soil moisture contents into average “forecasted” ET values at 16-30 day lead that agree well, during summer, with the values forecasted by the full NASA GEOS S2S system, indicating that soil moisture initialization dominates over forecasted meteorology in determining ET fluxes at subseasonal leads. When the simple model’s ET anomalies are interpreted in terms of T2M anomalies, a similar conclusion is reached for T2M: soil moisture initialization explains much (about 50% in the eastern half of the continental US) of the T2M anomalies produced by the full GEOS S2S system at 16-30 day lead, and the T2M forecasts produced by the simple model capture about half of the skill attained by the full system. The simple model’s framework is particularly conducive to an analysis of uncertainty in forecasts. Drier soils are generally found to induce larger uncertainty in ET (and thus T2M) forecasts, a result linked to the functional form relating ET to soil moisture in the simple model and verified by an analysis of the ensemble spreads within the forecasts produced by the full GEOS S2S system

Randal D Koster↗
DOE OSTI2022

Enhancing Risk Analysis of Accidental Release Using CFD Modeling and Machine Learning

This project report presents the results from the AU31 project that is entitled “Enhancing Risk Analysis of Accidental Release Using CFD Modeling and Machine Learning” performed between FY2021 and FY2022. The technical objective of this project is to enhance risk analysis of accidental release of radioactive materials and provide novel risk forecasting and capabilities for assessing unplanned hazardous chemical releases for the U.S. Department of Energy (DOE). The approach includes a cross-platform finite element analysis of airborne chemical release, machine learning (ML) for simulation and forecasting, and a big data approach to data management, transformation and analysis capable of handling petabyte-scale unstructured and structured data. The outcome of these findings were assembled and accessible as a final output via an intuitive web hosted geospatial-based user interface for demonstration and general dissemination. These new results also provide a mechanism of combining this knowledge in a risk analysis framework tool. The geospatial analysis and output can utilize 5 years of qualified meteorological data to illustrate projected plume footprints that are generated from computational aspects of this study. Specifically, we show the development of a geospatial risk analysis capabilities to respond to and plan for accidental release of radioactive materials and to provide a risk forecasting tool for assessing hazardous chemical releases for the DOE. The focus of this effort is germane to common practices in risk analysis; and it is highly relevant to the timely and transparent release of information related to materials at risk including hazardous materials and chemical vapors and rapid assessment of the potential impacts on the workforce at active sites across the DOE complex.

42 ENGINEERING↗
DOE OSTI2025

Sensitivity-based voltage constraints for optimal power flow in low-voltage distribution feeders

The optimal power flow (OPF) problem for distribution systems can include network details down to the low-voltage (LV) points of interconnection of individual customers. This paper addresses the implementation of voltage magnitude constraints, and sets forth a practicable approach for capturing the effects on voltage from the switching behavior of loads (e.g., heat pumps, air conditioners, water heaters, or pool pumps) and from the variability of renewable generation (e.g., rooftop solar). The proposed method adjusts the OPF voltage constraints based on forecasts of load and generation upper and lower bounds, in conjunction with sensitivity factors derived from the power flow equations. An illustrative OPF formulation is also provided, which incorporates transformer models that include core loss. We demonstrate that accurate modeling of these LV network components is critical to avoid voltage violations at customer points of interconnection. Furthermore, the ideas are validated through numerical case studies on a realistic distribution feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2013

The Relation Between Atmospheric Humidity and Temperature Trends for Stratospheric Water

We analyze the relation between atmospheric temperature and water vapor-a fundamental component of the global climate system-for stratospheric water vapor (SWV). We compare measurements of SWV (and methane where available) over the period 1980-2011 from NOAA balloon-borne frostpoint hygrometer (NOAA-FPH), SAGE II, Halogen Occultation Experiment (HALOE), Microwave Limb Sounder (MLS)/Aura, and Atmospheric Chemistry Experiment Fourier Transform Spectrometer (ACE-FTS) to model predictions based on troposphere-to-stratosphere transport from ERA-Interim, and temperatures from ERA-Interim, Modern Era Retrospective-Analysis (MERRA), Climate Forecast System Reanalysis (CFSR), Radiosonde Atmospheric Temperature Products for Assessing Climate (RATPAC), HadAT2, and RICHv1.5. All model predictions are dry biased. The interannual anomalies of the model predictions show periods of fairly regular oscillations, alternating with more quiescent periods and a few large-amplitude oscillations. They all agree well (correlation coefficients 0.9 and larger) with observations for higherfrequency variations (periods up to 2-3 years). Differences between SWV observations, and temperature data, respectively, render analysis of the model minus observation residual difficult. However, we find fairly well-defined periods of drifts in the residuals. For the 1980s, model predictions differ most, and only the calculation with ERA-Interim temperatures is roughly within observational uncertainties. All model predictions show a drying relative to HALOE in the 1990s, followed by a moistening in the early 2000s. Drifts to NOAA-FPH are similar (but stronger), whereas no drift is present against SAGE II. As a result, the model calculations have a less pronounced drop in SWV in 2000 than HALOE. From the mid-2000s onward, models and observations agree reasonably, and some differences can be traced to problems in the temperature data. These results indicate that both SWV and temperature data may still suffer from artifacts that need to be resolved in order to answer the question whether the large-scale flow and temperature field is sufficient to explain water entering the stratosphere.

Fueglistaler, S.↗