SEARCH · Engineering Papers

Results for “model skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2025

5-minute Wind Power Data based on WFIP2 WRF Simulation

The second Wind Forecast Improvement Project (WFIP2) was a public-private partnership funded by the U.S. Department of Energy and NOAA, aimed at enhancing the forecast skill of numerical weather prediction models for turbine-height winds in regions with complex terrain. An 18-month Weather Research and Forecasting (WRF) model simulation was conducted over the Pacific Northwest, with model outputs validated against observational data collected during WFIP2. Simulated wind speeds were used to estimate wind power generation using reV (the Renewable Energy Potential Model developed by NREL) at ten wind project sites. Two sets of results were produced: one using wind speeds extracted from the model grid cell at the project centroid, and another using wind speeds from the actual turbine locations. For each dataset, power output was calculated using both actual turbine-specific power curves and nine generic power curves to convert wind speed into power.

17 WIND ENERGY↗

DOE OSTI2025

Evaluation of the 2022 West Nile virus forecasting challenge, USA

Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

OpenStudio®-MCP [SWR-26-035]

OpenStudio®-MCP is a Model Context Protocol (MCP) server that lets AI assistants perform building energy modeling through natural language. Rather than requiring users to learn the OpenStudio® SDK, EnergyPlus® scripting, or Ruby/Python automation, the server translates conversational requests into sequences of tool calls that create models, design HVAC systems, run simulations, and extract results — all within a single chat session. The server's 124 tools are organized into a skills architecture where each skill encapsulates a domain of building energy modeling (envelope, HVAC, loads, weather, simulation, results) behind typed, LLM-friendly interfaces. High-leverage operations like applying ASHRAE 90.1 baseline systems or generating standards-compliant typical buildings are exposed as single tool calls that internally wire dozens of OpenStudio® objects. Bundled measures from ComStock™ and Openstudio® -common-measures-gem are wrapped with dedicated tools and typed arguments rather than exposed through a generic measure interface, so AI models get consistent, error-resistant recipes without needing to discover measure arguments at runtime. A key design decision is structured results extraction: six SQL-based tools return surgical ~300–1,000 token responses (end-use breakdowns, envelope summaries, HVAC sizing, timeseries data) instead of requiring the AI to parse ~100K-token raw HTML reports, making iterative design exploration practical within context window limits. The codebase is designed as a reference implementation — explicit, well-commented, and modular — so that other simulation engines (EnergyPlus® standalone, TRNSYS, DOE-2) can use it as a template for building their own MCP servers.

Ball, Brian [National Laboratory of the Rockies (N↗

DOE OSTI2026

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Semantic schemas like Haystack 4, Brick and ASHRAE standard 223 enable the structured, standardized, and machine-readable representation of building data, facilitating interoperability, data integration, and advanced analytics. However, extracting information from these models requires specialized expertise in SPARQL and other programming languages, skills that are not commonly found among building professionals. Recent advancements in Large Language Models (LLMs), such as ChatGPT, enable the construction of queries using natural language, making it easier for individuals to interact with these systems in a manner that resembles everyday speech. However, these methods have not yet been tested on building semantic ontologies. This paper introduces a novel workflow and tool for enabling users to ask questions about a specific building's data, using natural language and receive answers automatically generated by GPT-4o. Our approach integrates semantic ontologies with advanced LLM capabilities to automate three critical steps: (1) generating SPARQL queries to retrieve time series references from ontological models, (2) extracting the corresponding time series data from the Building Automation System, and (3) performing computations and visualizations tailored to the user's query. The proposed method simplifies access to BAS data, allowing both domain experts and non-specialists to conduct sophisticated analyses without needing extensive technical knowledge of semantic web technologies. By demonstrating this pipeline, we facilitate more accessible and scalable data-driven decision-making in building operations and management.

Mulayim, Ozan Baris↗

DOE OSTI2025

Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models

Mulayim, Ozan Baris↗

DOE OSTI2025

Performance Evaluation of Weather@home2 Simulations over West African Region

Weather and climate forecasting, using climate models, have become essential tools and life-savers in the West African region; in spite of the fact that climate models do not fully comply with attributes of forecast qualities—RASAP: reliability, association, skill, accuracy, and precision. The objective of this paper is to quantitatively evaluate, in comparison to CRU and ERA5 datasets, the RASAP compliance-level of the weather@home2 modeling system (w@h2). Findings from some statistical evaluations show that, to a moderately significant extent, w@h2 model provides useful information during the monsoon seasons; skills to capture the Little Dry Season over the Guinea zone; predictive skills for the onset season; ability to reproduce all the annual characteristics of the surface maximum air temperature over the region; as well as skill to detect heat waves that usually ravage West Africa during the boreal spring. The model displays traces of attributes that are needed for seasonal climate predictions and applications. Deficiencies in the quantitative reproducibility point to the facts that the model does provide a reliability akin to that of regional climate models. This paper further furnishes a prospective user with information on whether the model might be “useful or not” for a particular application.

West Africa↗

DOE OSTI2026

Computational Modeling of Atmospheric Processes at Texas Southern University

Texas Southern University (TSU) is strengthening its research program in atmospheric chemistry and physics with a climate science emphasis by leveraging partnerships with the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Facility, Brookhaven National Laboratory (BNL), and the Tracking Aerosol Convection Interactions ExpeRiment (TRACER). This RDPP-supported program focuses on secondary organic aerosols (SOAs) and reactive atmospheric species that influence cloud formation, precipitation processes, and radiative forcing. SOAs play a critical role in cloud microphysics and Earth’s energy balance, yet the chemical and physical mechanisms governing SOA–cloud interactions remain a significant source of uncertainty in predictive climate models. Through computational modeling, observational data analysis, and national laboratory collaboration, this program develops a skilled cohort of students trained in atmospheric science, environmental data analysis, and climate-relevant modeling. These research experiences build technical competencies that are transferable to careers in government laboratories, academia, and industry. By engaging students from historically underrepresented communities in high-impact climate research, TSU expands participation in the atmospheric sciences workforce while contributing meaningful scientific insights to DOE-supported ARM research activities. This partnership strengthens national capacity in climate science and supports the development of the next generation of atmospheric researchers.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021–22 and 2022–23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021–22 and 12 out of 18 models in 2022–23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021–22 and the 5th most accurate in the 2022–23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

EnergyPlus-MCP: A model-context-protocol server for ai-driven building energy modeling

Traditional building energy modeling with the EnergyPlus building performance simulation engine requires domain expertise, programming skills, and intensive manual efforts limiting its effective adoption. This paper introduces EnergyPlus-MCP, the first open-source Model Context Protocol (MCP) server specifically designed for EnergyPlus simulation workflows, establishing a new foundational infrastructure for AI-driven building energy modeling. The MCP server implements a layered architecture with 35 specialized tools spanning model management, editing and analysis, HVAC and other systems configuration inspection, and simulation execution, enabling Large Language Models to interact with EnergyPlus through conversational interfaces. The server addresses critical workflow barriers by automating model validation, streamlining energy efficiency measures modification, and providing intelligent output management with interactive visualization. Through practical demonstrations using a multi-zone building retrofit analysis, we show how the EnergyPlus-MCP server significantly reduces manual efforts while maintaining full simulation rigor. By providing accessible natural language interfaces to sophisticated building energy analysis, this approach enables scalable deployment of simulation expertise across public and private organizations, educational institutions, and research teams, fundamentally transforming traditional building energy modeling practices.

AI↗

DOE OSTI2025

Selection of Global Climate Model Data for Downscaling With Generative Machine Learning and Use in the Power Planning for Alignment of Climate and Energy Systems Project

The range of results from climate models and scenarios is important to the understanding of uncertainty in power planning analysis. A U.S. Department of Energy-funded analytic project called Power Planning for Alignment of Climate and Energy Systems is developing data and analytic methods to reflect the effects of climate change on key variables for power system planning, as part of the Grid Modernization Lab Consortium. This project will select and prepare global climate model results for use in power system planning models. A related report (Evaluation of Global Climate Models for Use in Energy Analysis) assesses the performance of various global climate models from the Coupled Model Intercomparison Project Phase 6 data archive for their historical skill with respect to energy system performance and for their future projections under multiple climate change scenarios. Building from that report, we describe the selection of a climate scenario (Shared Socioeconomic Pathway [SSP] 2-4.5) and five climate models: TaiESM1, EC-Earth3-CC, GFDL-CM4, EC-Earth3-Veg, and MPI-ESM1-2-HR. We describe the model selection criteria, which were based on the quality of the match between model results under historical conditions and on the representation of the range of future values for several variables. These results will be downscaled via an open-source generative machine learning method called Super-Resolution for Renewable Energy Resource Data with Climate Change Impacts.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2025

The NASA ACTIVATE Mission

The NASA Aerosol Cloud Meteorology Interactions over the Western Atlantic Experiment (ACTIVATE) conducted 162 joint flights with two aircraft over the northwest Atlantic to study aerosol–cloud interactions (ACIs), which represent the largest uncertainty in estimating total anthropogenic radiative forcing. The combination of a high-flying King Air and low-flying HU-25 Falcon, equipped with remote sensing and in situ instruments, characterized trace gases, aerosol particles, clouds, and meteorological variables with data collected nearly simultaneously below, within, and above marine boundary layer (MBL) clouds. Flights spanning warm and cold seasons across 3 years (2020–22) provided a broad range of conditions associated with aerosol particles, cloud properties (including particle size and phase), and meteorology, ideally suited for robust ACI calculations and assessing how well models simulate a wide range of MBL clouds from stratiform to cumulus. ACTIVATE data suggest that drivers of cloud droplet number concentration N d , including aerosol particles and MBL dynamics, vary between winter and summer months with a stronger potential to convert aerosol particles into cloud droplets in winter. Models of varying complexity not only highlight some skills in simulating winter and summer cloud types but also identify challenges that still need to be addressed such as treatment of turbulence, wet scavenging, and mesoscale organization. Remote sensing advances range from new retrieval methods for N d , cloud phase classification, vertically resolved aerosol and cloud condensation nuclei number concentration, and ocean surface wind speed. This work describes these scientific and technological advances along with efforts in outreach and open data science.

aerosol indirect effect↗

DOE OSTI2025

ClimGen: Learning the Forcing-Response Relationship in Climate System

Solar Radiation Management (SRM) is emerging as a potential geoengineering strategy to address the anthropogenic impact on climate, but its effective implementation requires an iterative and large ensemble of highly accurate and efficient climate projections. Traditional climate projections rely on executing computationally demanding and time-consuming numerical climate models. Recent advances in machine learning (ML) aim to enhance these approaches by emulating traditional methods. In this work, we propose a novel framework for directly learning the relationship between solar radiation flux at the top of the atmosphere and the corresponding surface temperature response. To evaluate the feasibility of this direct ML-based projection, we developed a dataset using an intermediate complexity model, incorporating a comprehensive suite of different forcing patterns and evaluation metrics to rigorously assess the ML model’s performance. We introduce a Conditional Denoising Diffusion Probabilistic Model (cDDPM) for this task, which demonstrates encouraging skill in representing climate statistics under previously unseen forcing patterns. This approach provides a promising pathway for direct climate projections by accurately learning the forcing-response relationship, with a wide range of applications in impact mitigation, emissions policy design, and SRM strategies.

Chen, Tse-Chun [BATTELLE (PACIFIC NW LAB)] (ORCID:↗

DOE OSTI2024

Wide range of possible trajectories of North Atlantic climate in a warming world

Decadal variability in the North Atlantic Ocean impacts regional and global climate, yet changes in internal decadal variability under anthropogenic radiative forcing remain largely unexplored. Here we use the Community Earth System Model 2 Large Ensemble under historical and the Shared Socioeconomic Pathway 3-7.0 future radiative forcing scenarios and show that the ensemble spread in northern North Atlantic sea surface temperature (SST) more than doubles during the mid-twenty-first century, highlighting an exceptionally wide range of possible climate states. Furthermore, there are strikingly distinct trajectories in these SSTs, arising from differences in the North Atlantic deep convection among ensemble members starting by 2030. We propose that these are stochastically triggered and subsequently amplified by positive feedbacks involving coupled ocean-atmosphere-sea ice interactions. Freshwater forcing associated with global warming seems necessary for activating these feedbacks, accentuating the impact of external forcing on internal variability. Further investigation on seven additional large ensembles affirms the robustness of our findings. By monitoring these mechanisms in real time and extending dynamical model predictions after positive feedbacks activate, we may achieve skillful long-lead North Atlantic decadal predictions that are effective for multiple decades.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Evaluation of Global Climate Models for Use in Energy Analysis

The interplay between energy, climate, and weather is becoming more complex due to increasing contributions of renewable energy generation, energy storage, electrified end uses, and the increasing frequency of extreme weather events. Energy system analyses commonly rely on meteorological inputs to estimate renewable energy generation and energy demand; however, these inputs rarely represent the estimated impacts of future climate change. Climate models and publicly available climate change datasets can be used for this purpose, but the selection of inputs from the myriad of available models and datasets is a nuanced and subjective process. In this work, we assess datasets from various global climate models (GCMs) from the Coupled Model Intercomparison Project Phase 6 (CMIP6). We present evaluations of their skills with respect to the historical climate and comparisons of their future projections of climate change for two climate change scenarios. We present the results for different climatic and energy system regions and include interactive figures in the accompanying software repository. Previous work has presented similar GCM evaluations, but none have presented variables and metrics specifically intended for comprehensive energy systems analysis including impacts on energy demand, thermal cooling, hydropower, water availability, solar energy generation, and wind energy generation. We focus on GCM output meteorological variables that directly affect these energy system components including the representation of extreme values that can drive grid resilience events. The objective of this work is not to recommend the best climate model and dataset for a given analysis, but instead to provide a reference to facilitate the selection of climate models and scenarios in subsequent work.

14 SOLAR ENERGY↗

DOE OSTI2025

Data-driven global ocean modeling for seasonal to decadal prediction

Accurate modeling of ocean dynamics is crucial for enhancing our understanding of complex ocean circulation processes, predicting climate variability, and tackling challenges posed by climate change. Although great efforts have been made to improve traditional numerical models, predicting global ocean variability over multiyear scales remains challenging. Here, we propose ORCA-DL (Oceanic Reliable foreCAst via Deep Learning), a data-driven three-dimensional ocean model for seasonal to decadal prediction of global ocean dynamics. ORCA-DL accurately simulates the three-dimensional structure of global ocean dynamics with high physical consistency and outperforms state-of-the-art numerical models in capturing extreme events, including El Niño–Southern Oscillation and upper ocean heat waves. Moreover, ORCA-DL stably emulates ocean dynamics at decadal timescales, demonstrating its potential even for skillful decadal predictions and climate projections. Our results demonstrate the high potential of data-driven models for providing efficient and accurate global ocean modeling and prediction.

Science & Technology - Other Topics↗

DOE OSTI2024

A Visual Analytic Platform for Interactive Validation of Human Mobility Simulations

Human mobility insights guide domain experts in an array of decisions, including critical infrastructure design, disaster response, epidemic modeling, national security, and policy making. Due to the inherent noise and privacy concerns in real-world individual-level mobility data, it is often preferred to leverage simulators that generate synthetic mobility data instead. However, it is critical to inspect and validate the output of such simulators to ensure the synthetic data is aligned with the characteristics of the population and the area of interest known to domain experts. While there exist many quantitative approaches for validating synthetic data, we argue it is also important to also validate such data qualitatively to capture aspects that are known to domain experts but difficult to quantify. In this work, we demonstrate a visual analytic platform that empowers domain experts to interact with their simulation outputs along spatial and temporal dimensions. By augmenting automated techniques and human skills, our visual analytic platform is a step towards interactive capabilities for model steering and quality control of mobility simulators.

Monadjemi, Shayan↗

DOE OSTI2024

Generative Artificial Intelligence Tools for Red Teams

This document analyzes the role of Generative Artificial Intelligence (GenAI) tools in cybersecurity, particularly for red teaming. While GenAI accelerates initial security assessments, its effectiveness wanes with complexity, necessitating experienced assessors. The review critiques marketing claims, highlights ethical concerns regarding uncensored models for cybercrime, and advocates for a robust defense strategy supported by skilled professionals.

97 MATHEMATICS AND COMPUTING↗