Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data-driven services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Use of Open Networks and Delay-Tolerant Protocol to Decrease WAN Latency of EOS near Real-Time Data

Since 1999, NASA's Earth Observing System Data Operations System (EDOS) project at Goddard Space Flight Center (GSFC) has provided high-rate data capture, level zero processing, and product distribution services for a majority of NASA's EOS (Earth Observing System) high-rate missions, including Terra, Aqua, Aura, ICESat, EO-1, SMAP, and OCO-2. EDOS high-rate science and engineering (150-300 Mbps) data-driven capture systems are deployed at 7 worldwide ground stations which are connected via both private (closed) and public (open) wide area networks (WANs) to the centralized EDOS Level Zero Processing Facility (LZPF) located at GSFC, where the data is processed and Level 0 products are distributed to users worldwide. All data transferred over the open networks to GSFC traverse an IPSec tunnel, providing the same level of security as a VPN connection. EDOS produces both time-based and near real-time products (session-based). Near real-time data products are produced from a single ground station contact; time-based products are produced from multiple ground station contacts. EDOS is the primary supplier of EOS Level 0 data to the NASA near real-time user community known as the Land, Atmosphere Near real-time Capability for EOS (LANCE). For the past few years, EDOS has streamlined its systems to reduce WAN latency for near real-time data delivery, including implementing Quality of Service (QoS), expanding closed network bandwidth, adding open network connections with more bandwidth, and implementing a delay-tolerant protocol to mitigate long round-trip times to remote ground stations.

Delay-Tolerant Protocol↗
NASA NTRSDate not supplied

Usability of Pre-flight Planning Interfaces for Supplemental Data Service Provider Tools to Support Uncrewed Aircraft System Traffic Management

Small uncrewed aircraft systems (sUASs) operate in low-altitude, uncontrolled airspace – where support services for their operators (UASOs) are not currently provided. NASA’s System-Wide Safety (SWS) project is identifying the potential risks and hazards to sUAS operations to provide, inform, and improve the designs of In-time Aviation Safety Management Systems (IASMS). The IASMS will include a suite of data-driven tools that compile and analyze data collected from aviation systems and environmental sources to predict hazards, and provide information to allow operators to mitigate these risks (Young et al., 2020). These risk and hazard services can be run and displayed to operators on graphical user interfaces (GUIs), as they relate to a vehicle(s)’ route of flight. These interfaces offer both a means to present hazard service output and offer an opportunity to test user understanding of the information, user decision making, and the best ways to present such data to an operator. Based on these future technologies and intended missions, it is important to investigate interface requirements and evaluate how operators might use these tools. Presenting salient and meaningful risk assessment information to operators is necessary to increase situation awareness and ultimately safety. Building on previous research (Feldman et al., 2022), a usability study comparing two GUIs was conducted to explore how individuals interacted with different styles of information displays. A series of pre-flight hazard and risk-assessment tasks were developed to evaluate participant performance using the Supplemental Data Service Provider Consolidated Dashboard and the Human Automation Team Interface System interfaces. Participants were trained to use both GUIs and their performance was analysed across different scenarios involving multiple sUASs. Performance on simple tasks and the System Usability Scale scores were reported by Feldman et al., 2023. Additional analyses and evaluations on more complex tasks (e.g., risk assessment, prioritization), workload and response times are examined in this paper.

sUAV interfaces↗
NASA NTRS2016

GES DISC Datalist Enables Easy Data Selection for Natural Phenomena Studies

In order to investigate and assess natural hazards such as tropical storms, winter storms, volcanic eruptions, floods, and drought in a timely manner, the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been developing an efficient data search and access service. Called Datalist, this service enables users to acquire their data of interest all at once, with minimum effort. A Datalistis a virtual collection of predefined or user-defined data variables from one or more archived data sets. Datalistsare more than just data. Datalistseffectively provide users with a sophisticated integrated data and services package, including metadata, citation, documentation, visualization, and data-specific services (e.g., subset and OPeNDAP), all available from one-stop shopping. The predefined Datalists, created by the experienced GES DISC science support team, should save a significant amount of time that users would otherwise have to spend. The Datalistservice is an extension of the new GES DISC website, which is completely data-driven. A Datalist, also known as data bundle, is treated just as any other data set. Being a virtual collection, a Datalistrequires no extra storage space.

Earth events↗
NASA NTRSDate not supplied

Hourly Stream Heights – A Short-term Deep Learning Prediction Model

River flooding can have a detrimental impact on a community by causing loss of life, loss or damage to property, and damage to infrastructure. Having the capability to forecast flooding events can prevent the loss of life and mitigate damage to property. A programmatic, data-driven approach using deep learning to forecast a stream’s gauge height every 6 hours has been developed by NASA’s Short-term Prediction Research and Transition Center (SPoRT) and is currently in operation. Based on our end user engagement and feedback, this medium-term product has been successfully adopted by several National Weather Service (NWS) Forecast Offices and River Forecast Centers (RFC). SPoRT is currently researching and developing a short-term hourly deep learning model which will be useful in forecasting flash flood events.

Michael Antia↗
DOE OSTI2026

Artificial Intelligence-Driven Management of Sustainable Energy Resources: Visibility, Operation, and Control

The rapid global transition toward sustainable energy resources (SERs) is reshaping how modern power systems are observed, optimized, and controlled. While SERs have significantly advanced decarbonization, their weather dependence, variability, and inverter-dominated characteristics challenge traditional, centralized, and deterministic grid operation. At the same time, the proliferation of high-resolution data from inverters, smart meters, and sensors offers unprecedented visibility into system dynamics. Yet, it also exceeds the analytical capability of conventional model-based approaches. Artificial intelligence (AI) provides a new foundation for addressing these challenges by bridging physical laws with data-driven learning, enabling accurate state awareness, adaptive operation, and coordinated control across distributed assets. This article examines how AI transforms the management of SER-rich power systems along three critical dimensions: 1) enhancing visibility by inferring behind-the-meter (BTM) activities, assessing SER flexibility, and reconstructing system states from sparse or noisy measurements; 2) improving operation through AI-enhanced SER service provision, volt/var control (VVC), and dynamic operating envelopes (DOE) for efficiency and security; and 3) advancing control by embedding learning-based intelligence into inverter coordination, voltage and frequency regulation, and long-term dispatch. Together, these developments reveal how AI can convert the variability of SERs from an operational challenge into a source of flexibility, resilience, and intelligence, paving the way toward sustainable, adaptive, and self-optimizing power systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

FIRE: A Failure-Adaptive RL Framework for Edge Computing Migrations

In edge computing, users' service profiles are migrated between edge servers due to user mobility. Reinforcement Learning (RL) frameworks have been proposed to do so, often trained on simulated data. However, existing RL frameworks overlook occasional server failures, which although rare, impact latency-sensitive applications like AR/VR and real- time obstacle detection. These rare failures, being not adequately represented in historical training data, pose a challenge for data-driven RL algorithms. We introduce FIRE, a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment. We propose FIRE-ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function. FIRE considers delay, migration, failure, and backup placement costs across individual and shared service profiles. We prove FIRE-ImRE's boundedness and convergence to optimality. Next, we introduce novel deep Q-learning (FIRE-ImDQL) and actor critic (FIRE-ImACRE) versions of our algorithm to enhance scalability. Here, we extend our framework to accommodate users with varying risk tolerances of rare failure events. Through trace-driven experiments, we show that FIRE reduces edge computing costs compared to vanilla RL and the greedy baseline in the event of failures.

Edge computing↗
DOE OSTI2025

Unified 0.25-degree gridded infrastructure-critical extreme weather for the United States from 1979 to 2100

Extreme weather events can severely disrupt critical infrastructure, triggering cascading effects on power, transportation, and essential services. However, standard weather and climate datasets often lack specialized variables necessary for hazard assessments. We present a unified dataset of infrastructure-critical weather and climate variables across the United States at 0.25° resolution, covering daily or sub-daily intervals from 1979 to 2100. The dataset includes temperature, dew point, wind gusts, precipitation partitioned by rain, snow, and freezing rain or ice pellets, lightning, and wildfire metrics. Historical conditions (1979-2023) are synthesized from observations and reanalysis products, while future projections are derived from 14 CMIP6 global climate models (historical, SSP245, and SSP585 experiments). Physically based and data-driven methods are used to estimate variables not directly provided by existing models. By integrating these variables into a single unified dataset, we enable consistent, high-resolution assessments of weather-related infrastructure risks across past and future periods, supporting wide-ranging applications in energy, transportation, water resources, emergency management, and beyond.

Climate and Earth system modelling↗
DOE OSTI2025

Development of a Discrepancy Checker for the Digital Twin in a Supervisory Control System for a Thermal Energy Delivery System

Defined as a virtual representation of a physical object, process, or service, and used to support real-world decision-making, a digital twin (DT) can be utilized to combine classical and novel frameworks in sensors, state predictions, and multi-input/multi-output systems, and to enable optimal autonomous operations. However, a DT’s usefulness largely depends on its ability to adequately mirror the state of its physical counterpart, and this adequacy should be reflected by the level of uncertainty in the underlying simulation models when estimating and predicting quantities of interest (QOIs). Moreover, simulation models in a DT may involve multiple fidelities of representations—ranging from physics-based models to data-driven ones—but classical uncertainty quantification (UQ) methods struggle to handle numerous uncertainty sources, nor are they designed for real-time applications. This work presents a UQ-based discrepancy checking and diagnosis tool for a DT-based supervisory control system applied to a thermal energy delivery system (TEDS) at Idaho National Laboratory. The discrepancy checker was developed using metadata from an automated DT development process, and these metadata included different combinations of physical model forms and model parameters, training data and hyperparameters for surrogate models, and design parameters for supervisory control systems. Next, correlations between the uncertainty results and the metadata were established and then applied to the DT operations. The discrepancy checker evaluates the discrepancies between model predictions from virtual and sensor measurements and backtraces them to the corresponding major sources of uncertainty. The discrepancy checker showed reasonable performance in detecting discrepancies and diagnosing sources of uncertainty in testing scenarios.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Machine Learning Based Metamodel for Faster Life Cycle Assessment of Large Portfolio of Buildings

Managing a large portfolio of buildings involves decisions on reuse, retrofit, renovation, rehabilitation, and new construction, influenced by trade-offs between performance metrics such as cost, time, and operational flexibility over the building's life cycle. Traditional life cycle assessment tools for evaluating these metrics can be labor- and compute-intensive, requiring extensive data and modeling for each building. Metamodels (or surrogate models) using machine learning have been explored as faster alternatives, but training these models has been hindered by the limited availability of comprehensive data on key life cycle metrics. Recent advancements in machine learning, particularly deep learning techniques like zero-shot and few-shot learning, allow models to learn from sparse or limited data. We propose a machine learning-based metamodel that leverages these techniques for rapid estimation of key building life cycle metrics. This presentation will cover the model architecture, data collection, training, and validation processes, along with an ongoing case study applied to a large portfolio of buildings. We will discuss the model's performance in terms of accuracy, compute time, limitations, and its potential for expanding to additional life cycle metrics. This data-driven approach offers a promising direction for the rapid evaluation of large building portfolios.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Data-Driven Modeling of High-Resolution Residential Load Profiles Using Low-Resolution Smart Meter Measurements

Accurate and high-resolution residential load profiles are essential for power system modeling, demand response planning, and effective grid operation. As the energy sector moves towards a more actively managed distribution system, the ability to understand residential energy consumption at a minute-by-minute scale becomes increasingly critical. High-resolution load profiles provide key insights into demand patterns and user behavior, enabling grid operators to design more effective energy solutions; however, residential load measurements in the field are typically recorded at low resolutions, such as 15-60 minutes, which makes it hard to study the characteristics of different residential customers. This paper addresses these challenges by introducing a data-driven approach to generate realistic, high-resolution residential load profiles based on lowre-solution measurements and weather information. The proposed method retains the key features of the actual residential load measurements while offering appliance-level energy consumption details for each residential building. The results demonstrate the effectiveness of the proposed load profile generator, proving its capability to support utilities in optimizing residential energy management and ensuring a more reliable and resilient grid.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Privacy First Path Analysis using Clickstream Data

In the modern digital economy, data-driven decision making is crucial for effectively meeting the ever-evolving demands of consumer engagement and satisfaction. Clickstream data has become invaluable for understanding customer behavior, yet concerns over privacy and security persist, especially with some internet service providers profiting from its sale. This article introduces an innovative methodology that blends experiential learning with advanced cryptographic techniques, including differential privacy and graph analytics. The core objective of this methodology is to estimate Customer Lifetime Value (CLV) by analyzing clickstream data, achieving an average prediction accuracy of 92.4% in user engagement levels while ensuring user anonymity through Recency, Frequency, and Monetary (RFM) analysis. Our study introduces the concept of a “data depositor” and a privacy manager, employing the composition theorem to merge non-adaptive queries effectively. Privacy budgets (? = 1.0, d = 10-5), sensitivity-specific techniques, and data partitioning were applied. Randomization and noise addition protect data integrity, with special handling for categorical values. This approach, differing from prior studies, offers a 12.6% improvement in privacy-preserving targeting accuracy while maintaining strict confidentiality, presenting a novel path forward in data-driven decision-making.

Frequency and Monetary (RFM) analysis↗
DOE OSTI2025

Computational materials reliability assessment of hydrogen fueled gas turbine power generation engines

The use of blended fuel sources in land based gas turbine engines drives variations in the resulting operational profile (temperatures and pressures) which can impact engine reliability. Furthermore, variability in the manufacture of components affects the resulting microstructure which directly impacts material performance and reliability. Currently, data-driven models are typically used for maintaining and inspecting fleets of engines. Without explicitly capturing material and operational sources of variability conservatism must be used in developing component-level reliability models. Therefore, there exists an opportunity to use information from materials-scale physics models to better inform reliability modeling and reduce conservatism; the impact is more cost-efficient operation and maintenance of current and future fleets. Specifically, this work establishes a computational framework for evaluating the probabilistic high temperature creep performance of hot-section Ni-based superalloys where uncertainty comes from both microstructural and operational variability. A novel high-fidelity physics model which phenomenologically captures grain-boundary sensitive phenomena has been established. A probabilistic calibration procedure was used to calibrate the model and capture uncertainty in the parameterized model coefficients. A design of experiments methodology was established for identifying informative microstructural digital representations for suitable for forward model evaluation. Results show that training a machine-learning surrogate using this design criteria outperforms random selection of microstructural representations. Finally, two surrogate models were developed: (1) a deterministic surrogate model which predicts the local field response given microstructure, constitutive model parameters, and operating conditions (stress, temperature) and (2) a probabilistic model, where uncertainty comes from constitutive law uncertainty, built using denoising diffusion probabilistic models which samples responses given (1) microstructure and (2) operating conditions. These surrogate models enable partner Siemens Energy to rapidly perform UQ analysis specific to creep deformation across a range of microstructures and operating conditions. The impact is that these ML and physics codes can be used to establish more advanced reliability models for the inspection, servicing, and maintenance of land based gas turbine engines.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Exploring Wildfire & Energy data toward State Prioritization Index (WESPI)

Energy infrastructure can both induce and suffer risks from wildfires ranging from direct damage to energy assets such as substations and power lines to Public Safety Power Shutoffs. Recent wildfire events underscore the need for data-driven approaches that help states and utilities proactively plan for wildfire risk. Existing national tools such as Federal Emergency Management Agency (FEMA)’s National Risk Index (NRI) are valuable for community hazard planning. However, they are less suited for energy infrastructure, as they emphasize population and building exposure rather than system vulnerabilities. In this paper, we explore relationships between energy and wildfire data and present a Wildfire-Energy State Prioritization Index (WESPI). Our methodology combines data from the US Forest Service’s Fire Simulation (FSIM) dataset with energy resilience metrics, historical fire incidents, and geospatial data on transmission lines and fire stations. Correlation analyses suggest that FSIM burn probability is more strongly associated with power outage metrics (ρ = 0.32) than NRI wildfire frequency, and counties with a greater density of fire stations experience more frequent, but less intense wildfires. We further leverage data for burn probability, transmission line density, and fire station density to develop a Wildfire-Energy State Prioritization Index (WESPI) to highlight counties where wildfire hazard, infrastructure exposure, and limited suppression capacity converge. The index provides a consistent, scalable framework for state energy offices and utilities to screen counties for vegetation management, optimization of outage management system deployment, and to inform wildfire mitigation plans.

Critical infrastructure↗
DOE OSTI2025

Network Slicing for Federated Learning in Operational Technology Environment

Industrial Control Systems (ICS) and Supervisory Control and Data Acquisition (SCADA) environments are essential to modern infrastructure, facing challenges in ensuring low-latency, high-throughput communication while mitigating cyber threats. This paper presents a framework integrating Federated Learning (FL) and network slicing with Quality of Service (QoS) to enable real-time monitoring without disrupting OT operations. Leveraging digital twin technology and Network Function Virtualization (NFV), the architecture supports predictive analytics and Industry 4.0 requirements. FL facilitates decentralized model training, preserving data privacy and scalability, though it introduces potential throughput constraints. Network slicing addresses this by creating dedicated virtualized segments optimized for performance and security. Advanced fault tolerance at the container and instance levels enhances system reliability. The proposed architecture ensures high throughput, low latency, and secure orchestration for real-time anomaly detection in OT networks. Performance evaluations validate its efficiency in throughput, deployment, and learning accuracy, providing a robust foundation for future ICS automation and data-driven decision-making.

Delgado, Brian G. Rodiles [University of Texas at ↗
DOE OSTI2026

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Data-Driven Approach to Real-World Degradation of Backsheets

The objectives of this project are as follows: • The population behavior of fielded modules in various conditions of use • Predictions of materials in specific climatic zones • Understanding of a module’s local environment in the field on its degradation It aims to understand how and what backsheet materials of photovoltaic modules degrade in the real world field. • Field Survey Protocol This project is started from the protocol, because all the data, information, domain knowledge are from experience of the real world field surveys, which is based on the protocol. The Protocol is explored from the experience of the field survey observations. With the increasing of the field surveys, it is refined for three versions, which are Task 1.0 (Section 3.1), Task 6.0 (Section 3.6), Task 10.0 (Section 3.10) respectively. It includes a document and a training video, which is able to direct other teams to follow the same procedure with the sites surveyed during this project. The documents include the detailed information but is not limited to the terminology definition, instruments SOP, preparation items for the surveys, form for the data collections, the order of the information collection. The final version of the protocol can be found at Appendix A, see Section 5. Additional, It can also be found at Open Science Framework (OSF), see Section 3.18 for detail. • Written Waiver Request Before staring the field surveys, the request of the waiver for international surveys is completed, because the limitation of climate zone in the United States, see Appendix B in Section 6 for the request documents. Unfortunately, only 1 international site from Taiwan, China can be finished, due to the COVID-19. • Field Survey According to the protocol we built in Section 3.1, 3.6 and 3.10, 41 sites have been surveyed across seven different climate zones (Cfa, Csa, Csb, BSk, Dfa, Dfb, Am). A variety of materials, including Polyethylene Naphthalate (PEN), Polyethylene Terephthalate (PET), Polyvinyl Fluoride (PVF), Polyvinylidene Fluoride (PVDF), Acrylic PVDF, Fluoroethylene Vinyl Ether (FEVE), and Glass, were identified. These sites are located in various states including California, South Carolina, New Mexico, Maryland, Ohio, Tennessee, Florida, Massachusetts, Illinois, Minnesota, Oregon, Colorado, and Taiwan, Republic of China. The ages of the sites ranged from 2 - 38 years in service and the field size varied from 1 MW - 25 MW. All requirements for the modeling have been satisfied. Some observations like ’Edge Effect’ for the rows and Junction box heating will also be a useful knowledge to build the model. Section 3.7 provides detailed information on the sites visited during this reporting period.

14 SOLAR ENERGY↗
NASA NTRS2008

The Heliophysics Data Environment Today

Driven by the nature of the research questions now most critical to further progress in heliophysics science, data-driven research has evolved from a model once centered on individual instrument Principal investigator groups and a circle of immediate collaborators into a more inclusive and open environment where data gathered ay great public cost must then be findable and useable throughout the broad national and international research community. In this paper and as an introduction to this special session, we will draw a picture of existing and evolving resources throughout the heliophyscs community, the capabilities and data now available to end users, and the relationships and complementarity of different elements in the environment today. We will cite the relative roles of mission and instrument data centers and resident archives, multi-mission data centers, and the growing importance of virtual discipline observatories and cross-cutting services including the evolution of a common data dictionary. We will briefly summarize our view of the most important challenges still faced by users and providers, and our vision in ow the efforts today can evolve into a more and more enabling data framework for the global research community to tap the widest range of existing missions and their data to address a full range of critical science questions from the scale of microphysics to the heliospheric system as a whole.

Fung, Shing F.↗
NASA NTRS2007

Simple, Script-Based Science Processing Archive

The Simple, Scalable, Script-based Science Processing (S4P) Archive (S4PA) is a disk-based archival system for remote sensing data. It is based on the data-driven framework of S4P and is used for data transfer, data preprocessing, metadata generation, data archive, and data distribution. New data are automatically detected by the system. S4P provides services such as data access control, data subscription, metadata publication, data replication, and data recovery. It comprises scripts that control the data flow. The system detects the availability of data on an FTP (file transfer protocol) server, initiates data transfer, preprocesses data if necessary, and archives it on readily available disk drives with FTP and HTTP (Hypertext Transfer Protocol) access, allowing instantaneous data access. There are options for plug-ins for data preprocessing before storage. Publication of metadata to external applications such as the Earth Observing System Clearinghouse (ECHO) is also supported. S4PA includes a graphical user interface for monitoring the system operation and a tool for deploying the system. To ensure reliability, S4P continuously checks stored data for integrity, Further reliability is provided by tape backups of disks made once a disk partition is full and closed. The system is designed for low maintenance, requiring minimal operator oversight.

Lynnes, Christopher↗