Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data flow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Modeling Grid Data Flows for Transmission and Distribution Operations: Review, Design, Next Steps

Operational scenarios of the power grids grow multifold to accommodate the diverse needs of both the utilities and end consumers, and the various other stakeholders in-between. To comprehensively model and apply analytics to support objectives and business functions of grid sectors, a reliable approach to characterize and design data flows is crucial. The flows bridge business functions with communications protocols, stakeholders such as the grid actors, and data interfaces comprising different data objects. Additionally, constraints applied to the flow such as cybersecurity, trust, privacy, and ownership among others intersect these entities, requiring the delineation of their interactions under different scenarios. This paper aims to not only highlight relevant research in the space of grid data flows, but also proposes, for the transmission-distribution sector, a novel modeling approach that marries the aforementioned entities: objectives, business functions, data interfaces, communication protocols, data stakeholders, and flow constraints. It elaborates on the design philosophy and the significance of each entity within the model and applies it to an example function of fault location, isolation and service restoration (FLISR). Finally, the next steps to extend the application of this data flow model for other practical operational scenarios are discussed.

Sundararajan, Aditya [ORNL] (ORCID:000000033577854↗
DOE OSTI2020

Efficient Binary Static Code Data Flow Analysis Using Unsupervised Learning

The ever increasing need to ensure that code is reliably, efficiently and safely constructed has fueled the evolution of popular static binary code analysis tools. In identifying potential coding flaws in binaries, tools such as IDA Pro are used to disassemble the binaries into an opcode/assembly language format in support of manual static code analysis. Because of the highly manual and resource intensive nature involved with analyzing large binaries, the probability of overlooking potential coding irregularities and inefficiencies is quite high. In this paper, a light-weight data, unsupervised data flow methodology is described which uses highly-correlated data flow graph (CDFGs) to identify coding irregularities such that analysis time and required computing resources are minimized. Such analysis accuracy and efficiency gains are achieved by using a combination of graph analysis and unsupervised machine learning techniques which allows an analyst to focus on the most statistically significant flow patterns while performing binary static code analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Environmental Data Flow Six Sigma Process Improvement Savings Overview

An overview of the Environmental Data Flow Six Sigma improvement project covers LANL’s environmental data processing following receipt from the analytical laboratories. The Six Sigma project identified thirty-three process improvements, many of which focused on cutting costs or reducing the time it took to deliver data to clients.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

S AP F LOWER : an automated tool for sap flow data preprocessing, gap-filling, and analysis using deep learning

Sap flow, a critical process in plant water use and ecosystem water cycles, is often measured using thermal dissipation probes (TDP) due to their ease of installation and continuous data collection. However, sap flow data frequently include noise, outliers, and gaps, creating challenges for analysis and requiring substantial manual processing. We developed S AP F LOWER , a tool that automates data preprocessing, model training, gap-filling, sapwood area scaling and modeling, and water use analysis. It integrates autocleaning, machine learning and deep learning models (e.g. random forest, Gaussian process regression, long short-term memory (LSTM), bidirectional LSTM (BiLSTM)), and efficient workflows to process sap flow data. S AP F LOWER can remove over 90% of noisy data while preserving legitimate variations and achieve high accuracy in gap-filling based on user-determined parameters. Random forest, LSTM, and BiLSTM models reduced root mean square error to 10% or less for long-term gaps. Model training and prediction can be performed efficiently within seconds. S AP F LOWER significantly enhances the efficiency and accessibility of TDP data analysis by automating complex tasks, enabling researchers without programming expertise to employ advanced techniques. Future improvements will focus on species-specific corrections for TDP and support for additional measurement methods. S AP F LOWER is openly available on GitHub (https://github.com/JiaxinWang123/SapFlower) and Zenodo (doi: 10.5281/zenodo.13665919).

ecosystem water balance↗
DOE OSTI2012

Colorado Heat Flow Data from IHFC

This layer contains the heat flow sites and data of the State of Colorado compiled from the International Heat Flow Commission (IHFC) of the International Association of Seismology and Physics of the Earth's Interior (IASPEI) global heat flow database. The data include different items: Item number, descriptive code, name of site, latitude and longitude, elevation, depth interval, number of temperature data, temperature gradient, number of conductivity measurement, average conductivity, number of heat generation measurements, average heat production, heat flow, number of individual sites, references, and date of publication.

15 GEOTHERMAL ENERGY↗
DOE OSTI2015

Evaluating the Efficacy of Wavelet Configurations on Turbulent-Flow Data

I/O is increasingly becoming a significant constraint for simulation codes and visualization tools on modern supercomputers. Data compression is an attractive workaround, and, in particular, wavelets provide a promising solution. However, wavelets can be applied in multiple configurations, and the variations in configuration impact accuracy, storage cost, and execution time. While the variation in these factors over wavelet configurations have been explored in image processing, they are not well understood for visualization and analysis of scientific data. To illuminate this issue, we evaluate multiple wavelet configurations on turbulent-flow data. Our approach is to repeat established analysis routines on uncompressed and lossy-compressed versions of a data set, and then quantitatively compare their outcomes. Our findings show that accuracy varies greatly based on wavelet configuration, while storage cost and execution time vary less. Overall, our study provides new insights for simulation analysts and visualization experts, who need to make tradeoffs between accuracy, storage cost, and execution time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bayesian inference of in-medium baryon-baryon scattering cross sections from HADES proton flow data

Within a Bayesian statistical framework using a Gaussian Process emulator for an isospin-dependent Boltzmann-Uehling-Uhlenbeck (IBUU) transport model simulator of heavy-ion reactions at intermediate energies, we infer from the HADES proton flow data the posterior probability distribution functions of in-medium baryon-baryon scattering cross section modification factor X with respect to free-space and the corresponding incompressibility K of nuclear matter as well as their correlation function. In conclusion, the mean value of X is found to be $X$ = $1.32_ {+0.28} ^ {-0.40} $ at 68% confidence level assuming the nuclear incompressibility K will not exceed 400 MeV, providing circumstantial evidence for enhanced baryon-braryon scattering cross sections in hot and dense nuclear matter.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Blockchain Empowered Provenance Framework for Sensor Identity Management and Data Flow Security in Fossil-based Power Plants

The overall goals of this project are divided into 3 important phases. First, a peer-to-peer SCADA network is to be established where identity profile of sensors from fossil-power plants will be stored in an immutable manner in order to verify data flow integrity in real time. Second, it is aimed to develop a Blockchain-based provenance platform to audit equipment operations along with data and process integrity violations. In this phase, various operational rules will be incorporated in the ledger on behalf of the operating sensor so its abnormal use can be easily detected. In addition, it is aimed to achieve provenance when the rate of transactions is exceedingly high, thus enabling to design a scalable provenance scheme that can work in real time when number of data collecting points or sensor units increase in fossil-based power plants. Finally, a working testbed will be developed to integrate the Blockchain platform with sensor-based fossil power plant comprising of Remote Terminal Units (RTU) and SCADA devices. Using multiple Raspberry-PIs mimicking the role of RTUs that are interfaced with multiple sensors devices will be deployed to serve as a critical component of fossil power plant and the performance metrics such as throughput, latency, bootstrapping time, etc., will be evaluated on the testbed.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2025

Advancing the utility of Data flow accelerators: Kokkos backend for Next Silicon

Next Silicon provides a novel reconfigurable accelerator, which is inherently a high-risk, early exploration effort. Higher readiness level for running NNSA mini-apps and initial mission applications on a data-flow accelerator Enable and demonstrate a new Kokkos back-end that directly drives an underlying dataflow accelerator

Garcia de Gonzalo, Simon Pedro [Sandia National La↗
DOE OSTI2026

Estimating IDP Origins Using ACLED Political Violence Events: Validation with Lebanon IDP Flow Data

A key input in modeling population distribution and flows following conflict events is the inclusion of Internally Displaced Person (IDP) flows between administrative units within a country. These flows are critical for capturing population movement and redistribution driven by current events, particularly conflict. In some cases, IDP destination data are available while origin data is incomplete or unavailable. This creates a gap in understanding where displacement is occurring, limiting the ability to model population redistribution accurately. Without origin data, it is not possible to reallocate population flows or accurately represent where displacement is occurring within the country. This report evaluates whether Armed Conflict Location Event Data (ACLED) political violence event data can be used to estimate IDP origin distributions when direct origin data are unavailable. The approach is validated using historical IDP flow data from Lebanon, where both origins and destinations are observed. Results show that ACLED event distributions strongly correspond to observed IDP-origin patterns, particularly when using cumulative 60-day event windows. The method is most reliable for identifying major origin districts and approximating proportional origin shares. However, it is not intended to reconstruct exact individual displacement flows, but rather to provide a probabilistic spatial allocation of displacement origins.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2013

Renormalization of Mass Flow Data in FCDPs for Thermal Efficiency Variation

The 40 Fuel Cycle Data Packages (FCDPs) to be used for the 2013 Evaluation and Screening (E&S) of Fuel Cycle Options have been developed using assumptions for the thermal efficiencies of the reactors in the different stages of the representative fuel cycle options for the 40 Evaluation Groups (EGs). Due to considerations about the potential for these different thermal efficiencies to bias the E&S results and maybe misinform readers, it has been requested to re-normalize mass flow data using the uniform thermal efficiency values for two additional sensitivity study cases.

Taiwo, Temitope A.↗
DOE OSTI2021

Modeling Data Flows with Network Calculus in Cyber-Physical Systems: Enabling Feature Analysis for Anomaly Detection Applications

The electric grid is becoming increasingly cyber-physical with the addition of smart technologies, new communication interfaces, and automated grid-support functions. Because of this, it is no longer sufficient to only study the physical system dynamics, but the cyber system must also be monitored as well to examine cyber-physical interactions and effects on the overall system. To address this gap for both operational and security needs, cyber-physical situational awareness is needed to monitor the system to detect any faults or malicious activity. Techniques and models to understand the physical system (the power system operation) exist, but methods to study the cyber system are needed, which can assist in understanding how the network traffic and changes to network conditions affect applications such as data analysis, intrusion detection systems (IDS), and anomaly detection. In this paper, we examine and develop models of data flows in communication networks of cyber-physical systems (CPSs) and explore how network calculus can be utilized to develop those models for CPSs, with a focus on anomaly and intrusion detection. This provides a foundation for methods to examine how changes to behavior in the CPS can be modeled and for investigating cyber effects in CPSs in anomaly detection applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Semantic Complex Event Processing over End-to-End Data Flows

Emerging Complex Event Processing (CEP) applications in cyber physical systems like SmartPower Grids present novel challenges for end-to-end analysis over events, flowing from heterogeneous information sources to persistent knowledge repositories. CEP for these applications must support two distinctive features - easy specification patterns over diverse information streams, and integrated pattern detection over realtime and historical events. Existing work on CEP has been limited to relational query patterns, and engines that match events arriving after the query has been registered. We propose SCEPter, a semantic complex event processing framework which uniformly processes queries over continuous and archived events. SCEPteris built around an existing CEP engine with innovative support for semantic event pattern specification and allows their seamless detection over past, present and future events. Specifically, we describe a unified semantic query model that can operate over data flowing through event streams to event repositories. Compile-time and runtime semantic patterns are distinguished and addressed separately for efficiency. Query rewriting is examined and analyzed in the context of temporal boundaries that exist between event streams and their repository to avoid duplicate or missing results. The design and prototype implementation of SCEPterare analyzed using latency and throughput metrics for scenarios from the Smart Grid domain.

Zhou, Qunzhi↗
DOE OSTI2024

Sap flow data, Jan 2022-Dec 2022, Panama: PNM,BCI, and SLZ

Sap flow was monitored at three Panamanian sites: Parque Natural Metropolitano (PNM), Barro Colorado Island (BCI), and Bosque Protector San Lorenzo (SLZ), using a total of 54 G-20mm sap flow sensors. Raw output from the data logger connected to 54 sap flow sensors are converted from xxx.dat. to xxx.csv. files. The metadata file (E-field log_v1-1_PNM_BCI_SLZ_sapflow_2022.xls) has information on locations where the sensors were installed, and other installation/maintenance details. No data processing or QA/QC was done on the raw datasets except conversion of dat. files to csv. files. Processed datasets have been uploaded separately. Contact mizanur.rahman@pnnl.gov or nate.mcdowell@pnnl.gov if you need to use this dataset for additional information.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Simulation process and data flow for a large system dynamics model

This paper documents the workflow and supporting technologies that a large system dynamics model, the biomass scenario model, employs to streamline the data preparation, simulation, quality control, and analysis process at the National Renewable Energy Laboratory. The workflow centers on automation of routine aspects of the flow of data between data stores, simulations, and visualizations. It enforces quality checks on data, reproducibility of computations, and traceability of results, while maintaining complete archives of modeling and analysis artifacts. The resulting frictionless simulation/analysis environment supports large-scale sensitivity analysis, interactive creation of ensembles of simulations, and rapid visualization-based exploration of simulation results.

09 BIOMASS FUELS↗
DOE OSTI2016

High-Performance data flows using analytical models and measurements

The combination of analytical models and measurements provide practical configurations and parameters to achieve high data transport rates: (a) buffer sizes and number of parallel streams for improved memory and file transfer rates, (b) Hamilton and Scalable TCP congestion control modules for memory transfers in place of default CUBIC, and (c) direct IO mode for Lustre file systems for wide-area transfers. Conventional parameter selection using full sweeps is impractical in many cases since it takes months. By exploiting the unimodality of throughput profiles, we developed the d-w method that significantly reduces the number of measurements needed for parameter identification. This heuristic method was effective in practice in reducing the measurements by about 90% for Lustre and XFS file transfers.

Rao, Nageswara S↗
DOE OSTI2022

interflow: A Python package to organize, calculate, and visualize sectoral interdependency flow data

Many economic sectors rely on an uninterrupted “upstream” supply of a resource to conduct their primary functions, leaving them vulnerable to adverse effects should that resource flow be interrupted or compromised (OECD, 2017; U.S. EPA, 2010). Well-known examples of these relationships include water demand by the energy sector (e.g., thermoelectric cooling for nuclear generation) (Grubert & Sanders, 2018; Webber, 2017) and energy demand by the water sector (e.g., electricity required to treat or move water in the public water sector) (Congressional Research Service, 2017) though many others exist. Being able to calculate and document these interdependencies and evaluate where the greatest cross-sectoral intensities and flows exist can reveal opportunities to enhance the overall network. Despite the implications and potential impacts, however, these interconnections and flows have been historically complex to analyze and understand. The interflow package provides a flexible tool to organize, calculate, and visualize (using Sankey diagrams and other visualizations) sectoral interdependency flows for multiple subsectors and resources. This tool can help decision-makers, researchers, and other audiences more easily pull meaning from these interdependencies to reveal multi-faceted opportunities and risks. interflow can help investigate questions such as (1) which sectors have high cross-resource dependencies, (2) how does demand for a resource in various sectors compare across regions, and (3) where the sectoral and regional opportunities are for enhanced efficiency, security, and resiliency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Using observational mean-flow data to drive large-eddy simulations of a diurnal cycle at the SWiFT site

Reproducing realistic date- and site-specific unsteady wind conditions in large-eddy simulations is becoming increasingly useful in wind energy. How to run a large-eddy simulation to match observed conditions, however, remains an open research question. One approach that has received considerable attention is mesoscale-to-microscale coupling, in which information about the mesoscale weather, most commonly acquired from a mesoscale numerical weather model, is passed on to a microscale model. In this paper, we demonstrate how the recently developed profile-assimilation technique, a form of mesoscale-to-microscale coupling, can be used to drive large-eddy simulations solely based on observed mean-flow profiles at a single location, bypassing the need for auxiliary mesoscale simulations. The new approach is evaluated for a diurnal cycle at the Scaled Wind Farm Technology site. Observed mean-flow profiles from the ground up to a height of 2 km are reconstructed by aggregating measurements from multiple instruments, and gaps in the data are infilled with natural neighbor interpolation. We perform nine simulations using various forcing approaches to deal with data limitations. The results show that it is indeed possible to drive microscale large-eddy simulation with observations using the profile-assimilation technique, notwithstanding large gaps in virtual potential temperature measurements. However, profile assimilation with vertical smoothing of the error between the desired and actual profiles is required. Without that smoothing, the microscale simulations develop unrealistically high turbulence levels under many situations. Finally, we show that simulated mesoscale data can account for missing observations, although care is needed as both data sources are not necessarily compatible.

17 WIND ENERGY↗