Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Management Platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Value of Information and Data Management Platform Promoting Modern Data Analysis Utilization. STTR Phase I, Final Technical Report

The immense growth in weather and river sensor data availability along with increases in computational power and algorithmic sophistication have enabled the widespread application of physical, machine learning, and artificial intelligence algorithms to river forecasting. The forecasts produced by widely varying model approaches are inherently limited in different ways, such as by the number and quality of training observations, producing biased predictions that achieve optimal skill only over specific ranges of weather and river conditions. Thus, despite the growing availability and new sources of forecasts, hydropower owners cannot fully leverage the information because they lack an effective mechanism to determine the “best” forecast for given conditions or objectives. Tools are needed to consolidate forecast information, demonstrate skill, and extract optimal forecast guidance from the range of possibilities. Such tools will increase uptake and beneficial use of the growing wealth of forecast information, and lead to improved operating decisions, optimal power generation, and enhanced system value. This research proposes to improve the value of forecasts by developing a broadly accessible verification framework and applying machine learning techniques to identify the ‘best’ forecast from an ensemble of sources given the current weather, river, and operating conditions.

13 HYDRO ENERGY↗

Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data

To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.

data management platform↗

Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data: Preprint

To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.

data management↗

FY 2024 Multidimensional Data Correlation Platform Data Management Infrastructure Progress: Materials Laboratory

This report provides an inventory of the equipment available at the ORNL Manufacturing Demonstration Facility (MDF) for sample preparation and material characterization, including both destructive and non-destructive techniques that generate critical data to support the development of the Multi-Dimensional Data Correlation (MDDC) framework. The success of the MDDC framework depends heavily on the quality and completeness of the data it can access. Therefore, it is essential to establish a comprehensive inventory of the technologies available to the Advanced Materials and Manufacturing Technologies (AMMT) multi-laboratory team. This starts by gathering information about the types of data they produce, the data collection and transfer protocols used, file formats, and data storage requirements for experiments. This information is then carefully evaluated to create the operations and trackables elements of the Damara Tern platform, which is the foundation of the MDDC framework.

36 MATERIALS SCIENCE↗

Design of an Open Smart Energy Gateway for Smart Meter Data Management

With the widespread deployment of electronic interval meters, commonly known as smart meters, came the promise of real-time data on electric energy consumption. Recognizing an opportunity to provide consumers access to their near real-time energy consumption data directly from their installed smart meter, we designed a mechanism for capturing those data for consumer use via an open smart energy gateway (OpenSEG). By design, OpenSEG provides a clearly defined boundary for equipment and data ownership. OpenSEG is an open-source data management platform to enable better data management of smart meter data. Effectively, it is an information architecture designed to work with the ZigBee Smart Energy Profile 1.x (SEP 1.x). It was specifically designed to reduce cyber-security risks and provide secure information directly from smart meters to consumers in near real time, using display devices already owned by the consumers. OpenSEG stores 48 hours of recent consumption data in a circular cache using a format consistent with commonly available archived (not real-time) consumption data such as Green Button, which is based on the Energy Services Provider Interface (ESPI) data standard. It consists of a common XML format for energy usage information and a data exchange protocol to facilitate automated data transfer upon utility customer authorization. Included in the design is an application program interface by which users can acquire data from OpenSEG for further post processing. A sample data display application is included in the initial software product. The data display application demonstrates that OpenSEG can help electricity use data to be retrieved from a smart meter and ported to a wide variety of user-owned devices such as cell phones or a user-selected database. This system can be used for homes, multi-family buildings, or small commercial buildings in California.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

FY 2025 Multidimensional Data Correlation Platform: Unified Software Architecture for Advanced Materials and Manufacturing Technologies Data Management and Processing

The Advanced Materials and Manufacturing Technologies (AMMT) program continues to advance a data-driven approach to demonstrate the utility of additive manufacturing for fabricating components for nuclear applications. A key scientific goal is to leverage data to better understand manufacturing outcomes and thereby improve the performance, reliability, and lifespan of nuclear components. Ultimately, this effort supports the development of standards for certification and qualification of additively manufactured components, enabling broader industry adoption. In support of this objective, the AMMT program is building and deploying a data management platform to record, index, analyze, and make available the manufacturing data generated across the AMMT program. In FY 2023, the team conceptualized the architecture of the platform and, in FY 2024, deployed the first functional version at the Oak Ridge National Laboratory (ORNL) Manufacturing Demonstration Facility (MDF). In FY 2025, the platform was officially opened to all AMMT members. To enable this expansion, core modifications and enhancements were developed, including improvements to the user interface and workflows for data entry and retrieval. Most notably, robust security and access control mechanisms were implemented to protect data and manage information sharing. This effort featured a logging system, protected views, and controlled access mechanisms. This report documents these enhancements and the transition of the platform into program-wide use.

36 MATERIALS SCIENCE↗

Grid Operator Analytics and Assessment Tools for Inverter- Based Resources Dominated Grid (GOAAT-IBR) Project Update

This presentation provides an update on the OPTIMA GOAAT project, with emphasis on the cloud-native data platform developed in-house to ingest, manage, and operationalize high-resolution power system data. Since our last NASPI presentation, accessible via OSTI ID #2671437, the project team advanced the design and deployment of a scalable architecture capable of handling both synchronized and non-synchronized streams, including PMU, point-on-wave (POW), COMTRADE, and SCADA data. These materials review the project status, recent progress, and key lessons learned. The core of the presentation examines the architecture and engineering of our cloud-native ingestion and data management platform. We then explain how pipelines were designed to collect, normalize, time-align, store, and serve heterogeneous data at scale. We will discuss design choices such as data models, streaming versus batch ingestion, storage tiers, and interoperability with analytics applications. Practical experiences with cloud-native technologies were shared during the event, including benefits, limitations, and integration challenges in a utility environment, along with methods used to improve performance, reduce latency, and optimize resource usage. The presentation also showcases user interface designs and visualization tools that convert raw measurements and analytics results into intuitive, actionable insights for operators and engineers. During the presentation examples were provided demonstrating how visualization, event views, and summarized analytics enhance situational awareness and support operational decision-making. These use cases illustrate how a well-designed data infrastructure can bridge the gap between high-volume measurements and practical grid operations.

Aminifar, Farrokh↗

DeepLynx Ecosystem 2025

Poor data integration and governance continue to plague complex engineering projects, resulting in missed cost, schedule, and performance targets. Departments operate in isolated systems with manual data exchange, creating fragmented information that compounds errors and leads to significant delays and cost overruns. The DeepLynx ecosystem addresses these challenges through an open-source, modular data management platform that transforms fragmented project data into an integrated digital thread. Built on a federated microservice architecture, the ecosystem comprises seven specialized tools centered around DeepLynx Nexus, a unified data catalog with hierarchical organization and graph-based navigation capabilities. The ecosystem includes: DeepLynx Stream for real-time timeseries data ingestion from industrial sources; DeepLynx Ingest for governed data uploads with formal review workflows; DeepLynx Lattice for ontology-based entity and relationship extraction; DeepLynx Run for workflow orchestration and secure AI/ML compute; DeepLynx Visualize for 3D digital twin visualization; and DeepLynx Insight for AI-assisted document analysis with traceable, grounded responses. Deployable in cloud, on-premise, or hybrid environments using containerized Docker applications and Helm charts, the DeepLynx ecosystem provides flexible infrastructure that adapts to organizational requirements. By consolidating project data into a unified data lake with role-based access controls and OAuth2 authentication, DeepLynx enables digital thread and digital twin capabilities that improve decision-making, reduce risk, and support complex engineering workflows throughout the project lifecycle.

42 - ENGINEERING↗

Digital Twin for Hydropower System Object Modeling: Alder Dam (FY2023)

Hydropower is the world's largest source of renewable electricity, and hydropower plants are distributed all over the world. Typical major components of a hydropower plant are the governor, excitation, generator, thrust bearing, hydraulic turbine, transformer, the main lead, metering and control, tailwater depression, and dissolved oxygen. For each component, various measures are taken. The measurements are acquired by various heterogeneous systems, including standalone sensors, programmable logic controllers (PLC), Supervisory control and data acquisition (SCADA), Internet of Things (IoT), and data acquisition and integration platforms such as OSI/PI. The measured data are often archived within the plant by a data management platform, and many institutions have cloud-based archive systems, such as Hydropower Research Institution (HRI), U.S. Army Corps of Engineers (USACE), and Columbia River Data Access in Real Time (DART). Object Modeling is a general framework for designing information systems. It focuses on objects, the actions they perform, and the messages they send to one another to cause those actions to be taken. The major differences among object modeling, network modeling, data modeling, and process modeling are that in the first we focus on the actions in response to information, objects which form the system, the actions they perform, and how they pass information to one another, while in the second we concentrate on where, when and how much information is moved, while in the third we focus on what information is moved and where it is moved, while in the last we focus on how it is moved and when it is moved. Object modeling was developed basically as a method to develop object-oriented systems and to support object-oriented programming. It describes the static structure of the system. The object Modeling Technique is easy to draw and use. That is why we choose object modeling to connect physical hydropower plants to Digital Twin. It recognizes the objects and the relationship between them. It identifies the attributes and functions of each class. Dynamic Modeling: It explains how objects respond to events. Functional Modeling indicates the processes executed in an object and how data changes when it moves to objects. It has been used in many applications like telecommunication, transportation, etc.

13 HYDRO ENERGY↗

An open-source data storage and visualization platform for collaborative qubit control

Developing collaborative research platforms for quantum bit control is crucial for driving innovation in the field, as they enable the exchange of ideas, data, and implementation to achieve more impactful outcomes. Furthermore, considering the high costs associated with quantum experimental setups, collaborative environments are vital for maximizing resource utilization efficiently. However, the lack of dedicated data management platforms presents a significant obstacle to progress, highlighting the necessity for essential assistive tools tailored for this purpose. Current qubit control systems are unable to handle complicated management of extensive calibration data and do not support effectively visualizing intricate quantum experiment outcomes. In this paper, we introduce Qubit Control Storage and Visualization ( QubiCSV ), a platform specifically designed to meet the demands of quantum computing research, focusing on the storage and analysis of calibration and characterization data in qubit control systems. As an open-source tool, QubiCSV facilitates efficient data management of quantum computing, providing data versioning capabilities for data storage and allowing researchers and programmers to interact with qubits in real time. The insightful visualization are developed to interpret complex quantum experiments and optimize qubit performance. QubiCSV not only streamlines the handling of qubit control system data but also improves the user experience with intuitive visualization features, making it a valuable asset for researchers in the quantum computing domain.

97 MATHEMATICS AND COMPUTING↗

Edge Bioinformatics

Edge Bioinformatics is a developmental bioinformatics and data management platform which seeks to supply laboratories with bioinformatics pipelines for analyzing data associated with common samples case goals. Edge Bioinformatics enables sequencing as a solution and forward-deployed situations where human-resources, space, bandwidth, and time are limited. The Edge bioinformatics pipeline was designed based on following USE CASES and specific to illumina sequencing reads. 1. Assay performance adjudication (PCR): Analysis of an existing PCR assay in a genomic context, and automated design of a new assay to resolve conflicting results; 2. Clinical presentation with extreme symptoms: Characterization of a known pathogen or co-infection with a. Novel emerging disease outbreak or b. Environmental surveillance

Lo, Chien-Chi↗

Edge Bioinformatics

Edge Bioinformatics is a developmental bioinformatics and data management platform which seeks to supply laboratories with bioinformatics pipelines for analyzing data associated with common samples case goals. Edge Bioinformatics enables sequencing as a solution and forward-deployed situations where human-resources, space, bandwidth, and time are limited. The Edge bioinformatics pipeline was designed based on following USE CASES and specific to illumina sequencing reads. 1. Assay performance adjudication (PCR): Analysis of an existing PCR assay in a genomic context, and automated design of a new assay to resolve conflicting results; 2. Clinical presentation with extreme symptoms: Characterization of a known pathogen or co-infection with a. Novel emerging disease outbreak or b. Environmental surveillance

Lo, Chien-Chi↗

The EGS Data Collaboration Platform: Enabling Scientific Discovery

Collaboration in the digital age has been stifled in recent years. Reasonable responses to legitimate security concerns have created a virtual landscape of silos and fortified castles incapable of sharing information efficiently. This trend is unfortunately opposed to the geothermal scientific community's migration toward larger, more collaborative projects. To facilitate efficient sharing of information between team members from multiple national labs, universities, and private organizations, the 'EGS Collab' team has developed a universally accessible, secure data collaboration platform and has fully integrated it with the U.S. Department of Energy's (DOE) Geothermal Data Repository (GDR) and the National Geothermal Data System (NGDS). This paper will explore some of the challenges of collaboration in the modern digital age, highlight strategies for active data management, and discuss the integration of the EGS Collab data management platform with the GDR to enable scientific discovery through the timely dissemination of information.

97 MATHEMATICS AND COMPUTING↗

CO 2 Storage Site Screening Platform Development and CO 2 Storage Resource Analysis in SECARB Offshore Reservoirs Using SAS Viya

A major goal of the SECARB Offshore Partnership (DE-FE0031557) is to screen deep saline aquifers and hydrocarbon reservoirs in the central Gulf of Mexico for CO 2 sequestration and CO 2 -enhanced oil and gas recovery (EOR/EGR) and estimate the corresponding CO 2 storage resources for select reservoirs. CO 2 storage potential associated with offshore CO 2 -EOR is considerable and likely represents “low hanging fruit” for near-term CO 2 storage given the in-place infrastructure in the region. It is for these reasons that this assessment focuses on oil and gas fields. To this end, three major objectives have been completed and include (1) managing geological data derived from different sources, (2) building a reservoir screening platform for CO 2 storage, and (3) ranking the reservoirs based on the estimated CO 2 storage resources. The SAS ® Viya platform was used for data management and analytics. The Viya platform is a cloud service platform that provides data integration, data management, quick analytics, data visualization, machine learning functions, and application programming interfaces (APIs) for multi-programming languages. Different sources of data containing geologic information, reservoir properties, and EOR/EGR information were collected, cleaned, formatted, and loaded into the SAS ® Viya platform for evaluation. The major geological characteristics of both shelf and deep-water areas of the central Gulf were examined and compared to define the appropriate reservoir screening criteria. Next, a CO 2 storage site screening system was built in the SAS ® Viya platform with the pre-defined criteria. Finally, the CO 2 storage resources of the screened reservoirs were calculated and reported at the BOEM field level to identify fields with the highest estimated CO 2 storage resource. The fields with the largest total estimated CO 2 storage resource are located in the Mississippi Canyon protraction area. Due to proximity to the Mississippi Delta (indicative of less infrastructure) and large estimated CO 2 storage resources, future development activities may wish to focus efforts in the Mississippi Canyon protraction area.

02 PETROLEUM↗

Standard Energy Efficiency Data Platform® [SWR-17-25]

The Standard Energy Efficiency Data Platform® (SEED Platform) provides public agencies and other organizations with a standardized but flexible, cost-effective, secure, enterprise data platform to manage portfolio scale building performance data from a variety of sources. The SEED Platform streamlines the administrative effort required by public agencies implementing building performance and transparency programs.

Source record↗

Grid Data Crossing Platform (GriD-Xing)

This project is to develop a data management and analytics platform called Grid Data Crossing (GriD-Xing) that integrates data processing, storage, analysis, and visualization functionalities for users at industry (e.g., EMS and DMS) and universities. It intends to increase the applicability of large set of multi-source data (e.g., SCADA, PMU, μPMU, AMI, inverter, load/DER forecasting, and cyber data) and eventually improve the resilience, reliability, and security of critical energy infrastructure. GriD-Xing also intends to be a research platform to identify the usefulness of large multi-source data via case studies of interest to industry (e.g., 2011 southwest cold weather and 2016 Blue Cut fire events). In the first year, the team focused on the development and demonstration of large multi-source data (e.g., PMU, μPMU, SCADA, AMI, and cyber data) processing, storage, and analysis. Specifically, the team designed GriD-Xing framework in functional and structural aspects, developed GriD-Xing algorithms for large multi-source grid data processing, storage, and analysis, and finally demonstrated GriD-Xing prototype algorithms performance through a series of tests and case studies using real-world and simulation data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Xi-cam : a versatile interface for data visualization and analysis

Xi-cam is an extensible platform for data management, analysis and visualization. Xi-cam aims to provide a flexible and extensible approach to synchrotron data treatment as a solution to rising demands for high-volume/high-throughput processing pipelines. The core of Xi-cam is an extensible plugin-based graphical user interface platform which provides users with an interactive interface to processing algorithms. Plugins are available for SAXS/WAXS/GISAXS/GIWAXS, tomography and NEXAFS data. With Xi-cam 's `advanced' mode, data processing steps are designed as a graph-based workflow, which can be executed live, locally or remotely. Remote execution utilizes high-performance computing or de-localized resources, allowing for the effective reduction of high-throughput data. Xi-cam 's plugin-based architecture targets cross-facility and cross-technique collaborative development, in support of multi-modal analysis. Xi-cam is open-source and cross-platform, and available for download on GitHub.

Pandolfi, Ronald J. (ORCID:0000000308248548)↗