Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

FPGA-based computing system for processing data in size, weight, and power constrained environments

Technologies that are well-suited for use in size, weight, and power (SWAP)-constrained environments are described herein. A host controller dispatches data processing instructions to hardware acceleration engines (HAEs) of one or more field programmable gate arrays (FPGAs) and further dispatches data transfer instructions to a memory controller, such that the HAEs perform processing operations on data stored in local memory devices of the HAEs in parallel with other data being transferred from external memory devices coupled to the FPGA(s) to the local memory devices.

Napier, Matthew↗
DOE OSTI2021

Experimental validation of shock propagation through a foam with engineered macro-pores

The engineered macro-pore foam provides a new way to study thermonuclear burn physics by utilizing capsules containing deuterated (D) foam and filling tritium (T) gas in the engineered macro-pores. The implosion of a thermonuclear capsule filled with an engineered macro-pore foam will be complex due to the interaction of a shock wave with the engineered macro-pores. It is our goal to quantify how substantially complex foam structures affect the shape of shock and bulk shock speed. A cylinder-shape shock tube experiment has been designed and performed at the Omega Laser Facility. In order to examine how a foam structure will affect shock propagation, we performed several tests varying (1) engineered macro-pore size, (2) average foam density, and (3) with/without neopentane (C 5 H 12 ) gas. X-ray radiographic data indicate that shock speed through engineered macro-pore foams depends strongly on average foam density and less on pore size. In this work, experimental shock propagation data helped guide two numerical simulation approaches: (1) a 2D simulation with homogenizing foams rather than explicitly simulating engineered macro-pores and (2) a 2D toroidal-pore approximation adopting a toroidal-tube geometry to model engineered macro-pores.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

DeepLynx Ecosystem 2025

Poor data integration and governance continue to plague complex engineering projects, resulting in missed cost, schedule, and performance targets. Departments operate in isolated systems with manual data exchange, creating fragmented information that compounds errors and leads to significant delays and cost overruns. The DeepLynx ecosystem addresses these challenges through an open-source, modular data management platform that transforms fragmented project data into an integrated digital thread. Built on a federated microservice architecture, the ecosystem comprises seven specialized tools centered around DeepLynx Nexus, a unified data catalog with hierarchical organization and graph-based navigation capabilities. The ecosystem includes: DeepLynx Stream for real-time timeseries data ingestion from industrial sources; DeepLynx Ingest for governed data uploads with formal review workflows; DeepLynx Lattice for ontology-based entity and relationship extraction; DeepLynx Run for workflow orchestration and secure AI/ML compute; DeepLynx Visualize for 3D digital twin visualization; and DeepLynx Insight for AI-assisted document analysis with traceable, grounded responses. Deployable in cloud, on-premise, or hybrid environments using containerized Docker applications and Helm charts, the DeepLynx ecosystem provides flexible infrastructure that adapts to organizational requirements. By consolidating project data into a unified data lake with role-based access controls and OAuth2 authentication, DeepLynx enables digital thread and digital twin capabilities that improve decision-making, reduce risk, and support complex engineering workflows throughout the project lifecycle.

42 - ENGINEERING↗
DOE OSTI2024

A comprehensive guide to CAN IDS data and introduction of the ROAD dataset

Although ubiquitous in modern vehicles, Controller Area Networks (CANs) lack basic security properties and are easily exploitable. A rapidly growing field of CAN security research has emerged that seeks to detect intrusions or anomalies on CANs. Producing vehicular CAN data with a variety of intrusions is a difficult task for most researchers as it requires expensive assets and deep expertise. To illuminate this task, we introduce the first comprehensive guide to the existing open CAN intrusion detection system (IDS) datasets. We categorize attacks on CANs including fabrication (adding frames, e.g., flooding or targeting and ID), suspension (removing an ID’s frames), and masquerade attacks (spoofed frames sent in lieu of suspended ones). We provide a quality analysis of each dataset; an enumeration of each datasets’ attacks, benefits, and drawbacks; categorization as real vs. simulated CAN data and real vs. simulated attacks; whether the data is raw CAN data or signal-translated; number of vehicles/CANs; quantity in terms of time; and finally a suggested use case of each dataset. State-of-the-art public CAN IDS datasets are limited to real fabrication (simple message injection) attacks and simulated attacks often in synthetic data, lacking fidelity. In general, the physical effects of attacks on the vehicle are not verified in the available datasets. Only one dataset provides signal-translated data but is missing a corresponding “raw” binary version. This issue pigeon-holes CAN IDS research into testing on limited and often inappropriate data (usually with attacks that are too easily detectable to truly test the method). The scarcity of appropriate data has stymied comparability and reproducibility of results for researchers. As our primary contribution, we present the Real ORNL Automotive Dynamometer (ROAD) CAN IDS dataset, consisting of over 3.5 hours of one vehicle’s CAN data. ROAD contains ambient data recorded during a diverse set of activities, and attacks of increasing stealth with multiple variants and instances of real (i.e. non-simulated) fuzzing, fabrication, unique advanced attacks, and simulated masquerade attacks. To facilitate a benchmark for CAN IDS methods that require signal-translated inputs, we also provide the signal time series format for many of the CAN captures. Our contributions aim to facilitate appropriate benchmarking and needed comparability in the CAN IDS research field.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Model-Based Approaches to Generate Knowledge from Data in a Plant Reliability Context

One challenge that nuclear power plant system engineers are facing is that the amount of equipment reliability (ER) data being continuously generated are extremely large. These data elements come in different forms: textual (e.g., condition reports) and numeric (e.g., generated by monitoring systems) and they provide system engineers with valuable insights and information regarding the discovery of anomalous behaviors or degradation trends, the identification of the possible causes behind such behaviors and trends, and the prediction of their direct consequences. This paper directly targets the generation of knowledge from ER data by putting “data into context”. Here, we employ model-based system engineering (MBSE) models of systems and assets to represent and capture their architecture and functional (i.e., cause-effect) relations. ER data elements are processed by identifying first which elements of the developed MBSE elements they are referring to. This task is much harder for textual data since the information contained in issue or maintenance reports needs to “be understood” by a computational tool. Here we called this process “knowledge extraction” where our methods to extract knowledge from textual data. Lastly, once numeric and textual ER data elements have been processed and “understood”, we discover possible cause-effect relations among them. This is performed by observing if a logical connection through the MBSE models exists, and if there is a temporal relation among them. The logic and temporal are the two main ingredients to perform “machine reasoning” from ER data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Experimental study of airpath electrification in an opposed-piston two stroke (OP2S) engine architecture

The opposed-piston two stroke (OP2S) engine shows potential as an alternative engine architecture to the conventional four stroke engine due to its high-power density, thermal efficiency, and versatile airpath management system. Since the pistons of a two-stroke engine do not pump the air into and out of the cylinder like in a four-stroke engine, the selection of the air induction devices and airpath actuators becomes critical to optimize engine performance. Both the pumping losses and the in-cylinder combustion process can be affected by the scavenging process in a two-stroke engine. Therefore, this study compares two different airpath configurations for the same family of OP2S engines and investigates performance metrics like scavenging control, pumping work, net indicated and brake efficiencies, and engine-out emissions associated with each airpath. Data was collected on a 3.2 L, two-cylinder OP2S engine with an electrically assisted turbocharger (EAT) and a 4.9 L displacement, three-cylinder engine with a variable geometry turbocharger (VGT) and a supercharger. The experiments consisted of speed and load sweeps for both engines at the same operating conditions to compare scavenge control in both architectures. For the three-cylinder layout, the SE sweep range was much higher, and the intake pressure could be independently varied with air flowrate, thus providing more flexibility for scavenging control. The supercharger and the VGT usage was optimized based on its efficiency map and thus, this layout had lower pumping losses compared to the EAT. The two-cylinder engine had a higher overall SE as compared to the three-cylinder engine, but the intake pressure and air flowrate could not be decoupled, leading to over scavenging and increased short circuiting of fresh charge into the exhaust.

Bhatt, Ankur [Clemson University, Clemson, SC, USA↗
DOE OSTI2025

FY24 Laboratory Directed Research and Development Annual Report

The Laboratory Directed Research and Development (LDRD) program yields foundational scientific research and development (R&D) essential to growing SRNL’s core competencies, in alignment with SRNL’s Strategic Plan to provide long-term benefits to the Department of Energy (DOE), the National Nuclear Security Administration (NNSA), and other customers and stakeholders. Five strategic goals are outlined in SRNL’s strategic plan: 1) Provide applied science and engineering for EM’s active clean-up sites and LM’s post closure management sites 2) Provide science-based solutions for gaps identified in nonproliferation strategic vision and support the government in activities impacting national security 3) Lead Science, Technology & Engineering as the central technical authority for processing tritium loaded reservoirs and support production of plutonium pits 4) Align science and energy security programs by focusing modern modeling, simulation, and data analytics tools on materials engineering and performance applications 5) Build a workforce for the future

Clark, Sue [Savannah River National Laboratory (SR↗
DOE OSTI2024

How to Build a Quantum Supercomputer: Scaling from Hundreds to Millions of Qubits

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for quantum algorithmic primitives on hundreds of physical qubits and proof-of-principle error-correction on a single logical qubit. Nevertheless, despite significant progress and excitement, the path toward a full-stack scalable technology is largely unknown. There are significant outstanding quantum hardware, fabrication, software architecture, and algorithmic challenges that are either unresolved or overlooked. These issues could seriously undermine the arrival of utility-scale quantum computers for the foreseeable future. Here, we provide a comprehensive review of these scaling challenges. We show how the road to scaling could be paved by adopting existing semiconductor technology to build much higher-quality qubits, employing system engineering approaches, and performing distributed quantum computation within heterogeneous high-performance computing infrastructures. These opportunities for research and development could unlock certain promising applications, in particular, efficient quantum simulation/learning of quantum data generated by natural or engineered quantum systems. To estimate the true cost of such promises, we provide a detailed resource and sensitivity analysis for classically hard quantum chemistry calculations on surface-code error-corrected quantum computers given current, target, and desired hardware specifications based on superconducting qubits, accounting for a realistic distribution of errors. Furthermore, we argue that, to tackle industry-scale classical optimization and machine learning problems in a cost-effective manner, heterogeneous quantum-probabilistic computing with custom-designed accelerators should be considered as a complementary path toward scalability.

Mohseni, Masoud↗
DOE OSTI2023

Tidal Turbine Benchmarking Project: Stage I - Steady Flow Blind Predictions: Preprint

This paper presents the first blind prediction stage of the Tidal Turbine Benchmarking Project being conducted and funded by the UK's EPSRC and Supergen ORE Hub. In this first stage, only steady flow conditions, at low and elevated turbulence (3.1%) levels, were considered. Prior to the blind prediction stage, a large laboratory scale experiment was conducted in which a highly instrumented 1.6m diameter tidal rotor was towed through a large towing tank in well-defined flow conditions with and without an upstream turbulence grid. Details of the test campaign and rotor design were released as part of this community blind prediction exercise. Participants were invited to use a range of engineering modelling approaches to simulate the performance and loads of the turbine. 26 submissions were received from 12 groups from across academia and industry using solution techniques ranging from blade resolved computational fluid dynamics through actuator line, boundary integral element methods, vortex methods to engineering Blade Element Momentum methods. The comparisons between experiments and blind predictions were extremely positive helping to provide validation and uncertainty estimates for the models, but also validating the experimental tests themselves. The exercise demonstrated that the experimental turbine data provides a robust data set against which researchers and design engineers can test their models and implementations to ensure robustness in their processes, helping to reduce uncertainty and provide increased confidence in engineering processes. Furthermore, the data set provides the basis by which modellers can evaluate and refine approaches.

benchmarking↗
DOE OSTI2023

Use of Convolutional Neural Network Image Classification and High-Speed Ion Probe Data Toward Real-Time Detonation Characterization in a Water-Cooled Rotating Detonation Engine

As rotating detonation engines (RDEs) progress in maturity, the importance of monitoring advancements toward development of active control becomes more critical. Experimental RDE data processing at time scales which satisfy real-time diagnostics will likely require the use of machine learning. This study aims to develop and deploy a novel real-time monitoring technique capable of determining detonation wave number, direction, frequency, and individual wave speeds throughout experimental RDE operational windows. To do so, the diagnostic integrates image classification by a convolutional neural network (CNN) and ionization current signal analysis. Wave mode identification through single-image CNN classification bypasses the need to evaluate sequential images and offers instantaneous identification of the wave mode present in the RDE annulus. Here, real-time processing speeds are achieved due to low data volumes required by the methodology, namely one short-exposure image and a short window of sensor data to generate each diagnostic output. The diagnostic acquires live data using a modified experimental setup alongside Pylon and PyDAQmx libraries within a python data acquisition environment. Lab-deployed diagnostic results are presented across varying wave modes, operating conditions, and data quality, currently executed at 3–4 Hz with a variety of iteration speed optimization options to be considered as future work. These speeds exceed that of conventional techniques and offer a proven structure for real-time RDE monitoring. The demonstrated ability to analyze detonation wave presence and behavior during RDE operation will certainly play a vital role in the development of RDE active control, necessary for RDE technology maturation toward industrial integration.

42 ENGINEERING↗
DOE OSTI2022

A review of computing-based automated fault detection and diagnosis of heating, ventilation and air conditioning systems

We report faults in Heating, Ventilation, and Air Conditioning (HVAC) systems of buildings result in significant energy waste in building operation. With fast-growing sensing data availability and advancement in computing, computational modeling has demonstrated strong capability to detect and diagnose HVAC system faults, hence, ensuring efficient building operation. This paper comprehensively reviews the state-of-the-art computing-based fault detection and diagnosis (FDD) for HVAC systems. Overall, the reviewed computing-based FDD methods are classified as two major approaches: knowledge-based and data-driven approaches. We then identify multiple important topics, including data availability, training data size, data quality, approach generality, capability, interpretability, and required modeling efforts, along with corresponding metrics to summarize the most updated FDD development. Generally, the knowledge-based approaches are further divided as physics-based modeling, Diagnostic Bayesian Network, and performance indicator-based methods while data-driven approaches include supervised learning, unsupervised learning, and regression and statistics-based methods. State-of-the-art FDD development, remaining challenges, and future research directions are further discussed to push forward FDD in practice. Availability of fault data, capability of existing methods to deal with complex fault situations (such as simultaneous faults), modeling interpretability for data-driven methods, and required engineering efforts for physics-based methods are identified as remaining challenges in FDD development. Improving modeling fidelity and reducing modeling efforts are essential for applying physics-based methods in real buildings. Meanwhile, addressing fault data availability, increasing algorithm adaptability, and handling multiple faults are essential to further enhance the applicability of data-driven FDD approaches.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Research Software Engineering Efforts for DataFlow: FY2021 Developments

DataFlow is a web application that helps scientific data to flow from one source location to another destination location. DataFlow helps scientists easily capture scientific metadata associated with an experiment and transmit both metadata and experimental data to a designated, centralized data storage resource. This report describes the software engineering efforts and architecture of the project for the fiscal year 2021 developments. We hope it effectively communicates findings from our work, challenges we have overcome, and how we will continue our development of DataFlow in the future.

42 ENGINEERING↗
DOE OSTI2023

A Theoretical Open Architecture Framework and Technology Stack for Digital Twins in Energy Sector Applications

Digital twin is often viewed as a technology that can assist engineers and researchers make data-driven system and network-level decisions. Across the scientific literature, digital twins have been consistently theorized as a strong solution to facilitate proactive discovery of system failures, system and network efficiency improvement, system and network operation optimization, among others. With their strong affinity to the industrial metaverse concept, digital twins have the potential to offer high-value propositions that are unique to the energy sector stakeholders to realize the true potential of physical and digital convergence and pertinent sustainability goals. Although the technology has been known for a long time in theory, its practical real-world applications have been so far limited, nevertheless with tremendous growth projections. In the energy sector, there have been theoretical and lab-level experimental analysis of digital twins but few of those experiments resulted in real-world deployments. There may be many contributing factors to any friction associated with real-world scalable deployment in the energy sector such as cost, regulatory, and compliance requirements, and measurable and comparable methods to evaluate performance and return on investment. Those factors can be potentially addressed if the digital twin applications are built on the foundations of a scalable and interoperable framework that can drive a digital twin application across the project lifecycle: from ideation to theoretical deep dive to proof of concept to large-scale experiment to real-world deployment at scale. This paper is an attempt to define a digital twin open architecture framework that comprises a digital twin technology stack (D-Arc) coupled with information flow, sequence, and object diagrams. Those artifacts can be used by energy sector engineers and researchers to use any digital twin platform to drive research and engineering. This paper also provides critical details related to cybersecurity aspects, data management processes, and relevant energy sector use cases.

Gourisetti, Sri, Nikhil Gupta (ORCID:0000000188778↗
DOE OSTI2025

On-Demand Column Joining for High Energy Physics

As the Large Hadron Collider (LHC) transitions into the High-Luminosity LHC (HL-LHC) era, the volume of data to be processed is expected to increase significantly. The CMS Experiment currently utilizes various data formats, including AOD, MiniAOD, and NanoAOD, each with different levels of detail and storage requirements. This paper addresses the challenges of data duplication and storage inefficiencies in high-energy physics (HEP) analyses by proposing an on-demand column-joining solution. This approach aims to reduce data duplication by enabling the dynamic combination of NanoAOD data with auxiliary information from larger data tiers, such as MiniAOD. The proposed solution leverages Trino, a high-performance distributed SQL query engine, to perform efficient and scalable data joins. Benchmarks using CMS OpenData demonstrate the feasibility of this approach, showing that it can handle large datasets with low latency. Integration with the scikit-hep ecosystem and the coffea analysis framework is also discussed, highlighting the potential for seamless end-to-end data processing and analysis. Ongoing and future work focuses on expanding benchmarks, integrating ServiceX for data transformation, and exploring the use of native object storage solutions.

Manganelli, Nicholas [Northeastern U.]↗
DOE OSTI2021

Advanced Health Information Technology Analytic Framework and Application to Hazard Detection

Health Information Technology (HIT) aims to improve healthcare outcomes by organizing and analyzing various health-related data. With data accumulating at a staggering rate, the importance of real-time analytics has been increasing dramatically, shifting the focus of informatics from batch processing to streaming analytics. HIT is also facing unprecedented challenges in adapting to this new requirement and leveraging advanced IT technologies. This paper introduces a HIT data and compute platform that supports multi-granularity real-time analytics from heterogeneous data sources. The paper first identifies functional requirements and proposes a framework that satisfies the requirements using state-of-the-art big data technologies including Apache Kafka, Spark Structured Streaming Engine, and Delta Lake. To demonstrate its capability to support data analytics in multiple time granularities analytics, a statistical process control-based hazard detection algorithm has been implemented on top of the framework to detect unexpected hazards from order cancellation data of the Department of US Veterans Affairs (VA) in near real-time.

Kumar, Mohit↗
DOE OSTI2025

Data Qualification Report: SRNL Glass Composition-Properties (ComPro) Database

The Savannah River National Laboratory Glass Composition-Properties (ComPro) database is an extensive database containing pertinent composition and durability data to support the accelerated clean-up mission at the Defense Waste Processing Facility. The activities described in this data qualification report were performed to support the information contained in the database. There were two objectives of the original data qualification process. The first objective was to review supporting documentation to determine if DOE/RW-0333P Quality Assurance Requirements and Description had been implemented during the original work. If the DOE/RW-0333P Quality Assurance Requirements and Description had not been directly implemented during the original work, the second objective was to determine if the controls that were used were adequate to meet the intent of the DOE/RW-0333P Quality Assurance Requirements and Description. The results of these two objectives and the activities performed to support these decisions are described in this document. An assessment of each dataset was made to determine if the data were RW-0333P Compliant, RW-0333P Equivalent or Non-RW-0333P Compliant. The original data qualification was performed in accordance with E7, Conduct of Engineering Manual, Procedure 3.70, Revision 4, Qualification of Data. The specific method that was used was Equivalent Controls as described in E7, 3.70. Revision 2 of this document adds supporting information for the RW-0333P Compliant datasets added to Revision 3 of the database.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

AL-ASMR: Active Learning of Atomistic Surrogate Models for Rare Events

Atomistic simulation with artificial intelligence (AI) is an emerging tool for understanding materials' properties and behaviors and predicting novel materials with optimized/targeted properties. Neural network potentials (NNPs) are outstanding in this field as they have shown a comparable accuracy to ab initio electronic structure calculations for reproducing potential energy surfaces while being several orders of magnitude faster. However, such NNPs can perform poorly outside of their training domain and typically fail catastrophically in the prediction of rare events in molecular dynamics (MD) simulations. For effective AL loops to distinguish the informative data from enhanced sampled configurations, we developed a decision engine by configurational similarity and uncertainty quantification (UQ) with data augmentation.

Jung, Gang Seob↗