Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Nuclear Engineering Enrollments and Degrees Survey, 2019 Data (Brief 82)

The 2019 survey includes degrees granted between September 1, 2018 and August 31, 2019, and enrollments for fall 2019. Thirty-four nuclear academic programs were surveyed with all but two programs responding. The enrollments and degrees data include students majoring in nuclear engineering or in an option program equivalent to a major. Some nuclear engineering programs have indicated that health physics option enrollments and degrees are also reported in the health physics enrollments and degrees survey.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

Impacts of swinging door lossy compression of synchrophasor data

As the number of operational Phasor Measurement Units (PMUs) is increasing, compression is becoming crucial for efficiently transferring and storing enormous amount of synchrophasor data continuously measured from a power system. Some utilities in the U.S. have already adopted a nonlinear compression technique, called Swinging Door Algorithm (SDA), assuming compressed data meets technical and regulatory requirements without resulting in any degradation of the measurements. This paper studies impacts of this compression algorithm on field measured and simulated PMU data, and presents results from spectral analyses of raw and decompressed measurements. The results show blind application of the SDA compression alters frequency content in the measurements and introduces false harmonics and aliasing. These can lead to false conclusions being drawn from the measurements about the power system condition.

42 ENGINEERING↗
DOE OSTI2021

The F-box protein gene exo-1 is a target for reverse engineering enzyme hypersecretion in filamentous fungi

Carbohydrate active enzymes (CAZymes) are vital for the lignocellulose-based biorefinery. The development of hypersecreting fungal protein production hosts is therefore a major aim for both academia and industry. However, despite advances in our understanding of their regulation, the number of promising candidate genes for targeted strain engineering remains limited. Here, we resequenced the genome of the classical hypersecreting Neurospora crassa mutant exo-1 and identified the causative point of mutation to reside in the F-box protein–encoding gene, NCU09899. The corresponding deletion strain displayed amylase and invertase activities exceeding those of the carbon catabolite derepressed strain ?cre-1, while glucose repression was still mostly functional in ?exo-1. Surprisingly, RNA sequencing revealed that while plant cell wall degradation genes are broadly misexpressed in ?exo-1, only a small fraction of CAZyme genes and sugar transporters are up-regulated, indicating that EXO-1 affects specific regulatory factors. Aiming to elucidate the underlying mechanism of enzyme hypersecretion, we found the high secretion of amylases and invertase in ?exo-1 to be completely dependent on the transcriptional regulator COL-26. Furthermore, misregulation of COL-26, CRE-1, and cellular carbon and nitrogen metabolism was confirmed by proteomics. Finally, we successfully transferred the hypersecretion trait of the exo-1 disruption by reverse engineering into the industrially deployed fungus Myceliophthora thermophila using CRISPR-Cas9. Our identification of an important F-box protein demonstrates the strength of classical mutants combined with next-generation sequencing to uncover unanticipated candidates for engineering. These data contribute to a more complete understanding of CAZyme regulation and will facilitate targeted engineering of hypersecretion in further organisms of interest.

Gabriel, Raphael↗
DOE OSTI2022

An underground, wireless, open-source, low-cost system for monitoring oxygen, temperature, and soil moisture

Abstract. The use of wireless sensor networks to measure soil parameters eliminates the need to remove sensors for field operations, such as tillage, thus allowing long-term measurements without multiple disturbances to soil structure. Wireless sensors also reduce above-ground cables and the risk of undesired equipment damage and potential data loss. However, implementing wireless sensor networks in field studies usually requires advanced and costly engineering knowledge. This study presents a new underground, wireless, open-source, low-cost system for monitoring soil oxygen, temperature, and soil moisture. The process of system design, assembly, programming, deployment, and power management is presented. The system can be left underground for several years without the need to change the battery. Emphasis was given on modularity so that it can be easily duplicated or changed if needed and deployed without previous engineering knowledge. Data from this type of system have a wide range of applications, including precision agriculture and high-resolution modelling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

CROCUS Urban Canyons - Space Science and Engineering Center (SPARC) Doppler lidar data

This is the netCDF format output from the Halo Photonics Streamline XR Doppler lidar that was deployed next to the Space Science and Engineering Center (SPARC) trailer at the University of Illnois-Chicago greenhouse parking lot during CROCUS Urban Canyons. The purpose of collecting this dataset is to provide vertical and horizontal wind profiles for studying the characteristics of turbulence over the urban canyon of Chicago. This data contains the radial velocity, intensity, and backscatter from the vertical profile, range height indicator, and sector scans that were performed over both Intensive Operating Period 1 and 2 of CROCUS Urban Canyons. There are four different types of files: * The Range Height Indicator (RHI) files contain scans that are along a constant azimuth, spanning the entire hemisphere of elevation values above the surface. * The Velocity Azimuth Display (VAD) files contain the raw radial velocity data from the 6-beam, 60 degree scans. * The User1 files contain stacked Plan Position Indicator scans over a 45 degree quadrant over downtown Chicago. * The Stare files contain vertically pointing scans. These are standard netCDF files that can be opened using xarray. The VAD scans can be processed from their raw radial velocities to horizontal wind speeds with the Atmospheric data Community Toolkit (https://arm-doe.github.io/ACT/).

EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC WINDS↗
DOE OSTI2021

Adaptive Spatially Aware I/O for Multiresolution Particle Data Layouts

Large-scale simulations on nonuniform particle distributions that evolve over time are widely used in cosmology, molecular dynamics, and engineering. Such data are often saved in an unstructured format that neither preserves spatial locality nor provides metadata for accelerating spatial or attribute subset queries, leading to poor performance of visualization tasks. Furthermore, the parallel I/O strategy used typically writes a file per process or a single shared file, neither of which is portable or scalable across different HPC systems. We present a portable technique for scalable, spatially aware adaptive aggregation that preserves spatial locality in the output. We evaluate our approach on two supercomputers, Stampede2 and Summit, and demonstrate that it outperforms prior approaches at scale, achieving up to 2.5× faster writes and reads for nonuniform distributions. Furthermore, the layout written by our method is directly suitable for visual analytics, supporting low-latency reads and attribute-based filtering with little overhead.

Usher, Will↗
DOE OSTI2025

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗
DOE OSTI2021

Challenges and Advances in Information Extraction from Scientific Literature: a Review

Scientific articles have long been the primary means of disseminating scientific discoveries. Over the centuries, valuable data and potentially groundbreaking insights have been collected and buried deep in the mountain of publications. In materials engineering, such data are spread across technical handbooks specification sheets, journal articles, and laboratory notebooks in myriad formats. Extracting information from papers on a large scale has been a tedious and time-consuming job to which few researchers have wanted to devote their limited time and effort, yet is an activity that is essential for modern data-driven design practices. However, in recent years, significant progress has been made by the computer science community on techniques for automated information extraction from free text. Yet, transformative application of these techniques to scientific literature remains elusive-due not to a lack of interest or effort but to technical and logistical challenges. Using the challenges in the materials science literature as a driving motivation, we review the gaps between state-of-the-art information extraction methods and the practical application of such methods to scientific texts, and offer a comprehensive overview of work that can be undertaken to close these gaps.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

Detection and Diagnosis of Data Integrity Attacks in Solar Farms Based on Multilayer Long Short-Term Memory Network

Photovoltaic (PV) systems are becoming more vulnerable to cyber threats. In response to this emerging concern, developing cyber-secure power electronics converters has received increased attention from the IEEE Power Electronics Society that recently launched a cyber-physical-security initiative. Here this letter proposes a deep sequence learning based diagnosis solution for data integrity attacks on PV systems in smart grids, including dc–dc and dc–ac converters. The multilayer long short-term memory networks are used to leverage time-series electric waveform data from current and voltage sensors in PV systems. The proposed method has been evaluated in a PV smart grid benchmark model with extensive quantitative analysis. As a comparison, we have evaluated classic data-driven methods, including K-nearest neighbor, decision tree, support vector machine, artificial neural network, and convolutional neural network. Comparison results verify performances of the proposed method for detection and diagnosis of various data integrity attacks on PV systems.

42 ENGINEERING↗
DOE OSTI2026

Three-Dimensional Grid Visualization for Planning Activities: A Dubai Case Study

National Laboratory of the Rockies (NLR), in collaboration with the Dubai Electricity and Water Authority (DEWA) and Infra-X, has undertaken the Energy Visualization Analysis Project. The aim of this project is to enhance analytical and 3D visualization capabilities for distribution network planning and renewable energy integration. As modern grid continues to evolve with large-scale solar PV deployment and emerging distributed energy resources (DERs), the ability to effectively analyze, visualize, and communicate complex grid behaviors has become increasingly critical. The project focuses on developing empirical use cases based on real distribution feeder data and engineering workflows, ensuring the outcomes are directly aligned with operational environment. Through time-series power flow simulations and nodal hosting capacity analysis, the study quantifies the impacts of high PV penetration on voltage and thermal limits within representative 11 kV feeders. These analyses identify specific nodes and conditions where DER integration challenges arise. Furthermore, a Battery Energy Storage System (BESS) optimization algorithm was applied to determine the optimal size and placement of storage systems that can mitigate network constraints and enhance hosting capacity. The comparative results between base-case and BESS-augmented scenarios clearly demonstrate improvements in network stability and load management efficiency. In parallel, the NLR team developed an immersive 3D visualization framework, enabling interactive exploration of grid simulations using commodity head-mounted display (HMD) systems. This framework transforms conventional 2D simulation data into spatially intuitive visual environments - allowing engineers to analyze feeder conditions, PV hosting potential, and BESS effects in real time. This report represents the first foundational phase in establishing a visualization-driven analytical ecosystem. It provides a methodological foundation for data integration, visualization architecture, and simulation-based decision support, paving the way for large-scale adoption of immersive visualization across DEWA's Smart Grid Initiative, R&D activities, and future network resilience studies.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Developing multi-gene CRISPRa/i programs to accelerate DBTL cycles in ABF hosts engineered for chemical production

This project developed and implemented a modular CRISPR activation and interference (CRISPRa/i) platform to accelerate strain optimization and pathway development for industrially relevant microbial hosts. By integrating multiplexed transcriptional perturbation tools with data-driven Design–Build–Test–Learn (DBTL) workflows, the team achieved reductions in cycle time and enhanced production of industrial aromatics, particularly 4-aminocinnamic acid (4-ACA), in Pseudomonas putida. Key accomplishments included: ● Development of a robust, tunable CRISPRa/i system in P. putida that enabled efficient multi-target gene regulation via guide RNA (gRNA) programs ● Completion of two full DBTL cycles, guided by machine learning (ML) models trained on transcriptomic and performance data, reducing engineering time by over 30% ● Optimization of multi-gene regulatory programs to balance expression of host and pathway modules, improve 4-ACA titers, and resolve metabolic bottlenecks ● Demonstration of system portability through a limited proof-of-concept extension in Acinetobacter baylyi, underscoring the generalizability of the approach ● Evaluation of strain performance on lignocellulosic biomass-derived substrates, demonstrating the feasibility of converting renewable carbon into aromatic building blocks These results illustrate the feasibility of applying ML-guided CRISPRa/i perturbation strategies to accelerate strain development in complex microbial systems. The resulting tools and datasets contribute to DOE objectives by improving platform predictability, reducing development costs, and enabling broader access to sustainable, economically viable bioproduction technologies.

09 BIOMASS FUELS↗
DOE OSTI2026

Leveraging large language models to address data scarcity in machine learning for graphene synthesis

Machine learning in experimental materials science faces significant challenges due to the scarcity of data, which are costly and time-consuming to generate, particularly when relying on in-house experiments. Literature data mining offers a potential solution but introduces issues like mixed data quality, inconsistent formats, and non-uniform reporting of synthesis parameters, resulting in partially missing and heterogeneous features across the dataset. Here, we propose data imputation and feature engineering methods that employ pre-trained large language models (LLMs) to enhance machine learning performance on scarce, heterogeneous datasets, demonstrated on graphene CVD synthesis data and the ML-HydPARK hydrogen storage dataset. GPT models perform data imputation via tailored prompting and semantic normalization of inconsistently reported features through embeddings, for example, to harmonize the complex nomenclature of CVD substrates. Beyond yielding more diverse and richer feature representations than traditional methods such as K-nearest neighbors (KNN) and Multivariate Imputation by Chained Equations (MICE), LLM-based data imputation is evaluated against dataset characteristics and prompting strategies. We vary the level of autonomy granted to the LLM, from generic prompting that leverages pre-trained knowledge for autonomous data generation to data-informed prompting that constrains outputs using target-specific information, and demonstrate which level of autonomy yields superior imputation performance across datasets and feature types. The proposed data engineering methods markedly improve downstream performance; for example, in graphene layer number classification using a support vector machine (SVM), binary accuracy increases from 39% to 65% and ternary accuracy from 52% to 72%. Fine-tuning experiments on both datasets show that combining our proposed LLM-based data imputation and feature encoding methods with numerical machine learning predictors outperforms standalone fine-tuned LLM predictors in data-scarce settings. The proposed strategies emphasize data enhancement techniques rather than refining learning architectures or regularizing loss functions, offering a broadly applicable framework for improving machine learning performance on scarce, inhomogeneous datasets.

Chemical vapor deposition↗
DOE OSTI2024

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗
DOE OSTI2021

Data Driven Fault Detection of Premixer Centerbody Degradation in a Swirl Combustor

This paper introduces a data-driven framework for combustor-focused, performance-based condition monitoring of gas turbines. Commercial condition monitoring systems typically generate huge amounts of data that make efficient onboard monitoring challenging. This paper focuses on quantifying combustor component degradation, using premixer centerbody degradation in a swirl stabilized combustor as a case study. The input for these analyses is acoustic pressure measurements acquired at various locations on the combustor. The diagnosis methodology is based on a classification framework and consists of 3 steps: 1) Data curation, 2) Feature Engineering, and 3) Diagnosis. Data curation ensures good quality of the data that is passed through the algorithm. Feature engineering deals with the extraction of the most informative features, from the most informative sensors, that can accurately capture the introduced fault. To perform diagnosis, the classification model is trained using experimentally acquired data and is then tested on a separate data set. The framework was able to achieve high classification accuracy (>99%) for training size as low as 30% of the total recorded observations. The low number of features required to achieve this accuracy suggests high potential for integration into existing onboard condition monitoring systems.

data driven methods, fault detection, swirl flames↗
DOE OSTI2023

Engineering-Scale Integrated Energy System Data Projection Demonstration via the Dynamic Energy Transport and Integration Laboratory

The objective of this study is to demonstrate and validate the Dynamic Energy Transport and Integration Laboratory (DETAIL) preliminary scaling analysis using Modelica language system-code Dymola. The DETAIL preliminary scaling analysis includes a multisystem integral scaling package between thermal-storage and hydrogen-electrolysis systems. To construct the system of scaled equations, dynamical system scaling (DSS) was applied to all governing laws and closure relations associated with the selected integral system. The existing Dymola thermal-energy distribution system (TEDS) facility and high-temperature steam electrolysis (HTSE) facility models in the Idaho National Laboratory HYBRID repository were used to simulate a test case and a corresponding scaled case for integrated system HYBRID demonstration and validation. The DSS projected data based on the test-case simulations and determined scaling ratios were generated and compared with scaled case simulations. The preliminary scaling analysis performance was evaluated, and scaling distortions were investigated based on data magnitude, sequence, and similarity. The results indicated a necessity to change the normalization method for thermal storage generating optimal operating conditions of 261 kW power and mass flow rate of 6.42 kg/s and the possibility of reselecting governing laws for hydrogen electrolysis to improve scaling predictive properties. To enhance system-scaling similarity for TEDS and HTSE, the requirement for scaling validation via physical-facility demonstration was identified.

08 HYDROGEN↗
DOE OSTI2021

DART-PFLOTRAN: An ensemble-based data assimilation system for estimating subsurface flow and transport model parameters

Ensemble-based Data Assimilation (EDA), based on the Monte Carlo approach, has been effectively applied to estimate model parameters through inverse modeling in subsurface flow and transport problems. However, implementation of EDA approach involves a complicated workflow that include setting up and executing ensemble forward model simulations, processing observations and model simulation results for parameter updates, and repeat for sequential or iterative EDA. To facilitate the management of such workflow and lower the barriers for adopting EDA-based parameter estimation in subsurface science, we develop a generic software frame-work linking the Data Assimilation Research Testbed (DART) with a massively parallel subsurface FLOw and TRANsport code PFLOTRAN. The new DART-PFLOTRAN leverages both the core data assimilation engines in DART and the computational power afforded by PFLOTRAN. In addition to the standard smoother and filtering options, DART-PFLOTRAN enables an iterative EDA workflow based on the Ensemble Smoother for Multiple Data Assimilation method (ES-MDA) to improve estimation accuracy for nonlinear forward problems. Here, we verify the implementation of ES-MDA in DART-PFLOTRAN using two synthetic cases designed to estimate static permeability and dynamic exchange fluxes across the riverbed, respectively, from continuous temperature measurements made across a depth profile. One-dimensional hydro-thermal simulations are performed in both cases to relate temperature responses with the parameters of interest. In the case of estimating dynamic parameters, we demonstrate the flexibility of DART-PFLOTRAN in automating sequential ES-MDA workflow, which will significantly reduce the time researchers spend on managing complex workflows in similar applications. Both studies yield accurate estimations of the parameters compared to their synthetic truth, while ES-MDA leads to more accurate estimation when a high level of nonlinearity exist between observed responses and unknown parameters. With a code base in Python and Fortran, DART-PFLOTRAN paves the way for applications in large-scale subsurface inverse modeling by automating the complex workflow of sequential ES-MDA that can be executed on various computing platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

The utility of Bayesian data reconciliation for separations

Data reconciliation methods for separation processes typically rely on classical statistical approaches to generate estimates of true mass flow rates from measurements. Knowledge regarding the uncertainty of these estimates has value in decision making, but is often not acquired. Bayesian approaches intrinsically quantify uncertainty; however, literature for Bayesian data reconciliation of separation processes is scarce. This publication outlines two Bayesian data reconciliation models and provides details for how the models were implemented for the BayesMassBal (V 1.0.0) software package written in R. To demonstrate the advantages of this approach for data reconciliation, the models were first applied to simulated data and then compared to a classical model through a Monte Carlo experiment. In this example, the Bayesian models were found to provide more accurate estimates of the simulated data, while also providing quantitative information on the estimate uncertainty. To demonstrate the use of the technique in a practical problem, the models were also applied to real data collected from a pilot-scale rare earth solvent extraction process. Here, this publication provides a small window into how Bayesian methods can be used for data reconciliation, but findings suggest Bayesian data reconciliation models for separation processes have distinct advantages over classical alternatives.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2021

Online Detection of Low-Quality Synchrophasor Data Considering Frequency Similarity

Here, this letter proposes a new approach for online detection of low-quality synchrophasor data under both normal and event conditions. The proposed approach utilizes the features of synchrophasor data in time and frequency domains to distinguish multiple regional PMU signals and detect low-quality synchrophasor data. It is more effective to detect low-quality data with apparently indistinguishable profiles. Case studies from recorded synchrophasor measurements verify the effectiveness of the proposed approach for detecting low-quality synchrophasor data in frequency, voltage magnitude and voltage angle.

42 ENGINEERING↗