Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Feature engineering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Feature Engineering and Ensemble Methods for Imbalanced ICS Intrusion Detection: Pipeline Audit and Constrained Evaluation

Industries are becoming increasingly connected and are more vulnerable to cyberattacks due to the widened attack surface. Industrial Control Systems (ICS) are among the most critical sectors that malicious actors can target, as such attacks can cause significant operational disruption and physical damage. It is imperative to detect such attacks as early as possible. This paper evaluates constraint-conditioned optimistic performance estimates for traditional ML models in ICS intrusion detection (i.e., estimates obtained under contiguous, non-shuffled temporal evaluation without test-set alteration, but with pre-split feature engineering that may introduce temporal leakage, due to dataset constraints). Our findings are threefold. First, we quantify how iterative feature engineering affects tree-based ensemble performance and examine how pipeline decisions (split strategy, sampling scope, and cleaning policy) can inflate or reduce reported IDS results under constraint-bound evaluation. Second, we compare intrinsic class-imbalance handling across ensemble models. Third, under our current pipeline constraints (including pre-split feature engineering), CatBoost achieves the best performance on Water Storage Tank (accuracy: 0.9831, class-1 F1: 0.9682), while Light- GBM achieves the best performance on Gas Pipeline (accuracy: 0.9618, class-1 F1: 0.9086).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Feature engineering descriptors, transforms, and machine learning for grain boundaries and variable-sized atom clusters

Abstract Obtaining microscopic structure-property relationships for grain boundaries is challenging due to their complex atomic structures. Recent efforts use machine learning to derive these relationships, but the way the atomic grain boundary structure is represented can have a significant impact on the predictions. Key steps for property prediction common to grain boundaries and other variable-sized atom clustered structures include: (1) describing the atomic structure as a feature matrix, (2) transforming the variable-sized feature matrix to a fixed length common to all structures, and (3) applying a machine learning algorithm to predict properties from the transformed matrices. We examine how these steps and different combinations of engineered features impact the accuracy of grain boundary energy predictions using a database of over 7000 grain boundaries. Additionally, we assess how different engineered features support interpretability, offering insights into the physics of the structure-property relationships.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Enhancing dimensionality prediction in hybrid metal halides via feature engineering and class-imbalance mitigation

We present a machine learning (ML) framework for predicting the structural dimensionality of hybrid metal halides (HMHs), including organic-inorganic perovskites, using a combination of chemically-informed feature engineering and advanced class-imbalance handling techniques. This study is motivated by the small and highly imbalanced nature of experimentally available HMH datasets, which limits the applicability and reliability of conventional ML approaches. The dataset, consisting of 494 HMH structures, is highly imbalanced across dimensionality classes (0D, 1D, 2D, 3D), posing significant challenges to predictive modeling. To mitigate this limitation, the dataset was augmented to 1336 samples using the synthetic minority oversampling technique, enabling improved learning of underrepresented dimensionality classes while preserving chemically meaningful feature relationships. We developed interaction-based descriptors designed to capture coupled steric and polarity effects relevant to dimensionality prediction, which are not readily captured by standard single-parameter or composition-only descriptors. These descriptors are integrated into a multi-stage workflow combining feature selection, ensemble stacking, and performance optimization. Our approach significantly improves F1-scores for underrepresented classes, achieving robust cross-validation performance across all dimensionalities. This work demonstrates a generalizable strategy for extracting reliable and interpretable structure–dimensionality relationships from limited experimental data, enabling pre-synthesis screening of organic cations and providing a practical blueprint for small-data ML in hybrid materials systems.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Data and Code for Understanding Generative AI Content with Embedding Models

This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).

Vargas, Max [Pacific Northwest National Laboratory↗
DOE OSTI2025

Performance evaluation of automated data-driven feature extraction and selection methods for practical and scalable building energy consumption prediction models

Here, this study quantifies the impact of automated feature engineering methods (feature extraction and selection) on the quality and accuracy of machine learning models that predict building energy consumption. The case study compares model performance for three main scenarios: baseline (no feature extraction and selection), feature extraction only, and feature extraction combined with feature selection (filter and/or wrapper methods) for fully trained machine learning models for 200 metered/sub-metered energy measurements across 118 real buildings. For consistency, the same machine learning model architecture (a black box deep learning neural network with probabilistic forecast output) was used for all scenarios. Based on results, all feature engineering methods provided noticeable prediction accuracy improvements (e.g., 29%-68% median prediction improvement) compared to baseline scenarios. However, in this application, feature selection methods provide little practical value due to their limited performance gains and high computational cost. Smarter algorithm development supported by better computational environments will be needed before feature selection methods can reliably and efficiently improve predictive model performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Performance Comparison of Machine Learning Models for Ultrasonic Nondestructive Evaluation of Alkali-Silica Reaction in Concrete

Alkali-silica reaction (ASR) causes concrete degradation, leading to cracking, rebar corrosion, and reduced structural integrity, which raises safety concerns. Ultrasonic nondestructive evaluation (NDE) effectively assesses concrete properties and monitors ASR progression. However, its deployment and analysis require specialized expertise and subjective interpretation. As computational power increases, artificial intelligence (AI) and machine learning (ML) algorithms are increasingly being used to automate NDE data analysis across various industries for AI-assisted automation. Regulatory agencies are adapting to this technological shift, prompting a need to evaluate current ML technologies’ capabilities and limitations in assessing concrete material properties and damage. This report presents a comparative analysis of four ML regression models for predicting concrete material damage induced by ASR expansion using long-term ultrasonic data monitoring. The models investigated include linear regression (LR), support vector regression (SVR), shallow neural networks (NN), and deep neural networks (DNN). LR, SVR, and shallow NN models use features extracted from ultrasonic signals, whereas the DNN model processes time-domain ultrasonic signals and frequency spectra directly. The study systematically compared the models’ performance from various perspectives, including model input, prediction performance, and generalization ability. The findings indicate significant variability in model performance, with some ML algorithms achieving very high or very low prediction accuracy depending on the preprocessing and feature engineering (extraction and selection) applied. Key insights include the observation that shallow ML models (LR, SVR, and shallow NNs) require meticulous preprocessing and feature extraction to achieve high accuracy. In contrast, the DNN model, although it bypasses the need for feature engineering, necessitates extensive preprocessing to mitigate noise and computational demands. The SVR model emerged as the top performer among the shallow models, and the DNN model exhibited superior performance on specific datasets but struggled with generalization across specimens from different batches. Additionally, the SVR model is sensitive to temperature variations, whereas the DNN model is robust in this regard. Using recurrent neural networks is recommended for future ASR expansion prediction studies. Recurrent neural networks’ inherent ability to capture temporal dependencies and long-term patterns makes them well suited for analyzing sequential ultrasonic monitoring data. Overall, the results and conclusions of this study could provide insights into the capabilities and effectiveness of ML when applied to ultrasonic NDE data and help identify best practices for using ML for ultrasonic NDE of concrete material properties.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Understanding Generative AI Content with Embedding Models

The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.

Vargas, Max↗
DOE OSTI2024

Targeted Biomining and Machine Learning Approaches in Critical Minerals Revealed by a Biogeochemical Survey of a Coal Mine Drainage Remediation System

Abandoned coal mine drainage (AMD) remediation systems in Pennsylvania can concentrate critical minerals and materials (CMM) at levels comparable to mining-grade ores. Remediation systems have varying engineering features and are open to the environment, resulting in diverse microbial colonization and seasonal climate influences that may impact CMM speciation. The location of CMMs, the types of bacterial communities tolerant of these pollutant conditions, and the influence of localized climate on CMM rich remediation systems are not well characterized. Through a one-year spatiotemporal survey of biogeochemistry at a remediation system, we have initiated the process to address these questions. Rare Earth Elements (REE) ranged 180-1,200 ppm and greater than 1,500 bacterial ASVs were classified via 16S sequencing. Analyses indicate biogeochemical differences are heavily influenced by engineering features. Additionally, REE precipitants correlate strongly with the elements Al, Cu, Zn, Be, and U. Unearthing these trends has refined our line of inquiry to explore biological mining opportunities more closely with these metals. Furthermore, we created a Machine Learning Model for predicting AMD REE content, with 89% accuracy, using the data from this study and several others. Further training data is required to create a more reputable model. Recently, global research efforts have prioritized modeling work or the use of the few historical surveys to design experiments. Through our data, we challenge this approach, emphasizing the importance of expanding fundamental survey efforts prior to advanced product design and experimentation.

critical minerals↗
DOE OSTI2025

TRUST Sensors in Environments: Accelerometers (SE-A) Report Release FY25

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainty in Simulations and Tests (TRUST) project is to quantify and help increase confidence in specific areas of computa tional and experimental capabilities that are applicable to the development, on-target assessment, and qualification of current and future delivery environments. The TRUST project consists of five "low-complexity", single-feature testbeds used to conduct experiments with accompanying models and simulations. Each of the five single feature testbeds aims to isolate an engineering feature or behavior of interest, then work towards improving the fundamental engineering understanding of that feature. Additionally, the single feature testbeds are used to identify capability development needs that can help reduce model and experimental uncertainty.

42 ENGINEERING↗
DOE OSTI2024

TRUST Sensors in Environments: Accelerometers (SE-A) Report, Release FY24

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainty in Simulations and Tests (TRUST) project is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to the development, on-target assessment, and qualification of current and future delivery environments. The TRUST project consists of five "low-complexity", single-feature testbeds used to conduct experiments with accompanying models and simulations. Each of the five single feature testbeds aims to isolate an engineering feature or behavior of interest, then work towards improving the fundamental engineering understanding of that feature. Additionally, the single feature testbeds are used to identify capability development needs that can help reduce model and experimental uncertainty.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) v1

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) is a comprehensive data visualization and analysis application focused on working with COLTRIMS (COLd Target Recoil Ion Momentum Spectroscopy) data, which is used in atomic and molecular physics experiments. The application offers several powerful features: - Data uploading and processing capabilities for COLTRIMS files - Multiple visualization methods using UMAP (Uniform Manifold Approximation and Projection) for dimensionality reduction - Interactive selection of data points across multiple views - Feature engineering through various methods: - Manual feature selection from calculated physics parameters - Deep autoencoder for dimension reduction - Genetic programming for discovering meaningful features - Mutual information-based feature selection - Multiple clustering approaches (DBSCAN, KMeans, Agglomerative) - Quality metrics for evaluating clustering results - Export capabilities for selections and generated features

Daoud, Hazem [Lawrence Berkeley National Laborator↗
DOE OSTI2026

Modeling Multi-View Impedance-Based Cross-Geometry SOH Estimator for Li-ion Batteries

Abstract: Accurately estimating battery’s State of Health (SOH) remains challenging when models must generalize across cell designs and operating conditions. Most Electrochemical Impedance Spectroscopy (EIS)-based approaches either (i) hand-engineer a few Nyquist-plot features for shallow models—fast but does not generalize across geometries—or (ii) learn directly from Nyquist plots with deep networks, which removes manual feature extraction, yet still limited to a single plot type. As a result, cross-geometry robustness and deployability on constrained Internet of Things (IoT) devices remain open problems. We propose a compact Convolutional Neural Network (CNN) (∼ 10k parameters) that takes multi-representation EIS inputs—Nyquist (real/imaginary) and phase–magnitude (|Z|/ϕ) stacked as four channels, so the model can learn complementary degradation signatures while remaining small enough for fast inference. We build a dataset from cyclic aging of two geometries (LG INR18650MJ1 cylindrical cells and LIR2032 coin cells), acquire EIS every ten cycles from 10 kHz to 10 mHz (10 points/decade), and evaluate with leave-one-cell-out testing strategy. We further study fusion vs. single-representation inputs and assess feasibility for on-device deployment (e.g., NVIDIA Jetson device). The results show that training on multiple EIS representations improves SOH estimation accuracy and cross-geometry generalization compared to single-representation models, which uses only Nyquist or phase–magnitude plots. This design targets accurate, generalizable SOH prediction without manual feature engineering while enabling practical real-time use.

Bakr, Ahmed [The University of Alabama (UA)]↗
DOE OSTI2024

Automated Framework for Groundwater Monitoring Using DWT with LSTM and Transformers

Environmental monitoring is critical for safeguarding public health and ecological well-being. Traditional data structuring and workflow monitoring methods consume significant time and effort, hindering timely insights and effective decision-making. Our study addresses this challenge by presenting an AI framework that automates data cleaning, structuring, and modeling processes, specifically targeting applications in groundwater monitoring. By leveraging automation for data processing and model training, our framework establishes a novel and efficient paradigm for environmental monitoring, with its potential application to the vast network of over a hundred Department of Energy Environmental Management (DoE-EM) cleanup sites across the country. It analyzes data streams from a network of groundwater Internet-of-Things (IoT) sensors deployed at the Savannah River Site (SRS) for prediction modeling. This allows human experts to focus on analysis and decision-making, ultimately leading to better environmental outcomes.The framework employs multivariate time-series forecasting methods to study and model the behavior of varying chemical analytes. The continuous learning process is enabled by utilizing deep learning techniques. It allows the framework to become more nuanced in its analysis over time, adapting to the specific characteristics of the environmental site and the evolving nature of contaminant behavior. Deep learning models known for sequence modeling, LSTM, and Transformers are employed for time series forecasting. Data processing and structuring are essential components significantly impacting the final model's performance. This hypothesis was proven by presenting a comparative analysis of model performance with processed and unprocessed data. The feature engineering approach utilized was the Discrete Wavelet Transform, which works well with time series data.

Discrete Wavelet Transform (DWT)↗
DOE OSTI2026

Early calendar life and health prediction of silicon batteries via machine learning with uncertainty quantification

Lithium-ion batteries with silicon anodes promise high energy density but are limited by calendar lifetime. Reducing the long iteration time to obtain experimental results requires predicting calendar lifetime early in a cell's life. In this study, we demonstrate that lightweight machine learning models with feature engineering can provide calendar lifetime estimates from early electrochemical signals. After 1 month of electrochemical aging, the best models achieve 10% error in calendar-life prediction and can separate "bad" from "good" lifetime cells with a mean F1 score of 0.857. As battery systems exhibit inherent variability, four methods for uncertainty quantification are compared, and confidence intervals are demonstrated with an uncertainty of +-3.6 months in lifetime prediction. A feature importance analysis indicates that early patterns in voltage decay are the strongest indicators of calendar lifetime. Finally, this modeling approach has high error when generalizing to new electrode chemistries or testing conditions but with appropriately low confidence.

25 ENERGY STORAGE↗
DOE OSTI2025

Machine Learning Analysis of Temperature-Strain Relationships for Structural Health Monitoring of Pipes: Self-powered wireless sensor system for health monitoring of liquid-sodium cooled fast reactors

This report presents machine learning (ML) analysis of temperature-strain relationships for structural health monitoring of nuclear reactor stainless steel (SS) pipes with the strain gauge sensor directly printed on the pipe with a 3D conformal aerosol jet printer. We investigate correlations for two sensor pairs installed on the same SS304 pipe: commercial K-type thermocouple with a printed gold strain gauge (TC3-SG3), and commercial K-type thermocouple with commercial Kyowa strain gauge (TC0-SG0). The temperature ranges for the sensor pairs TC0-SG0 and TC3-SG3 are 20.00°C to 266.37°C and 39.95°C to 219.28°C respectively. ML algorithms in this study include Linear Regression (baseline method), Ridge Regression, Lasso Regression, and Gradient Boosting. Performance evaluation metrics include Root Mean Square Error (RMSE), Mean Square Error (MSE), Mean Absolute Error (MAE), R 2 Score, and Explained Variance. Using advanced feature engineering techniques, we extracted 27 temperature-based features and 30 strategic inclusion features. The best performance was obtained with the Gradient Boosting method, which achieves prediction accuracy of R 2 = 0.9999 and RMSE = 7.69 μStrain for TC0-SG0, and R 2 = 0.9998 and RMSE = 18.03 μStrain for TC3-SG3. While the temperature-strain correlations are weaker for the gauge directly printed on the pipe than for the commercial strain gauge, deployment-ready performance exceeding industry standards is achieved for both sensor pairs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Edge ML for CAN bus intrusion detection in AVs

Autonomous Vehicles (AVs) are revolutionizing transportation, but their reliance on interconnected cyber-physical systems exposes them to unprecedented cybersecurity risks. This study addresses the critical challenge of detecting real-time cyber intrusions in self-driving vehicles by leveraging a dataset from the Udacity self-driving car project. We simulate four high-impact attack vectors, Denial of Service (DoS), spoofing, replay, and fuzzy attacks, by injecting noise into spatial features (e.g., bounding box coordinates) to replicate adversarial scenarios. We develop and evaluate two lightweight neural network architectures (NN-1 and NN-2) alongside a logistic regression baseline (LG-1) for intrusion detection. The models achieve exceptional performance, with NN-2 attaining an AUC score of 93.15% and 93.15% accuracy, demonstrating their suitability for edge deployment in AV environments. Through explainable AI techniques, we uncover unique forensic fingerprints of each attack type, such as spatial corruption in fuzzy attacks and temporal anomalies in replay attacks, offering actionable insights for feature engineering and proactive defense. Visual analytics, including confusion matrices, ROC curves, and feature importance plots, validate the models' robustness and interpretability. This research sets a new benchmark for AV cybersecurity, delivering a scalable, field-ready toolkit for Original Equipment Manufacturers (OEMs) and policymakers. By aligning intrusion fingerprints with SAE J3061 automotive security standards, we provide a pathway for integrating machine learning into safety-critical AV systems. Our findings underscore the urgent need for security-by-design AI, ensuring that AVs not only drive autonomously but also defend autonomously. This work bridges the gap between theoretical cybersecurity and life-preserving engineering, offering a leap toward safer, more secure autonomous transportation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Leveraging large language models to address data scarcity in machine learning for graphene synthesis

Machine learning in experimental materials science faces significant challenges due to the scarcity of data, which are costly and time-consuming to generate, particularly when relying on in-house experiments. Literature data mining offers a potential solution but introduces issues like mixed data quality, inconsistent formats, and non-uniform reporting of synthesis parameters, resulting in partially missing and heterogeneous features across the dataset. Here, we propose data imputation and feature engineering methods that employ pre-trained large language models (LLMs) to enhance machine learning performance on scarce, heterogeneous datasets, demonstrated on graphene CVD synthesis data and the ML-HydPARK hydrogen storage dataset. GPT models perform data imputation via tailored prompting and semantic normalization of inconsistently reported features through embeddings, for example, to harmonize the complex nomenclature of CVD substrates. Beyond yielding more diverse and richer feature representations than traditional methods such as K-nearest neighbors (KNN) and Multivariate Imputation by Chained Equations (MICE), LLM-based data imputation is evaluated against dataset characteristics and prompting strategies. We vary the level of autonomy granted to the LLM, from generic prompting that leverages pre-trained knowledge for autonomous data generation to data-informed prompting that constrains outputs using target-specific information, and demonstrate which level of autonomy yields superior imputation performance across datasets and feature types. The proposed data engineering methods markedly improve downstream performance; for example, in graphene layer number classification using a support vector machine (SVM), binary accuracy increases from 39% to 65% and ternary accuracy from 52% to 72%. Fine-tuning experiments on both datasets show that combining our proposed LLM-based data imputation and feature encoding methods with numerical machine learning predictors outperforms standalone fine-tuned LLM predictors in data-scarce settings. The proposed strategies emphasize data enhancement techniques rather than refining learning architectures or regularizing loss functions, offering a broadly applicable framework for improving machine learning performance on scarce, inhomogeneous datasets.

Chemical vapor deposition↗
DOE OSTI2025

Techno-economic analysis of sugarcane bagasse and straw conversion into cellulosic ethanol via consolidated bioprocessing

Cellulosic biofuels offer a sustainable alternative to fossil fuels and a means to mitigate climate change. Consolidated bioprocessing (CBP) featuring engineered thermophilic bacteria, combined with mechanical disruption during fermentation (cotreatment), has potential to lower production costs compared to featuring thermochemical pretreatment and added cellulase. A techno-economic analysis was conducted (230 million L ethanol/year) from sugarcane bagasse and straw at stand-alone facilities generating electricity from residues. Three scenarios were evaluated: Conventional, featuring hydrothermal pretreatment, fungal cellulase, and yeast fermentation (current commercial standard); Mid-term CBP, relying on bagasse solubilization without pretreatment or cotreatment; and Mature CBP, incorporating cotreatment but no pretreatment. Results for these scenarios in this order were: fixed capital investment (CapEx) $\$$589M, $\$$658M, and $\$$472M; net annual revenue (EBITDA) $\$$56M, $\$$96M, and $\$$94M; and minimum ethanol selling price 0.73, 0.61, and 0.48 US$\$$/L. Payback periods were 10.5, 6.9, and 5.1 years, while all scenarios showed <5 years at European prices for scales >100M L/year. Sensitivity and risk analysis highlighted ethanol price as the most critical variable. It is notable that Mid-term CBP had shorter payback times and better overall economic feasibility compared to Conventional. Our results underscore opportunities for research-driven innovation on low-cost cellulosic ethanol technologies in Brazil and elsewhere.

Consolidated bioprocessing↗