Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data normalization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗
DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗
DOE OSTI2026

2025 Annual INMM Graph and Tables for High Purity Germanium Detector Normalization Presentation

The data set includes gamma spectroscopy peak data for measurements taken with two different high purity germanium detectors using a mixed nuclide source and a U-235 fuel rod. There are a total of 5 specific energy peaks that were analyzed for the mixed nuclide source stemming from Am-241, Cd-109, Cs-137, and Co-60. There are a total of 3 specific energy peaks that were analyzed for the U-235 fuel rod. The data set includes the calculations and results from using a linear correction factor, absolute efficiency curve, and relative efficiency curve to compare the net peaks counts from two different detectors.

Drumm, Natalie Daphne [Sandia National Laboratorie↗
DOE OSTI2025

Raspberry Pi–powered temperature monitoring of growth chamber microclimates

While controlled environments are desirable for growing and measuring plants, growth chambers and greenhouses typically have microclimates that impact plant growth, development, and stress responses. Furthermore, opening and closing the doors of a controlled environment introduces variation in the environment, especially at temperature extremes, affecting both the measurements and the organisms within. Using multiple temperature data loggers to normalize results can be cost-prohibitive and rarely offers real-time feedback on temperature status. We used low-cost single-board computers, cameras, and temperature sensors to manage and capture growth chamber temperatures while acquiring plant image data. Detailed here are methods to document microclimates within a growth chamber so that data can be normalized to measured temperature information. This protocol describes a low-cost method for automated temperature monitoring, which enables both high-throughput measurements of temperature along with plant growth and stress responses via plant imaging.

Plant Sciences↗
DOE OSTI2025

Benchmarking Concentration and Extraction Methods for Wastewater-Based Surveillance of Eight Human Respiratory Viruses: Implications for Rapid Application to Novel Pathogens

To provide early warning and support a rapid response to a novel virus through wastewater surveillance, it would be ideal to understand in advance which concentration and extraction methods are likely to be effective for dPCR-based methods, depending on virus characteristics. In this study, we spiked raw wastewater samples with eight human respiratory viruses and processed them with four methods that concentrate and/or extract nucleic acids from both liquid and solid fractions (Promega, Nanotrap, and InnovaPrep) or only the solid fraction of wastewater (Solids). Our findings provide encouraging evidence that all four methods combined with dPCR could detect an emerging virus in wastewater, although they differed in sensitivity. The pattern of recovery efficiency for adenoviruses, coronaviruses, and influenza A viruses was consistent across methods, with Promega producing higher median recovery efficiencies, while distinct patterns were observed for coxsackieviruses. We also normalized the concentration data with two endogenous fecal indicators, PMMoV and Carjivirus (formerly crAssphage). We found that normalization could reduce method-associated differences if the indicator exhibited a recovery pattern similar to that of the target virus. These findings can guide the selection of concentration and extraction methods for wastewater monitoring based on the properties of target viruses, thus enhancing pandemic preparedness.

Biological and medical sciences↗
DOE OSTI2025

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗
DOE OSTI2026

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗
DOE OSTI2026

Temporal Convolutional Network Using Empirical Mode Decomposition to Detect Faults in Grid Connected Systems

Grid-connected power electronic systems require timely and reliable fault detection to prevent equipment damage and reduce downtime. This paper presents a forecasting-based anomaly detection pipeline that decomposes voltage and current measurements into intrinsic mode functions (IMFs) using empirical mode decomposition (EMD), then trains a causal temporal convolutional network (TCN) on normal-operation IMF data to predict short-horizon future dynamics. Deviations between forecasts and observations are summarized as reliability-weighted residual scores and thresholded per sensor using robust statistics with temporal persistence constraints to suppress false positives. To reduce runtime, EMD is performed on downsampled signals for detection, while raw-rate EMD is applied only within a short region of interest for high-frequency interpretability near detected events. Results on a simulated grid-connected converter system demonstrate that IMF-domain forecasting improves anomaly separability relative to raw-signal forecasting and provides interpretable evidence of faults across decomposition channels.

Sutton, Elizabeth [ORNL] (ORCID:0009000078885935)↗
DOE OSTI2026

APOLLO: a facility-scale differentiable virtual accelerator for Fermilab

As the design complexity of modern accelerators grows, there is more interest in using advanced simulations that have fast execution time or yield additional insights like gradients. The FAST/IOTA facility has been working on implementing and experimentally validating an end-to-end digital twin that is both fast and gradient-aware, allowing for rapid prototyping of new software and experiments with minimal beam time costs. Our framework integrates physics and ML codes for linac and ring simulation through a set of generic interfaces between surrogate and physics-based sections. To reproduce device inputs and outputs, system state is exposed as a deterministic discrete event simulator. Because Fermilab is undergoing control system transition, both EPICS and ACNET frontends are supported. Recently, we have begun transitioning to a new community lattice standard, PALS, as well as developing standardized infrastructure for data ingest and normalization to prepare for model calibration during FAST proton injector commissioning. We discuss implementation details as well as challenges, and future plans to extend modelling to main complex proton accelerators like PIPII and Booster.

Kuklev, Nikita [Fermilab]↗
DOE OSTI2024

Applying Machine‐Learning Methods to Laser Acceleration of Protons: Lessons Learned From Synthetic Data

ABSTRACT In this study, we consider three different machine‐learning methods—a three‐hidden‐layer neural network, support vector regression, and Gaussian process regression—and compare how well they can learn from a synthetic data set for proton acceleration in the Target Normal Sheath Acceleration regime. The synthetic data set was generated from a previously published theoretical model by Fuchs et al. 2005 that we modified. Once trained, these machine‐learning methods can assist with efforts to maximize the peak proton energy, or with the more general problem of configuring the laser system to produce a proton energy spectrum with desired characteristics. In our study, we focus on both the accuracy of the machine‐learning methods and the performance on one GPU including memory consumption. Although it is arguably the least sophisticated machine‐learning model we considered, support vector regression performed very well in our tests.

Desai, Ronak↗
DOE OSTI2026

Dataset for "A primer on forest structure measurement with lidar for ecologists"

This repository includes data and code accompanying the case study included in the manuscript "A primer on forest structure measurement with lidar for ecologists" (submitted to Ecosphere). We compiled lidar datasets from multiple platforms in a common area to: 1. Demonstrate how differences in sensor characteristics influence density and resolution of lidar data. 2. Provide open-source, co-located datasets for users to further inspect differences in lidar data. 3. Provide example code to perform basic lidar analysis. This case study is meant to allow readers to get hands-on experience with real-world data from different platforms. This case study is not meant to be a rigorous comparison of derived ecological metrics among all sensors; such comparisons can be found throughout other publications referenced throughout the main manuscript. Code includes basic functions in R commonly used to visualize and manipulate lidar data accessible with a normal laptop computer; more sophisticated algorithms for advanced users are also referenced throughout the main manuscript. Terrestrial laser scanning (TLS), mobile laser scanning (MLS), UAS laser scanning (ULS), airborne laser scanning (ALS), and spaceborne laser scanning (SLS) data were collected within the Smithsonian Environmental Research Center (SERC) forest dynamics plot in Maryland, USA. TLS, MLS, and ALS data were collected within 1 month of the 2021 growing season; ULS data were collected in November 2020 (“leaf-off” data) and July 2022 (“leaf-on” data).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

PNNL-Predictive-Phenomics/ProteoMeter

ProteoMeter is a Python package that assists in the statistical analysis of global proteomics, protein post-translation modification (PTM), and limited proteolysis (LiP) data. It contains batch correction, normalization, and statistical testing methods, as well as functions that "roll up" peptide-level data to the single-site level. It has a robust user configuration system, allowing it to flexibly integrate different types of experiment designs. For basic usage, a simple configuration file provides the essential functionality. Advanced users have access to the entire statistical pipeline for fine-tuning analyses. Processed data is easily exported to many common spreadsheet and data-frame formats.

Rozum, Jordan [Pacific Northwest National Lab]↗
DOE OSTI2025

Generative learning of densities on manifolds

A generative modeling framework is proposed that combines diffusion models and manifold learning to efficiently sample data densities on manifolds. The approach utilizes Diffusion Maps to uncover possible low-dimensional underlying (latent) spaces in the high-dimensional data (ambient) space. Two approaches for sampling from the latent data density are described. The first is a score-based diffusion model, which is trained to map a standard normal distribution to the latent data distribution using a neural network. The second one involves solving an Itô stochastic differential equation in the latent space. Additional realizations of the data are generated by lifting the samples back to the ambient space using Double Diffusion Maps , a recently introduced technique typically employed in studying dynamical system reduction; here the focus lies in sampling densities rather than system dynamics. The proposed approaches enable sampling high dimensional data densities restricted to low-dimensional, a priori unknown manifolds. The efficacy of the proposed framework is demonstrated through a benchmark problem and a material with multiscale structure.

Double diffusion maps↗
DOE OSTI2025

The HydroBio Dataset: a new data resource for evaluating existing and potential hydropower capacity and freshwater biodiversity in the conterminous United States

Hydropower is a critical source of affordable and reliable electricity and energy system stability services in the United States. Opportunities to expand US hydropower production include retrofitting existing non-powered dams to produce power, retrofitting existing hydropower dams to improve efficiency or increase capacity, or constructing new hydropower infrastructure on currently unregulated river reaches. We created the HydroBio Dataset, which summarizes existing and potential hydropower capacity and freshwater biodiversity at the sub-basin scale in the conterminous US to contextualize existing and potential grid contributions with the freshwater ecosystems in which dams are situated. We demonstrate a use-case of this dataset by rescaling and comparing potential non-powered dam nominal capacity to rarity-threat-weighted freshwater species richness for sub-basins where both types of data exist. On average, normalized freshwater biodiversity exceeded normalized potential non-powered dam nominal capacity in these sub-basins. Potential non-powered dam nominal capacity was concentrated in sub-basins in the Upper Mississippi and Ohio hydrologic regions while freshwater biodiversity was concentrated in the South Atlantic-Gulf, Ohio, and Tennessee hydrologic regions. Additionally, non-powered dams and existing hydropower dams are located in sub-basins with similar indices of freshwater biodiversity. The HydroBio Dataset adds an additional ecological dimension of context to our understanding of current and potential future US hydropower capabilities and is a valuable decision support tool for stakeholders tasked with balancing gains in services to the US power grid with the public and environmental benefits of freshwater ecosystems.

Biodiversity↗
DOE OSTI2026

Documentation, assessment, and improvements of fission gas modeling capabilities in BISON

Fission gas behavior critically affects nuclear fuel performance, influencing thermal conductivity, microstructure, swelling, and fuel-cladding interactions. The Simple Integrated Fission Gas Release and Swelling (Sifgrs) model, implemented within the BISON fuel performance code, provides advanced capabilities for modeling these phenomena. This manuscript reviews and assesses Sifgrs’ modular structure and submodels describing intragranular, intergranular, and release mechanisms. The modular implementation enables rapid improvements as our understanding advances and new fuel forms emerge. After an overview of capabilities, we present a comprehensive validation suite, comparing predictions to experimental data from tests under normal and transient conditions, including reproduction of the Vitanza curve. Results demonstrate Sifgrs’ accuracy, robustness, and applicability. Current limitations and ongoing efforts are discussed, including mechanistic, multiscale modeling, uncertainty quantification, and validation within the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, supporting the safe and efficient use of nuclear fuels.

BISON↗
DOE OSTI2026

Performance Modeling of Tandem Photovoltaics: A Yearlong Outdoor Degradation Analysis of a Ga⁢As//Si Minimodule

We present a performance modeling and degradation analysis framework for tandem photovoltaic modules, building upon established procedures for crystalline silicon devices and adapting them to account for the spectral sensitivity of multijunction technologies. The methodology employs filter criteria to select outdoor measurements close to standard test conditions (STC) under stable spectral and ambient conditions, followed by normalization of power production data with corrections for temperature, irradiance, and precipitable water vapor. We demonstrate this framework using a mechanically stacked four-terminal gallium arsenide (Ga⁢As) // silicon (Si) tandem solar minimodule deployed outdoors from October 2019 to January 2021 in Golden, Colorado, USA. We determined degradation rates of −4.1 ±0.2%/year for the Ga⁢As subcell and −2.5 ±0.9%/year for the Si subcell, with analysis of individual performance metrics indicating that packaging degradation, particularly delamination, was the dominant failure mode. Simulations using PVcircuit, an open-source equivalent-circuit solver, confirmed these findings. The presented methodology provides a reproducible foundation for performance modeling and degradation analysis of emerging tandem technologies.

14 SOLAR ENERGY↗
DOE OSTI2025

Deep Factorization Machine Learning for Disaggregation of Transmission Load Profiles with High Penetration of Behind-The-Meter Solar

The ever-growing integration of distributed energy resources (DERs), especially behind-the-meter (BTM) solar generations, poses imperative operational challenges to system operators such as regional transmission organizations (RTOs). It is important for RTOs to effectively and accurately extract actual load profiles at the transmission level for a single node with significant BTM solar injection. This paper first illustrates the necessity of disaggregating the daily actual load profile of a single node. Furthermore, by segmenting nodes with selected timeseries features, nodes with significant BTM solar generation are identified. Lastly, a bi-level framework is proposed, comprising reference node disaggregation and DeepFM nodal disaggregation, aimed at disaggregating the nodal load profiles from which system operators require more information. By adopting a hybrid Deep Factorization Machine (DeepFM) model, the model achieve accurate results by extracting both linear and nonlinear relations between nodes in the same region and the zonal load and nodal load profile. To overcome the lack of ground truth, this paper segments the load profile into daytime, nighttime, and zero-crossing points and utilizes the latter two for evaluation purposes. The proposed disaggregation procedure is validated using real world, minute-level, normalized, and anonymized nodal data in the PJM service territory.

42 ENGINEERING↗