Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data normalization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗
DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗
DOE OSTI2026

2025 Annual INMM Graph and Tables for High Purity Germanium Detector Normalization Presentation

The data set includes gamma spectroscopy peak data for measurements taken with two different high purity germanium detectors using a mixed nuclide source and a U-235 fuel rod. There are a total of 5 specific energy peaks that were analyzed for the mixed nuclide source stemming from Am-241, Cd-109, Cs-137, and Co-60. There are a total of 3 specific energy peaks that were analyzed for the U-235 fuel rod. The data set includes the calculations and results from using a linear correction factor, absolute efficiency curve, and relative efficiency curve to compare the net peaks counts from two different detectors.

Drumm, Natalie Daphne [Sandia National Laboratorie↗
DOE OSTI2025

Raspberry Pi–powered temperature monitoring of growth chamber microclimates

While controlled environments are desirable for growing and measuring plants, growth chambers and greenhouses typically have microclimates that impact plant growth, development, and stress responses. Furthermore, opening and closing the doors of a controlled environment introduces variation in the environment, especially at temperature extremes, affecting both the measurements and the organisms within. Using multiple temperature data loggers to normalize results can be cost-prohibitive and rarely offers real-time feedback on temperature status. We used low-cost single-board computers, cameras, and temperature sensors to manage and capture growth chamber temperatures while acquiring plant image data. Detailed here are methods to document microclimates within a growth chamber so that data can be normalized to measured temperature information. This protocol describes a low-cost method for automated temperature monitoring, which enables both high-throughput measurements of temperature along with plant growth and stress responses via plant imaging.

Plant Sciences↗
NASA NTRS2009

General Purpose Data-Driven Monitoring for Space Operations

As modern space propulsion and exploration systems improve in capability and efficiency, their designs are becoming increasingly sophisticated and complex. Determining the health state of these systems, using traditional parameter limit checking, model-based, or rule-based methods, is becoming more difficult as the number of sensors and component interactions grow. Data-driven monitoring techniques have been developed to address these issues by analyzing system operations data to automatically characterize normal system behavior. System health can be monitored by comparing real-time operating data with these nominal characterizations, providing detection of anomalous data signatures indicative of system faults or failures. Data-driven techniques have a number of advantages over other methods for monitoring complex space vehicles. Unlike model-based systems, the developer does not need to understand or encode the internal operation of the system. The knowledge required to monitor the system is automatically derived from archived data from system operation. Unlike rule-based systems, data-driven systems do not require system analysts to define nominal relationships among sensors. Analysts can and often do determine these relationships for a system with few sensors; it is more difficult to analytically determine the nominal relationship among a large number of sensors. Data-driven techniques are not limited to low-dimensional spaces and work as effectively with dozens of parameters as they do with a few. Knowledge bases formed by data-driven techniques are also easy to update. As the operating envelope of the monitored system is expanded, data-driven techniques can be quickly retrained to incorporate the new behavior into the knowledge base. The expertise and time-consuming process of updating a model or rule base to maintain consistency with the new operation is not required. The Inductive Monitoring System (IMS) is a data-driven system health monitoring software tool that has been successfully applied to several aerospace applications. IMS uses a data mining technique called clustering to analyze archived system data and characterize normal interactions between parameters. This characterization, or model, of nominal operation is stored in a knowledge base that can be used for real-time system monitoring or analysis of archived events. System data is compared with the nominal IMS model to produce a measure of how well current system behavior matches the normal behavior defined by the training data. Significant deviations from the nominal system model can provide alerts to system malfunctions or precursors of significant failures. The scope of IMS based data-driven monitoring applications continues to expand with current development activities. Successful IMS deployment in the International Space Station (ISS) flight control room to monitor ISS attitude control systems has led to applications in other ISS flight control disciplines, such as thermal control. It has also generated interest in data-driven monitoring capability for Constellation, NASA's program to replace the Space Shuttle with new launch vehicles and spacecraft capable of returning astronauts to the moon, and then on to Mars. Several projects are currently underway to evaluate and mature the IMS technology and complementary tools for use in the Constellation program. These include an experiment on board the Air Force TacSat-3 satellite, and ground systems monitoring for NASA's Ares I-X and Ares I launch vehicles. The TacSat-3 Vehicle System Management (TVSM) project is a software experiment to integrate fault and anomaly detection algorithms and diagnosis tools with executive and adaptive planning functions contained in the flight software on-board the Air Force Research Laboratory TacSat-3 satellite. The TVSM software package will be uploaded after launch to monitor spacecraft subsystems such as power and guidance, navigation, and control (GN&C). It will analyze data in real-time to demonstrate detection of faults and unusual conditions, diagnose problems, and react to threats to spacecraft health and mission goals. The experiment will demonstrate the feasibility and effectiveness of integrated system health management (ISHM) technologies with both ground and on-board experiments. Initially, the TVSM software will run open loop, providing system health information and recommendations to ground operators, without automatically performing fault-mitigating corrective actions. After the end of the satellite's mission, closed loop tests combining TVSM monitoring and diagnosis with reactive capabilities by the flight software will be performed. In addition to monitoring for long periods of actual operation, the experiment will include fault injection into TacSat-3 data as well as commanded operations to test and evaluate automatic ISHM monitoring and recovery under controlled conditions.

Satellites↗
NASA NTRS2009

General Purpose Data-Driven System Monitoring for Space Operations

Modern space propulsion and exploration system designs are becoming increasingly sophisticated and complex. Determining the health state of these systems using traditional methods is becoming more difficult as the number of sensors and component interactions grows. Data-driven monitoring techniques have been developed to address these issues by analyzing system operations data to automatically characterize normal system behavior. The Inductive Monitoring System (IMS) is a data-driven system health monitoring software tool that has been successfully applied to several aerospace applications. IMS uses a data mining technique called clustering to analyze archived system data and characterize normal interactions between parameters. This characterization, or model, of nominal operation is stored in a knowledge base that can be used for real-time system monitoring or for analysis of archived events. Ongoing and developing IMS space operations applications include International Space Station flight control, satellite vehicle system health management, launch vehicle ground operations, and fleet supportability. As a common thread of discussion this paper will employ the evolution of the IMS data-driven technique as related to several Integrated Systems Health Management (ISHM) elements. Thematically, the projects listed will be used as case studies. The maturation of IMS via projects where it has been deployed, or is currently being integrated to aid in fault detection will be described. The paper will also explain how IMS can be used to complement a suite of other ISHM tools, providing initial fault detection support for diagnosis and recovery.

Satellites↗
NASA NTRS2012

General Purpose Data-Driven System Monitoring for Space Operations

Modern space propulsion and exploration system designs are becoming increasingly sophisticated and complex. Determining the health state of these systems using traditional methods is becoming more difficult as the number of sensors and component interactions grows. Data-driven monitoring techniques have been developed to address these issues by analyzing system operations data to automatically characterize normal system behavior. The Inductive Monitoring System (IMS) is a data-driven system health monitoring software tool that has been successfully applied to several aerospace applications. IMS uses a data mining technique called clustering to analyze archived system data and characterize normal interactions between parameters. This characterization, or model, of nominal operation is stored in a knowledge base that can be used for real-time system monitoring or for analysis of archived events. Ongoing and developing IMS space operations applications include International Space Station flight control, spacecraft vehicle system health management, launch vehicle ground operations, and fleet supportability. As a common thread of discussion this paper will employ the evolution of the IMS data-driven technique as related to several Integrated Systems Health Management (ISHM) elements. Thematically, the projects listed will be used as case studies. The maturation of IMS via projects where it has been deployed or is currently being integrated to aid in fault detection will be described. The paper will also explain how IMS can be used to complement a suite of other ISHM tools, providing initial fault detection support for diagnosis and recovery

Space Propulsion↗
DOE OSTI2025

Benchmarking Concentration and Extraction Methods for Wastewater-Based Surveillance of Eight Human Respiratory Viruses: Implications for Rapid Application to Novel Pathogens

To provide early warning and support a rapid response to a novel virus through wastewater surveillance, it would be ideal to understand in advance which concentration and extraction methods are likely to be effective for dPCR-based methods, depending on virus characteristics. In this study, we spiked raw wastewater samples with eight human respiratory viruses and processed them with four methods that concentrate and/or extract nucleic acids from both liquid and solid fractions (Promega, Nanotrap, and InnovaPrep) or only the solid fraction of wastewater (Solids). Our findings provide encouraging evidence that all four methods combined with dPCR could detect an emerging virus in wastewater, although they differed in sensitivity. The pattern of recovery efficiency for adenoviruses, coronaviruses, and influenza A viruses was consistent across methods, with Promega producing higher median recovery efficiencies, while distinct patterns were observed for coxsackieviruses. We also normalized the concentration data with two endogenous fecal indicators, PMMoV and Carjivirus (formerly crAssphage). We found that normalization could reduce method-associated differences if the indicator exhibited a recovery pattern similar to that of the target virus. These findings can guide the selection of concentration and extraction methods for wastewater monitoring based on the properties of target viruses, thus enhancing pandemic preparedness.

Biological and medical sciences↗
DOE OSTI2025

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗
DOE OSTI2026

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗
DOE OSTI2026

Temporal Convolutional Network Using Empirical Mode Decomposition to Detect Faults in Grid Connected Systems

Grid-connected power electronic systems require timely and reliable fault detection to prevent equipment damage and reduce downtime. This paper presents a forecasting-based anomaly detection pipeline that decomposes voltage and current measurements into intrinsic mode functions (IMFs) using empirical mode decomposition (EMD), then trains a causal temporal convolutional network (TCN) on normal-operation IMF data to predict short-horizon future dynamics. Deviations between forecasts and observations are summarized as reliability-weighted residual scores and thresholded per sensor using robust statistics with temporal persistence constraints to suppress false positives. To reduce runtime, EMD is performed on downsampled signals for detection, while raw-rate EMD is applied only within a short region of interest for high-frequency interpretability near detected events. Results on a simulated grid-connected converter system demonstrate that IMF-domain forecasting improves anomaly separability relative to raw-signal forecasting and provides interpretable evidence of faults across decomposition channels.

Sutton, Elizabeth [ORNL] (ORCID:0009000078885935)↗
DOE OSTI2026

APOLLO: a facility-scale differentiable virtual accelerator for Fermilab

As the design complexity of modern accelerators grows, there is more interest in using advanced simulations that have fast execution time or yield additional insights like gradients. The FAST/IOTA facility has been working on implementing and experimentally validating an end-to-end digital twin that is both fast and gradient-aware, allowing for rapid prototyping of new software and experiments with minimal beam time costs. Our framework integrates physics and ML codes for linac and ring simulation through a set of generic interfaces between surrogate and physics-based sections. To reproduce device inputs and outputs, system state is exposed as a deterministic discrete event simulator. Because Fermilab is undergoing control system transition, both EPICS and ACNET frontends are supported. Recently, we have begun transitioning to a new community lattice standard, PALS, as well as developing standardized infrastructure for data ingest and normalization to prepare for model calibration during FAST proton injector commissioning. We discuss implementation details as well as challenges, and future plans to extend modelling to main complex proton accelerators like PIPII and Booster.

Kuklev, Nikita [Fermilab]↗
DOE OSTI2026

Dataset for "A primer on forest structure measurement with lidar for ecologists"

This repository includes data and code accompanying the case study included in the manuscript "A primer on forest structure measurement with lidar for ecologists" (submitted to Ecosphere). We compiled lidar datasets from multiple platforms in a common area to: 1. Demonstrate how differences in sensor characteristics influence density and resolution of lidar data. 2. Provide open-source, co-located datasets for users to further inspect differences in lidar data. 3. Provide example code to perform basic lidar analysis. This case study is meant to allow readers to get hands-on experience with real-world data from different platforms. This case study is not meant to be a rigorous comparison of derived ecological metrics among all sensors; such comparisons can be found throughout other publications referenced throughout the main manuscript. Code includes basic functions in R commonly used to visualize and manipulate lidar data accessible with a normal laptop computer; more sophisticated algorithms for advanced users are also referenced throughout the main manuscript. Terrestrial laser scanning (TLS), mobile laser scanning (MLS), UAS laser scanning (ULS), airborne laser scanning (ALS), and spaceborne laser scanning (SLS) data were collected within the Smithsonian Environmental Research Center (SERC) forest dynamics plot in Maryland, USA. TLS, MLS, and ALS data were collected within 1 month of the 2021 growing season; ULS data were collected in November 2020 (“leaf-off” data) and July 2022 (“leaf-on” data).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

PNNL-Predictive-Phenomics/ProteoMeter

ProteoMeter is a Python package that assists in the statistical analysis of global proteomics, protein post-translation modification (PTM), and limited proteolysis (LiP) data. It contains batch correction, normalization, and statistical testing methods, as well as functions that "roll up" peptide-level data to the single-site level. It has a robust user configuration system, allowing it to flexibly integrate different types of experiment designs. For basic usage, a simple configuration file provides the essential functionality. Advanced users have access to the entire statistical pipeline for fine-tuning analyses. Processed data is easily exported to many common spreadsheet and data-frame formats.

Rozum, Jordan [Pacific Northwest National Lab]↗
DOE OSTI2025

Generative learning of densities on manifolds

A generative modeling framework is proposed that combines diffusion models and manifold learning to efficiently sample data densities on manifolds. The approach utilizes Diffusion Maps to uncover possible low-dimensional underlying (latent) spaces in the high-dimensional data (ambient) space. Two approaches for sampling from the latent data density are described. The first is a score-based diffusion model, which is trained to map a standard normal distribution to the latent data distribution using a neural network. The second one involves solving an Itô stochastic differential equation in the latent space. Additional realizations of the data are generated by lifting the samples back to the ambient space using Double Diffusion Maps , a recently introduced technique typically employed in studying dynamical system reduction; here the focus lies in sampling densities rather than system dynamics. The proposed approaches enable sampling high dimensional data densities restricted to low-dimensional, a priori unknown manifolds. The efficacy of the proposed framework is demonstrated through a benchmark problem and a material with multiscale structure.

Double diffusion maps↗
DOE OSTI2025

The HydroBio Dataset: a new data resource for evaluating existing and potential hydropower capacity and freshwater biodiversity in the conterminous United States

Hydropower is a critical source of affordable and reliable electricity and energy system stability services in the United States. Opportunities to expand US hydropower production include retrofitting existing non-powered dams to produce power, retrofitting existing hydropower dams to improve efficiency or increase capacity, or constructing new hydropower infrastructure on currently unregulated river reaches. We created the HydroBio Dataset, which summarizes existing and potential hydropower capacity and freshwater biodiversity at the sub-basin scale in the conterminous US to contextualize existing and potential grid contributions with the freshwater ecosystems in which dams are situated. We demonstrate a use-case of this dataset by rescaling and comparing potential non-powered dam nominal capacity to rarity-threat-weighted freshwater species richness for sub-basins where both types of data exist. On average, normalized freshwater biodiversity exceeded normalized potential non-powered dam nominal capacity in these sub-basins. Potential non-powered dam nominal capacity was concentrated in sub-basins in the Upper Mississippi and Ohio hydrologic regions while freshwater biodiversity was concentrated in the South Atlantic-Gulf, Ohio, and Tennessee hydrologic regions. Additionally, non-powered dams and existing hydropower dams are located in sub-basins with similar indices of freshwater biodiversity. The HydroBio Dataset adds an additional ecological dimension of context to our understanding of current and potential future US hydropower capabilities and is a valuable decision support tool for stakeholders tasked with balancing gains in services to the US power grid with the public and environmental benefits of freshwater ecosystems.

Biodiversity↗
DOE OSTI2026

Documentation, assessment, and improvements of fission gas modeling capabilities in BISON

Fission gas behavior critically affects nuclear fuel performance, influencing thermal conductivity, microstructure, swelling, and fuel-cladding interactions. The Simple Integrated Fission Gas Release and Swelling (Sifgrs) model, implemented within the BISON fuel performance code, provides advanced capabilities for modeling these phenomena. This manuscript reviews and assesses Sifgrs’ modular structure and submodels describing intragranular, intergranular, and release mechanisms. The modular implementation enables rapid improvements as our understanding advances and new fuel forms emerge. After an overview of capabilities, we present a comprehensive validation suite, comparing predictions to experimental data from tests under normal and transient conditions, including reproduction of the Vitanza curve. Results demonstrate Sifgrs’ accuracy, robustness, and applicability. Current limitations and ongoing efforts are discussed, including mechanistic, multiscale modeling, uncertainty quantification, and validation within the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, supporting the safe and efficient use of nuclear fuels.

BISON↗