Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Preprocessing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

An interlaboratory comparison of mid-infrared spectra acquisition: Instruments and procedures matter

Diffuse reflectance spectroscopy has been extensively employed to deliver timely and cost-effective predictions of a number of soil properties. However, although several soil spectral laboratories have been established worldwide, the distinct characteristics of instruments and operations still hamper further integration and interoperability across mid-infrared (MIR) soil spectral libraries. In this study, we conducted a large-scale ring trial experiment to understand the lab-to-lab variability of multiple MIR instruments. By developing a systematic evaluation of different mathematical treatments with modeling algorithms, including regular preprocessing and spectral standardization, we quantified and evaluated instruments' dissimilarity and how this impacts internal and shared model performance. We found that all instruments delivered good predictions when calibrated internally using the same instruments' characteristics and standard operating procedures by solely relying on regular spectral preprocessing that accounts for light scattering and multiplicative/additive effects, e.g., using standard normal variate (SNV). When performing model transfer from a large public library (the USDA NSSCKSSL MIR library) to secondary instruments, good performance was also achieved by regular preprocessing (e. g., SNV) if both instruments shared the same manufacturer. However, significant differences between the KSSL MIR library and contrasting ring trial instruments responses were evident and confirmed by a semi-unsupervised spectral clustering. For heavily contrasting setups, spectral standardization was necessary before transferring prediction models. Non-linear model types like Cubist and memory-based learning delivered more precise estimates because they seemed to be less sensitive to spectral variations than global partial least square regression. In summary, the results from this study can assist new laboratories in building spectroscopy capacity utilizing existing MIR spectral libraries and support the recent global efforts to make soil spectroscopy universally accessible with centralized or shared operating procedures.

58 GEOSCIENCES↗
DOE OSTI2025

Subject-specific modeling framework for particle deposition using computational fluid dynamics

Quantifying particle deposition and dose in the respiratory tract requires a physiologically realistic representation and reproducible computational workflows. However, existing modeling frameworks, such as the International Commission on Radiological Protection (ICRP) compartmental models and the Multiple Path Particle Dosimetry (MPPD) tool, lack detailed deposition profiles and subject-specific capabilities. The combination of advances in computer vision algorithms applied to the respiratory tract and Computational Fluid and Particle Dynamics (CFPD) allows high-fidelity simulations of particle behavior in anatomically accurate geometries derived from individual CT scans. The segmentation, preprocessing, and file preparation task for a CFPD simulation was often time-consuming, and no prior studies to-date have yet presented a fully automated framework. This work presents a fully automated workflow to obtain individualized particle deposition profiles in the human respiratory tract. The pipeline starts with segmenting upper and lower airway geometries using morphological and deep learning-based methods, generating three-dimensional (3D) models from CT imaging data. Next, a series of algorithms are presented to quality check and prepare the 3D geometry for a CFD or CFPD simulation. The preprocessing step includes correcting geometric artifacts, enforcing a physically consistent mesh, and automatically identifying and capping multiple outlets, which is required for CFD/CFPD simulations. These processed models are then input into open-source (OpenFOAM) or commercial (StarCCM+) CFD solvers, where flow and transient particle transport equations — including turbulence and particle–wall interactions are solved under realistic breathing conditions. Finally, the resulting particle deposition profiles can be integrated with Monte Carlo radiation transport codes and state-of-the-art computational phantoms to assess organ-specific absorbed doses in scenarios of radioactive aerosol inhalation. The presented work streamlines respiratory tract segmentation, preprocessing for CFD/CFPD simulations, and integration with dose assessment workflows, reducing manual intervention and improving access to high-fidelity, subject-specific modeling. The high precision in predicted particle deposition and dose distributions can improve personalized treatment strategies in respiratory medicine and refine dose estimates for radiation protection.

AI↗
DOE OSTI2023

Arithmetic Primitives for Efficient Neuromorphic Computing

Neuromorphic computing is steadily gaining popularity in many scientific and engineering disciplines. However, one of the biggest problems that has prevented widespread usage of neuromorphic computing is the lack of efficient encoding methods. Traditional encoding methods such as binning, rate encoding, and temporal encoding are based on unary encoding and generate a large number of spikes for certain applications, making them less energy efficient. Lack of better encoding methods has also prevented preprocessing operations from being carried out on neuromorphic computers. As a result, over 99% of the time can be spent on data preprocessing and data transfer operations in some cases, leading to an inefficient workflow. In this paper, we present preliminary results that would enable us to efficiently encode data and perform basic arithmetic operations on neuromorphic computers. First, we present a neuromorphic approach for the two’s complement encoding of numbers and leverage it to devise addition and multiplication circuits, which could be used in preprocessing operations on neuromorphic computers. We test our approach on the SuperNeuroMAT simulator. Our results indicate that two’s complement is a highly efficient encoding method in terms of time, space, and energy complexity and that the addition and multiplication circuits produce accurate results on two numbers having arbitrary precision.

Wurm, Ahna↗
DOE OSTI2023

Providing Geospatial Intelligence through a Scalable Imagery Pipeline

This chapter describes ORNL’s (Oak Ridge National Laboratory’s) contributions to imagery preprocessing for geospatial intelligence research and development (R&D) in four sections. First, we discuss challenges involved in building an effective imagery preprocessing workflow and the world-class high-performance computing (HPC) resources at ORNL available to process petabytes of imagery data. Second, we highlight how we developed imagery preprocessing tools over three decades while paving the way for our current cutting-edge machine learning and computer vision algorithms that are impacting humanitarian and disaster response efforts. Third, we discuss how PIPE modules work together to turn raw images into analysis-ready datasets. Fourth, we look toward the future and discuss planned advancements to PIPE and computing trends that will affect geospatial intelligence R&D.

Reith, Andrew↗
DOE OSTI2020

Herbaceous Feedstock 2018 State of Technology Report

The U.S. Department of Energy (DOE) promotes the production of advanced liquid transportation fuels from lignocellulosic biomass by funding fundamental and applied research that advances the state of technology (SOT). As part of its involvement with this mission, Idaho National Laboratory (INL) completes an annual SOT report for biomass feedstock logistics. This report summarizes supply system impacts of Bioenergy Technologies Office (BETO)-funded research and development efforts at INL and elsewhere (such as the High-Tonnage Feedstock Logistics projects (Webb et al. 2013a, Webb et al. 2013b, Webb et al. 2013c, Webb and Sokhansanj 2014, Sokhansanj et al. 2014) that lead to improvements in feedstock supply systems. These include improvements to and observed performance of innovative harvest and collection methods, storage technologies, transportation and handling approaches, and advanced preprocessing technologies. Biomass quality and variability, and the interface between feedstock quality and conversion performance are key drivers in addition to delivered feedstock cost. In this report, we estimate the benefits of R&D improvements to individual supply system unit operations, and present the status of feedstock logistics technology development for converting biomass into biofuels. These analyses are supported by experimental data where possible, and help to align the SOT relative to the cost goals defined in the Multi-Year Program Plan. The 2018 Herbaceous SOT aligned feedstock logistic design with current biorefinery’s design capacity utilized by biochemical conversion platform. Currently biochemical conversion platform utilizes a 725,000 dry ton/year biorefiney design for the techno economic analysis. Hence, feedstock delivered cost in the 2018 Herbaceous SOT is calculated based on biorefinery’s 725,000 dry ton design capacity instead of 800, 000 dry ton capacity utilized in the 2017 Herbaceous SOT. Biomass availabilities in this SOT were updated to year 2018 data from the 2016 Billion-Ton Report (BT16) (DOE 2016a), with the exception of switchgrass, for which the 2018 Herbaceous SOT utilized the 2019 switchgrass availability data from BT16. The BT16 report (DOE 2016a) does not project switchgrass availability in 2018; the soonest switchgrass is available in the BT16 report is 2019. Therefore, availability of switchgrass for this analysis was that projected for 2019. The 2018 Herbaceous SOT incorporates same technologies utilized in the 2017 Herbaceous SOT. However, a sensitivity analysis is performed to understand the impact of variation of process parameters on those technologies on feedstock logistic cost. New R&D data that shows the variations of process parameters affecting process performance is incorporated in the 2018 SOT to measure the variations in delivered feedstock cost. The 2018 Herbaceous SOT has also provided projected delivered feedstock of 2022 design case based on near term technical target under BETO funded R&D project. Finally, updated biorefinery size of 725,000 dry ton/year was incorporated within least-cost formulation model to select optimal siting and depot scales during optimization of the least cost blend. This modification to the optimization algorithm allows the trade-off between the cost of increased supply radius and the savings from selecting biomass from higher producing counties to be assessed. Such optimization has also showed the economic benefit of decentralized depots in comparison to centralized preprocessing co-located with the biorefinery by decoupling the biorefinery and feedstock locations. The 2018 Herbaceous SOT report documents the current modeled cost of a herbaceous feedstock supply system (from harvest to the pretreatment reactor throat, including grower payment) for hydrocarbon fuel production via biochemical conversion, based on equipment and processes now available or potentially available in the near term. The modeled cost also considers both the required quality and the availability of the biomass resources. The 2018 Herbaceous SOT predicts a modeled delivered feedstock cost of $83.67/dry ton (2016$); this is a $0.23/dry ton (2016$) decrease from the 2017 Herbaceous SOT. The modification of biorefinery’s designed capacity and increased projected biomass availability in the same supply shed contributed to this modeled cost reduction. The least-cost formulation model to optimally site and scale local distributed preprocessing depots also contributed to the cost reduction by considering county-level grower payment and distance from the biorefinery as variables in the optimization algorithm. Sensitivity analysis on various process parameters that affect delivered feedstock cost in the 2018 Herbaceous SOT shows that the delivered cost could varies from $80.45-$88.83/dry ton. The top factors that causes such variations are: effective baling rate, bale density, hammer mill throughput, interest rate and storage dry matter loss.

09 BIOMASS FUELS↗
DOE OSTI2020

Improved Advanced Biomass Logistics Utilizing Woody and other Feedstocks in the Northeast and Pacific Northwest (Final Report)

Willow and poplar short rotation woody crops (SRWC) have shown promise with regards to environmental benefits and rural development but wide adoption lags due to underdeveloped markets and supply systems. High costs associated with harvesting, handling and transportation (40-60% of delivered cost) have impeded expansion. A better understanding of these systems will create opportunities to improve efficiency, reduce costs, and realize environmental benefits and impacts. The project’s goal was to lower the delivered cost of hybrid poplar in the Northwest and willow in the Northeast by optimizing harvesting and logistics supply systems while maintaining or improving biomass quality along the supply chain. Over 3,400 Mg of biomass and 300 ha of willow and poplar were monitored over a range of crop and field conditions. Feedstock quality as affected by storage and preprocessing were shown to improve or maintain feedstock quality. Modeled harvesting costs ranged from $38–61 Mg -1 dry; when including delivery and preprocessing feedstock costs ranged between $79-83 Mg -1 dry for willow and $106-116 Mg -1 dry for poplar. Costs for willow minimized when hot water extraction and high-moisture densification preprocessing were used. Models also suggest that social and regional factors could further reduce costs. Results will give guidance to feedstock growers, harvesting and logistic operations, biorefinery project developers, and policy makers developing SRWC to support a growing bioeconomy.

, biomass↗
DOE OSTI2024

Performance Comparison of Machine Learning Models for Ultrasonic Nondestructive Evaluation of Alkali-Silica Reaction in Concrete

Alkali-silica reaction (ASR) causes concrete degradation, leading to cracking, rebar corrosion, and reduced structural integrity, which raises safety concerns. Ultrasonic nondestructive evaluation (NDE) effectively assesses concrete properties and monitors ASR progression. However, its deployment and analysis require specialized expertise and subjective interpretation. As computational power increases, artificial intelligence (AI) and machine learning (ML) algorithms are increasingly being used to automate NDE data analysis across various industries for AI-assisted automation. Regulatory agencies are adapting to this technological shift, prompting a need to evaluate current ML technologies’ capabilities and limitations in assessing concrete material properties and damage. This report presents a comparative analysis of four ML regression models for predicting concrete material damage induced by ASR expansion using long-term ultrasonic data monitoring. The models investigated include linear regression (LR), support vector regression (SVR), shallow neural networks (NN), and deep neural networks (DNN). LR, SVR, and shallow NN models use features extracted from ultrasonic signals, whereas the DNN model processes time-domain ultrasonic signals and frequency spectra directly. The study systematically compared the models’ performance from various perspectives, including model input, prediction performance, and generalization ability. The findings indicate significant variability in model performance, with some ML algorithms achieving very high or very low prediction accuracy depending on the preprocessing and feature engineering (extraction and selection) applied. Key insights include the observation that shallow ML models (LR, SVR, and shallow NNs) require meticulous preprocessing and feature extraction to achieve high accuracy. In contrast, the DNN model, although it bypasses the need for feature engineering, necessitates extensive preprocessing to mitigate noise and computational demands. The SVR model emerged as the top performer among the shallow models, and the DNN model exhibited superior performance on specific datasets but struggled with generalization across specimens from different batches. Additionally, the SVR model is sensitive to temperature variations, whereas the DNN model is robust in this regard. Using recurrent neural networks is recommended for future ASR expansion prediction studies. Recurrent neural networks’ inherent ability to capture temporal dependencies and long-term patterns makes them well suited for analyzing sequential ultrasonic monitoring data. Overall, the results and conclusions of this study could provide insights into the capabilities and effectiveness of ML when applied to ultrasonic NDE data and help identify best practices for using ML for ultrasonic NDE of concrete material properties.

36 MATERIALS SCIENCE↗
DOE OSTI2021

BETO 2021 Peer Review - Rational Design of Robust Reactor Feeding Systems for Heterogeneous Cellulosic and Agricultural Wastes Based on Biomass Quality Characteristics

Consistent and reliable preprocessing, conveyance, and reactor in-feed systems, particularly for low-cost waste feedstocks, remains a major technical challenge for the emerging Bioeconomy. By identifying critical biomass attributes and connecting them to flow and conversion behavior, science-driven system designs can address these often-overlooked solids handling challenges. The Wonderful Company (TWC) is the world's largest almond and pistachio grower, generating 250,000 dry tons/year of waste material including hulls, shells, and wood (>5 million tons/year industry wide in the U.S.). This project seeks to turn this environmental and economic liability into a sustainable and profitable resource, targeting conversion via gasification to syngas for electricity and bio-char, by addressing related material handling and feeding challenges. Based on optimized preprocessing strategies, bulk material flow, and thermal conversion properties, an overall system design will be developed. The methodology will be tested with FCIC's benchmark loblolly pine residues, to demonstrate the robustness of the overall approach and provide insight and guidance for future systems. The project will culminate in extended field trials to demonstrate an improved continuous feeding system with a commercial biomass-to-electricity gasifier vendor, and an economic analysis demonstrating a reduction in electricity production costs by maximizing on-stream time while minimizing preprocessing and CapEx costs.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Understanding Biomass and Polymer Feedstock Variability Through Analytical Pyrolysis and Two Dimensional Gas Chromatography and Mass Spectrometry

Moving toward a circular carbon economy requires enabling reuse of carbon-based macromolecules like those in biomass and polymers. Meeting this goal requires increased upcycling of waste products into higher value products. For biomass waste such as corn stover, pine needles, or bark, one approach is using pyrolysis to convert these feedstocks into bio-oil that can be used for liquid fuel or transformed into other hydrocarbon based products, like bio-polymers. Waste plastics can be upcycled or recycled into new products, or alternatively converted into bio-oil by pyrolysis. Macromolecules from biomass and polymers are challenging to convert into liquid fuel or chemical feedstocks, due to large and sometimes heterogeneous networks of polymeric bonds. We work with the Idaho National Laboratory’s Biomass Feedstock National User Facility and Department of Energy’s Feedstock Conversion Interface Consortium to develop a molecular understanding of biomass variability that occurs naturally and as a result of storage or preprocessing techniques applied to feedstocks, to understand chemical reactions occurring during pyrolysis and other biomass or polymer conversion techniques. Using two dimensional gas chromatography mass spectrometry coupled to analytical pyrolysis provides insight into the molecular changes that occur during the pyrolysis event, and these insights can extend to bench or pilot scale pyrolysis conversions. Understanding pyrolysis of biomass and polymers at a molecular level contributes to development of new tools to predict and manipulate the optimum conditions and identify the best storage, preprocessing, and conversion techniques to maximize output of specific high-value chemical species in the conversion to fuels, or to characterize feedstocks for blending before pyrolysis. Identification and understanding of the molecular level changes resulting from novel preprocessing techniques that can be used to control and manipulate the chemical output of bench or pilot scale pyrolysis is another focus of investigation, including the use of gamma irradiation, acid, or enzyme pre-treatments. We also focus on the impacts of storage conditions and techniques, which can affect moisture levels, degradation, and impacts from naturally occurring microbes in the environment.

09 BIOMASS FUELS↗
DOE OSTI2020

Pilot plant reliability metrics for grinding and fast pyrolysis of woody residues

Here we report on the effects of loblolly pine residue variability on material throughput, pilot plant uptime, operator intervention, product yield, and product quality for grinding, fast pyrolysis, and hydrotreating operations. Preprocessing throughput using a hammer mill varied between 31-48% of nameplate capacity (5 tons/hr). Grinder overloads in the size reduction step were more prevalent for lower ash and higher moisture materials. Fast pyrolysis throughput varied between 57-72% of nameplate capacity (20 kg/hr) and bio-oil yields varied between 46-53% (feedstock carbon to oil, dry basis). During fast pyrolysis operations, downtime was caused by bridging in the feed and char removal systems and plugging in the condensation system. Cohesion of feedstock and char leading to system plugging was less frequent for higher ash feedstocks, and differences in condenser plugging behavior between high and low ash feedstocks were observed. The catalyst stability of the bio-oil stabilization step was strongly dependent on the sulfur content in the bio-oil, which was higher for the high-ash residue oils. Lower moisture content in the starting biomass was consistent with lower sulfur content in bio-oil. Yields and properties of hydrotreated fuel products showed minimal deference among the bio-oils.

Klinger, Jordan↗
DOE OSTI2023

Design of an Integrated Solids Handling System to Maximize Syngas Process Reliability (Cooperative Research and Development Final Report)

The National Renewable Energy Laboratory (NREL), in partnership with Wonderful Renewable Energy (WRE) and Idaho National Laboratory (INL), plans to develop a general methodology for designing integrated biorefinery solids preprocessing, handling, and feeding systems based on the chemical, physical, and mechanical attributes of the starting biomass material. This attribute-driven approach will include detailed feedstock property measurements, iterative computational modeling, and bench-scale testing to design systems for preprocessing, handling, and reactor in-feed, up to and including the selection of the conversion reactor. The initial tests and system design will be conducted using waste material from almond and pistachio growing and production operations (shells, hulls, and wood), targeting the conversion of this material to syngas for electricity production. The methodology will then be generalized to other feedstocks. The purpose of this project is to design an integrated solids handling system to maximize the process reliability of converting almond and pistachio waste to electricity. The design methodology and workflow developed from this example will then be applied to the Feedstock Conversion Interface Consortium (FCIC) benchmark loblolly pine residues, thus demonstrating the robustness of the overall design approach and providing insight and guidance for future conversion systems. V-Grid Energy Systems was brought on as a subcontractor to provide gasifiers and labor to complete gasifier runs.

09 BIOMASS FUELS↗
DOE OSTI2021

FCIC DFO--Moisture Management and Optimization in Municipal Solid Waste Feedstock through Mechanical Processing

The project goal is to produce densified product, focusing on characteristics of bulk density, durability, and moisture content, to reduce the cost of preprocessing. The objective of the project is to efficiently manage and handle the properties of MSW feedstock through INL-developed fractional milling, high moisture pelleting, and low temperature drying preprocessing technologies .

09 BIOMASS FUELS↗
DOE OSTI2026

Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials

This article examines how data readiness for AI principles apply to large scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, life sciences, and materials—to identify common preprocessing patterns and domain‐specific constraints. We introduce a two‐dimensional readiness model that combines canonical preprocessing patterns with a five‐level operational readiness scale, both tailored to high‐performance computing (HPC) environments. This construct helps outline key challenges in transforming large‐scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross‐domain support for scalable and reproducible AI for science. Finally, we evaluate this maturity matrix in the context of case studies including ClimaX (climate), AFLOW (materials), OpenFold (proteomics), and DIII‐D fusion disruption‐prediction workflows, from which we distill lessons learned and provide recommendations to guide practitioners in developing robust AI‐readiness pipelines. Finally, we discuss remaining cross‐cutting challenges that persist across scientific domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Addressing Inherent Challenges to Chemical Relithiation of Cycled End‐of‐Life Cathode Materials

Recycling end‐of‐life (EOL) lithium‐ion batteries (LIBs) is important to retain valuable resources from critical materials present in EOL battery waste. Direct recycling methods offer an opportunity to recover intact valuable cathode materials with minimal re‐processing. An important step of the direct recycling process is relithiation which is used to restore lithium content to EOL cathode materials. However, little has been done to study how preprocessing steps such as washing or binder removal may affect relithiation methods in the direct recycling process. Here, the evolution of fluorine byproducts left over from preprocessing steps during a low‐temperature chemical redox mediator relithiation process is tracked. A facile washing step is presented as a solution for mediating adverse effects of surface contamination on the chemical relithiation performance. The structure, lithium content, and electrochemical performance of relithiated EOL NMC 622 material that underwent a pre‐relithiation washing step to remove fluorine byproducts is shown to match that of pristine NMC 622. In this work, it is showed that redox mediator relithiation as a part of a direct recycling process is a promising low energy method that can be applied to EOL material with inherent surface impurities if the proper pre‐relithiation processing steps are implemented.

25 ENERGY STORAGE↗
DOE OSTI2025

Enriched immersed finite element and isogeometric analysis: algorithms and data structures

Immersed finite element methods provide a convenient analysis framework for problems involving geometrically complex domains, such as those found in topology optimization and microstructures for engineered materials. However, their implementation remains a major challenge due to, among other things, the need to apply nontrivial stabilization schemes and generate custom quadrature rules. This article introduces the robust and computationally efficient algorithms and data structures comprising an immersed finite element preprocessing framework. The input to the preprocessor consists of a background mesh and one or more geometries defined on its domain. The output is structured into groups of elements with custom quadrature rules formatted such that common finite element assembly routines may be used without or with only minimal modifications. The key to the preprocessing framework is the construction of material topology information, concurrently with the generation of a quadrature rule, which is then used to perform enrichment and generate stabilization rules. While the algorithmic framework applies to a wide range of immersed finite element methods using different types of meshes, integration, and stabilization schemes, the preprocessor is presented within the context of the extended isogeometric analysis. This method utilizes a structured B-spline mesh, a generalized Heaviside enrichment strategy considering the material layout within individual basis functions’ supports, and face-oriented ghost stabilization. Using a set of examples, the effectiveness of the enrichment and stabilization strategies is demonstrated alongside the preprocessor’s robustness in geometric edge cases. Additionally, the performance and parallel scalability of the implementation are evaluated.

Computer implementation↗
DOE OSTI2025

Towards automated and real-time multi-object detection of anguilliform fishes from sonar data using YOLOv8 deep learning algorithm

Eels (Anguilla spp.), including American eels (Anguilla rostrata), European eels (Anguilla anguilla), and Japanese eels (Anguilla japonica), are species of critical management and regulatory concern due to their vulnerability to various stressors during downstream migrations. Accurate and efficient detection of migrating eels can improve our understanding of fish behaviors and fish-hydraulic structure interactions, thereby facilitating the design, operation, and optimization of more effective downstream passage facilities from both biological and economic perspectives. However, a real-time, automated framework for detecting migrating eels in real-world applications is currently lacking. Leveraging imaging sonar as a reliable technology for fish passage monitoring, field data are acquired using imaging sonar and then converted to single sonar frames/images for subsequent analysis. In this study, a framework based on the You Only Look Once Version 8 (YOLOv8)-based convolutional neural network is proposed for multi-object detection of eels and non-eel fish using the sonar images after image subtraction and additional wavelet denoising. The results from both training and testing phases demonstrate that the framework's ability can successfully detect both eels and non-eel fish in preprocessed sonar images, achieving F1-scores and mAP@0.50 exceeding 0.84. Additionally, the incorporation of wavelet denoising during preprocessing slightly improve detection performance. Furthermore, the transferability of this framework from eel to lamprey detection is demonstrated to be feasible given the similar morphological characteristics of these two species. Overall, the proposed framework achieves accurate and efficient detection of migrating eels, providing reliable and real-time information that can help conserve vulnerable eel and eel-like populations.

Deep learning↗
DOE OSTI2020

The good, the bad, and the ugly: Data-driven load profile discord identification in a large building portfolio

Reducing the overall energy consumption and associated greenhouse gas emissions in the building sector is essential for meeting our future sustainability goals. Recently, smart energy metering facilities have been deployed to enable monitoring of energy consumption data with hourly or subhourly temporal resolution. This unprecedented data collection has created various opportunities for advanced data analytics involving load profiles (e.g., building energy benchmarking programs, building-to-grid integration, and calibration of urban-scale energy models). These applications often need preprocessing steps to detect daily load profile discords, such as: 1) outliers due to system malfunctions (the bad) and 2) irregular energy consumption patterns, such as those resulting from holidays (the ugly) compared to normal consumption patterns (the good). However, current preprocessing methods predominantly focus on filtering using statistical threshold values, which fail to capture the contextual discords of daily profiles. In addition, discord detection algorithms in building research are often aimed at finding individual building-level discords, which are not suitable at a large scale. Thus, here, we develop a method for automated load profile discord identification (ALDI) in a large portfolio of buildings (more than 100 buildings). Specifically, ALDI 1) uses the matrix profile (MP) method to quantify the similarities of daily subsequences in time series meter data, 2) compares daily MP values with typical-day MP distributions using the Kolmogorov-Smirnov test, and 3) identifies daily load profile discords in a large building portfolio. We evaluate ALDI using the metering data of both an academic campus and a residential neighborhood. Our results demonstrate that ALDI efficiently discovers measurement errors by system malfunctions and low energy consumption days in the academic campus portfolio, and it detects unique load shape patterns likely driven by occupant behavior and extreme weather conditions in the residential neighborhood.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Common Column Identification for Table Similarity Detection in Electrified Transportation Data Lakes

Electrified transportation often requires researchers and operators to interact with datasets from a wide range of sources and disciplines, such as transportation, power systems, public health, policies, and regulations. These datasets vary in quality and format, making it difficult to understand, preprocess, and identify key columns representing real-world entities or values for indexing and joining, which can negatively impact downstream analysis and operation. Existing solutions are limited, requiring extensive manual customization or data expertise to utilize. In this article, we propose a multi-layered approach to automatically identify key columns to expedite preprocessing and aid in analysis of electrified transportation data. Our method leverages a dynamic ontology to identify common fields and an information theory-based strategy for edge cases that are difficult to generalize. Evaluations on a number of datasets from data.gov and kaggle.com show improved performance of our methods over several baseline techniques, and our ablation analyses illustrate the efficacy of individual components of our method. Our case studies also demonstrate that our methods have the potential to improve analysis of electrified transportation data and aid in automatic integration of such datasets.

33 ADVANCED PROPULSION SYSTEMS↗