Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Preprocessing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

FCIC DFO--Moisture Management and Optimization in Municipal Solid Waste Feedstock through Mechanical Processing

The project goal is to produce densified product, focusing on characteristics of bulk density, durability, and moisture content, to reduce the cost of preprocessing. The objective of the project is to efficiently manage and handle the properties of MSW feedstock through INL-developed fractional milling, high moisture pelleting, and low temperature drying preprocessing technologies .

09 BIOMASS FUELS↗

Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials

This article examines how data readiness for AI principles apply to large scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, life sciences, and materials—to identify common preprocessing patterns and domain‐specific constraints. We introduce a two‐dimensional readiness model that combines canonical preprocessing patterns with a five‐level operational readiness scale, both tailored to high‐performance computing (HPC) environments. This construct helps outline key challenges in transforming large‐scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross‐domain support for scalable and reproducible AI for science. Finally, we evaluate this maturity matrix in the context of case studies including ClimaX (climate), AFLOW (materials), OpenFold (proteomics), and DIII‐D fusion disruption‐prediction workflows, from which we distill lessons learned and provide recommendations to guide practitioners in developing robust AI‐readiness pipelines. Finally, we discuss remaining cross‐cutting challenges that persist across scientific domains.

97 MATHEMATICS AND COMPUTING↗

Addressing Inherent Challenges to Chemical Relithiation of Cycled End‐of‐Life Cathode Materials

Recycling end‐of‐life (EOL) lithium‐ion batteries (LIBs) is important to retain valuable resources from critical materials present in EOL battery waste. Direct recycling methods offer an opportunity to recover intact valuable cathode materials with minimal re‐processing. An important step of the direct recycling process is relithiation which is used to restore lithium content to EOL cathode materials. However, little has been done to study how preprocessing steps such as washing or binder removal may affect relithiation methods in the direct recycling process. Here, the evolution of fluorine byproducts left over from preprocessing steps during a low‐temperature chemical redox mediator relithiation process is tracked. A facile washing step is presented as a solution for mediating adverse effects of surface contamination on the chemical relithiation performance. The structure, lithium content, and electrochemical performance of relithiated EOL NMC 622 material that underwent a pre‐relithiation washing step to remove fluorine byproducts is shown to match that of pristine NMC 622. In this work, it is showed that redox mediator relithiation as a part of a direct recycling process is a promising low energy method that can be applied to EOL material with inherent surface impurities if the proper pre‐relithiation processing steps are implemented.

25 ENERGY STORAGE↗

Enriched immersed finite element and isogeometric analysis: algorithms and data structures

Immersed finite element methods provide a convenient analysis framework for problems involving geometrically complex domains, such as those found in topology optimization and microstructures for engineered materials. However, their implementation remains a major challenge due to, among other things, the need to apply nontrivial stabilization schemes and generate custom quadrature rules. This article introduces the robust and computationally efficient algorithms and data structures comprising an immersed finite element preprocessing framework. The input to the preprocessor consists of a background mesh and one or more geometries defined on its domain. The output is structured into groups of elements with custom quadrature rules formatted such that common finite element assembly routines may be used without or with only minimal modifications. The key to the preprocessing framework is the construction of material topology information, concurrently with the generation of a quadrature rule, which is then used to perform enrichment and generate stabilization rules. While the algorithmic framework applies to a wide range of immersed finite element methods using different types of meshes, integration, and stabilization schemes, the preprocessor is presented within the context of the extended isogeometric analysis. This method utilizes a structured B-spline mesh, a generalized Heaviside enrichment strategy considering the material layout within individual basis functions’ supports, and face-oriented ghost stabilization. Using a set of examples, the effectiveness of the enrichment and stabilization strategies is demonstrated alongside the preprocessor’s robustness in geometric edge cases. Additionally, the performance and parallel scalability of the implementation are evaluated.

Computer implementation↗

Towards automated and real-time multi-object detection of anguilliform fishes from sonar data using YOLOv8 deep learning algorithm

Eels (Anguilla spp.), including American eels (Anguilla rostrata), European eels (Anguilla anguilla), and Japanese eels (Anguilla japonica), are species of critical management and regulatory concern due to their vulnerability to various stressors during downstream migrations. Accurate and efficient detection of migrating eels can improve our understanding of fish behaviors and fish-hydraulic structure interactions, thereby facilitating the design, operation, and optimization of more effective downstream passage facilities from both biological and economic perspectives. However, a real-time, automated framework for detecting migrating eels in real-world applications is currently lacking. Leveraging imaging sonar as a reliable technology for fish passage monitoring, field data are acquired using imaging sonar and then converted to single sonar frames/images for subsequent analysis. In this study, a framework based on the You Only Look Once Version 8 (YOLOv8)-based convolutional neural network is proposed for multi-object detection of eels and non-eel fish using the sonar images after image subtraction and additional wavelet denoising. The results from both training and testing phases demonstrate that the framework's ability can successfully detect both eels and non-eel fish in preprocessed sonar images, achieving F1-scores and mAP@0.50 exceeding 0.84. Additionally, the incorporation of wavelet denoising during preprocessing slightly improve detection performance. Furthermore, the transferability of this framework from eel to lamprey detection is demonstrated to be feasible given the similar morphological characteristics of these two species. Overall, the proposed framework achieves accurate and efficient detection of migrating eels, providing reliable and real-time information that can help conserve vulnerable eel and eel-like populations.

Deep learning↗

The good, the bad, and the ugly: Data-driven load profile discord identification in a large building portfolio

Reducing the overall energy consumption and associated greenhouse gas emissions in the building sector is essential for meeting our future sustainability goals. Recently, smart energy metering facilities have been deployed to enable monitoring of energy consumption data with hourly or subhourly temporal resolution. This unprecedented data collection has created various opportunities for advanced data analytics involving load profiles (e.g., building energy benchmarking programs, building-to-grid integration, and calibration of urban-scale energy models). These applications often need preprocessing steps to detect daily load profile discords, such as: 1) outliers due to system malfunctions (the bad) and 2) irregular energy consumption patterns, such as those resulting from holidays (the ugly) compared to normal consumption patterns (the good). However, current preprocessing methods predominantly focus on filtering using statistical threshold values, which fail to capture the contextual discords of daily profiles. In addition, discord detection algorithms in building research are often aimed at finding individual building-level discords, which are not suitable at a large scale. Thus, here, we develop a method for automated load profile discord identification (ALDI) in a large portfolio of buildings (more than 100 buildings). Specifically, ALDI 1) uses the matrix profile (MP) method to quantify the similarities of daily subsequences in time series meter data, 2) compares daily MP values with typical-day MP distributions using the Kolmogorov-Smirnov test, and 3) identifies daily load profile discords in a large building portfolio. We evaluate ALDI using the metering data of both an academic campus and a residential neighborhood. Our results demonstrate that ALDI efficiently discovers measurement errors by system malfunctions and low energy consumption days in the academic campus portfolio, and it detects unique load shape patterns likely driven by occupant behavior and extreme weather conditions in the residential neighborhood.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Common Column Identification for Table Similarity Detection in Electrified Transportation Data Lakes

Electrified transportation often requires researchers and operators to interact with datasets from a wide range of sources and disciplines, such as transportation, power systems, public health, policies, and regulations. These datasets vary in quality and format, making it difficult to understand, preprocess, and identify key columns representing real-world entities or values for indexing and joining, which can negatively impact downstream analysis and operation. Existing solutions are limited, requiring extensive manual customization or data expertise to utilize. In this article, we propose a multi-layered approach to automatically identify key columns to expedite preprocessing and aid in analysis of electrified transportation data. Our method leverages a dynamic ontology to identify common fields and an information theory-based strategy for edge cases that are difficult to generalize. Evaluations on a number of datasets from data.gov and kaggle.com show improved performance of our methods over several baseline techniques, and our ablation analyses illustrate the efficacy of individual components of our method. Our case studies also demonstrate that our methods have the potential to improve analysis of electrified transportation data and aid in automatic integration of such datasets.

33 ADVANCED PROPULSION SYSTEMS↗

The Effect of Air Separations on Fast Pyrolysis Products for Forest Residue Feedstocks

This study investigates the intricate relationship between biomass preprocessing and pyrolysis product yields, employing the air classification technique for the treatment of loblolly pine residues with varying moisture content. A comprehensive exploration of the physicochemical properties of air-classified loblolly pine informs a sophisticated pyrolysis simulation model. Given the complex and multifaceted nature of biomass pyrolysis, operating across diverse temporal and spatial scales, a pyrolysis kinetics-based CFD–DEM simulation method is employed to predict product yields. Results showed that the elevated moisture content amplifies particle adhesiveness, necessitating augmented air velocities for effective separation, thereby influencing the efficiency of the separation process. While carbon and hydrogen contents exhibit relative stability across diverse moisture contents and blower frequencies, the oxygen content undergoes noticeable changes. For example, the oxygen contents were measured as 29.2 and 38.6 wt% in the light fraction of 30% moisture content sample at blower frequencies of 10 and 20 Hz, respectively. An intriguing finding emerges from pyrolysis simulation, indicating that a lower blower frequency in air classification moderately enhances bio-oil yield and significantly improves its quality, particularly in terms of water content. For instance, the water content in the bio-oil was about 1.5% and 10% in the heavy and light fractions, respectively from 10% moisture sample under 15 Hz blower frequency. In summary, a detailed understanding and strategic manipulation of critical material attributes in biomass through efficient fractionation techniques are imperative for advancing fast pyrolysis as a sustainable avenue for renewable energy and chemical production.

09 BIOMASS FUELS↗

Using soil library hyperspectral reflectance and machine learning to predict soil organic carbon: Assessing potential of airborne and spaceborne optical soil sensing

Soil organic carbon (SOC) is a key variable to determine soil functioning, ecosystem services, and global carbon cycles. Spectroscopy, particularly optical hyperspectral reflectance coupled with machine learning, can provide rapid, efficient, and cost-effective quantification of SOC. However, how to exploit soil hyperspectral reflectance to predict SOC concentration, and the potential performance of airborne and satellite data for predicting surface SOC at large scales remain relatively underknown. Here, this study utilized a continental-scale soil laboratory spectral library (37,540 full-pedon 350–2500 nm reflectance spectra with SOC concentration of 0–780 g·kg –1 across the US) to thoroughly evaluate seven machine learning algorithms including Partial-Least Squares Regression (PLSR), Random Forest (RF), K-Nearest Neighbors (KNN), Ridge, Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM) along with four preprocessed spectra, i.e. original, vector normalization, continuum removal, and first-order derivative, to quantify SOC concentration. Furthermore, by using the coupled soil-vegetation-atmosphere radiative transfer model, we simulated twelve airborne and spaceborne hyper/multi-spectral remote sensing data from surface bare soil laboratory spectra to evaluate their potential for estimating SOC concentration of surface bare soils. Results show that LSTM achieved best predictive performance of quantifying SOC concentration for the whole data sets (R 2 = 0.96, RMSE = 30.81 g·kg –1 ), mineral soils (SOC ≤ 120 g·kg –1 , R 2 = 0.71, RMSE = 10.60 g·kg –1 ), and organic soils (SOC > 120 g·kg –1 , R 2 = 0.78, RMSE = 62.31 g·kg –1 ). Spectral data preprocessing, particularly the first-order derivative, improved the performance of PLSR, RF, Ridge, KNN, and ANN, but not LSTM or CNN. We found that the SOC models of mineral and organic soils should be distinguished given their distinct spectral signatures. Finally, we identified that the shortwave infrared is vital for airborne and spaceborne hyperspectral sensors to monitor surface SOC. This study highlights the high accuracy of LSTM with hyperspectral/multispectral data to mitigate a certain level of noise (soil moisture <0.4 m 3 ·m –3 , green leaf area < 0.3 m 2 ·m –2 , plant residue <0.4 m 2 ·m –2 ) for quantifying surface SOC concentration. Forthcoming satellite hyperspectral missions like Surface Biology and Geology (SBG) have a high potential for future global soil carbon monitoring, while high-resolution satellite multispectral fusion data can be an alternative.

54 ENVIRONMENTAL SCIENCES↗

Distribution of Bound and Free Water in Anatomical Fractions of Pine Residues and Corn Stover as a Function of Biological Degradation

Biomass quality is influenced by water’s abundance, distribution, and status in relation to other chemical species within the polymer matrix. Water interacts with polymers that make up the cell walls, and these interactions govern the physical and chemical changes that occur during the storage and preprocessing of biomass feedstocks. Time-domain nuclear magnetic resonance (TD-NMR) was employed to explore variations in the physical constraints of water within the lignocellulosic microstructure in distinct anatomical fractions of biomass and as a function of biological degradation. The Carr–Purcell–Meiboom–Gill sequence, when combined with knowledge of the chemical composition and physical structure of pine residues and corn stover anatomical fractions, gives an accurate measurement of the bound and free water. In this work, the impacts of storage and biological degradation were investigated to elucidate changes in the status and distribution of water within distinct plant tissues. We also investigate how degradation during storage affects water interactions in different pine residues (e.g., bark, branch, and needle) and corn stover (e.g., cob, leaf, and stalk) anatomical fractions using transverse relaxation times (T2). As demonstrated herein, TD-NMR provides quantitative data on lignocellulosic biomass–water interactions within anatomical fractions, which can further aid in the investigation of preprocessing effects on feedstock quality. Our findings suggest that biological heating enhances biomass–water interactions at the cellular and macromolecular scale. In addition, analysis of three-dimensional scanning electron microscopy reconstructions indicates that surface roughness wavelengths align with microscale roughness, suggesting that pine forestry residue and corn stover particles have primarily hydrophobic exterior surfaces. Furthermore, this study offers multiscale insights into understanding the microstructure, wettability, and chemical environment that dictate diffusion, enzyme access, and recalcitrance of lignocellulosic biomass.

09 BIOMASS FUELS↗

Comment on “Five Decades of Observed Daily Precipitation Reveal Longer and More Variable Drought Events Across Much of the Western United States”

Abstract Changes in precipitation patterns with climate change could have important impacts on human and natural systems. Zhang et al. (2021, https://doi.org/10.1029/2020gl092293 ) report trends in daily precipitation patterns over the last five decades in the western United States, focusing on meteorological drought. They report that dry intervals (calculated at the annual or seasonal level) have increased across much of the southwestern U.S., with statistical assessment suggesting the results are statistically robust. However, Zhang et al. (2021, https://doi.org/10.1029/2020gl092293 ) preprocess their annual (or seasonal) averages to compute 5‐year moving window averages before using established statistical techniques for trend analysis that assume independence about some fixed trend. Here we show that the moving window preprocessing violates that independence assumption and inflates the statistical significance of their trend estimates. This raises questions about the robustness of their results. We conclude by discussing the difficulty of adjusting for spatial structure when assessing time trends in a regional context.

54 ENVIRONMENTAL SCIENCES↗

Diffractive optical computing in free space

Abstract Structured optical materials create new computing paradigms using photons, with transformative impact on various fields, including machine learning, computer vision, imaging, telecommunications, and sensing. This Perspective sheds light on the potential of free-space optical systems based on engineered surfaces for advancing optical computing. Manipulating light in unprecedented ways, emerging structured surfaces enable all-optical implementation of various mathematical functions and machine learning tasks. Diffractive networks, in particular, bring deep-learning principles into the design and operation of free-space optical systems to create new functionalities. Metasurfaces consisting of deeply subwavelength units are achieving exotic optical responses that provide independent control over different properties of light and can bring major advances in computational throughput and data-transfer bandwidth of free-space optical processors. Unlike integrated photonics-based optoelectronic systems that demand preprocessed inputs, free-space optical processors have direct access to all the optical degrees of freedom that carry information about an input scene/object without needing digital recovery or preprocessing of information. To realize the full potential of free-space optical computing architectures, diffractive surfaces and metasurfaces need to advance symbiotically and co-evolve in their designs, 3D fabrication/integration, cascadability, and computing accuracy to serve the needs of next-generation machine vision, computational imaging, mathematical computing, and telecommunication technologies.

36 MATERIALS SCIENCE↗

GNET2: an R package for constructing gene regulatory networks from transcriptomic data

Abstract Motivation The Gene Network Estimation Tool (GNET) is designed to build gene regulatory networks (GRNs) from transcriptomic gene expression data with a probabilistic graphical model. The data preprocessing, model construction and visualization modules of the original GNET software were developed on different programming platforms, which were inconvenient for users to deploy and use. Results Here, we present GNET2, an improved implementation of GNET as an integrated R package. GNET2 provides more flexibility for parameter initialization and regulatory module construction based on the core iterative modeling process of the original algorithm. The data exchange interface of GNET2 is handled within an R session automatically. Given the growing demand for regulatory network reconstruction from transcriptomic data, GNET2 offers a convenient option for GRN inference on large datasets. Availability and implementation The source code of GNET2 is available at https://github.com/jianlin-cheng/GNET2. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗

Convolutional Neural Networks for Hydrometeor Classification using Dual Polarization Doppler Radars

Traditional fuzzy logic hydrometeor classification algorithm is a common way to classify precipitation type from dual polarization doppler radar. We propose a deep learning-based method to estimate hydrometeors efficiently using observed radar variables such as horizontal reflectivity (Z H ), differential reflectivity (Z DR ), correlation coefficient (ρ HV ) and specific differential phase (K DP ) from National Weather Service NEXRAD collected at Vance AFB facility at the first elevation angle from January 1st, 2015 to July 31th, 2019. We stack matrices of values from these four polarimetric variables as one 3D array. Samples are preprocessed and divided into training, validation and test set with four target hydrometeor categories (Ice Crystals (IC), Dry Snow (DS), Light and/or Moderate Rain (RA) and Big Drops (rain) (BD)). We developed and optimized five Convolutional Neural Networks (CNNs) architectures and achieved an accuracy of 87.23% and 93.736% respectively using modified ResNet with two different input data selection strategies for hydrometeor classification. Training data selection strategies were important to ensure use of available samples in training for robust performance evaluated by applying the models to novel time period beyond what was use to train the model. Seasonal variation in atmospheric conditions lead to seasonal patterns of liquid vs solid forms of precipitation, that poses challenge for classifier and offer insights into domain specific approaches required for problem of hydrometeor identification. Computationally efficient and scalable approach for classification of hydrometeors offer opportunties to effectively use the large volumes of rich time series of radar observations that are becoming increasingly available.

Lu, Yuping↗

Robust Event Classification Using Imperfect Real-world PMU Data

Here, this paper studies robust event classification using imperfect real-world phasor measurement unit (PMU) data. By analyzing the real-world PMU data, we find it is challenging to directly use this dataset for event classifiers due to the low data quality observed in PMU measurements and event logs. To address these challenges, we develop a novel machine learning framework for training robust event classifiers, which consists of three main steps: data preprocessing, fine-grained event data extraction, and feature engineering. Specifically, the data preprocessing step addresses the data quality issues of PMU measurements (e.g., bad data and missing data); in the fine-grained event data extraction step, a model-free event detection method is developed to accurately localize the events from the inaccurate event timestamps in the event logs; and the feature engineering step constructs the event features based on the patterns of different event types, in order to improve the performance and the interpretability of the event classifiers. Based on the proposed framework, we develop a workflow for event classification using the real-world PMU data streaming into the system in real time. Using the proposed framework, robust event classifiers can be efficiently trained based on many off-the-shelf lightweight machine learning models. Numerical experiments using the real-world dataset from the Western Interconnection of the U.S power transmission grid show that the event classifiers trained under the proposed framework can achieve high classification accuracy while being robust against low-quality data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Using Temporal Information from Human Mobility Data to Detect Anchor Points

Spatiotemporal mobility data are available in massive quantities, but large quantities of data typically include fewer variables or data fields. Often, the only available fields are User ID, Longitude, Latitude, Timestamp (ULLT). This raises an important question: how much can we infer about human mobility patterns using only these four fields? With ULLT data, we do not know individuals' socioeconomic status information or when they are visiting their anchor points (AP) or locations (such as homes, places of employment, or schools), and it is a modern challenge to use this data to infer these characteristics. When detecting anchor locations with limited input information, verification and validation (VV) are significant challenges. This paper addresses the problem of identifying individuals' anchor locations using only temporal information from spatiotemporal datasets with limited attributes. Our approach does not explicitly use latitude and longitude during analysis. Locationbased information is only employed in the preprocessing stage to identify periods of movement (trips) and stops (dwelling). Beyond this step, all analysis is based on temporal patterns. In theory, if stops and dwell times could be detected through alternative means, our method could function entirely without location-based input. We demonstrate this methodology on the 2017 National Household Travel Survey (NHTS) data, because it includes a carefully designed and collected time use survey with representative sampling and labeled ground truth. The high-quality survey data allows us to test the accuracy of our methods because NHTS contains intended place labels and agent/user characteristics. We have also applied our validated AP identification algorithm on very large-scale GPS based trajectory data for Patterns-of-Life (PoL) assessment and other applications, but due to space limit that could not be presented here.

McBride, Liz [ORNL] (ORCID:0000000286925869)↗

Data Readiness for Scientific AI at Scale

This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, bio/health, and materials—to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework that combines canonical preprocessing patterns with a five-level operational readiness scale, both tailored to high-performance computing (HPC) environments. This framework helps outline key challenges in transforming large-scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

Brewer, Wes [ORNL] (ORCID:0000000236393956)↗

Distribution Substation Planning Toolkit (dsp-toolkit) v1.0

The Distribution Substation Planning Toolkit (DSP Toolkit) is a software suite designed to streamline the planning and optimization of distribution substations. This toolkit offers a comprehensive set of tools and APIs for data curation, short-term electric load forecasting, and weather-sensitive load adjustment, making it an essential resource for utility companies, engineers, and researchers. Features • Data Preprocessing and Curation: Efficiently manage and preprocess large datasets to ensure high-quality input for analysis. • Short-Term Load Forecasting: Utilize data-driven models to predict short-term electric loads accurately. • Weather-Sensitive Modeling: Automatically adjust load forecasts based on weather data to predict future peak demands more precisely. Uses The DSP Toolkit is ideal for planning and optimizing distribution substations, providing a user-friendly interface and comprehensive documentation. It is suitable for both novice and experienced users, facilitating efficient and accurate planning processes. Advantages • Efficiency: Automates complex planning tasks, reducing manual effort and minimizing errors. • Scalability: Handles large datasets and complex models, making it suitable for large-scale projects. • Community and Support: Open-source with active community contributions, ensuring continuous improvement and support. • Extensibility: Easily extendable with custom modules and plugins, allowing users to tailor the toolkit to their specific needs. The DSP Toolkit stands out by offering a robust, flexible, and user-friendly solution for distribution substation planning. Public Abstract

Li, Han [Lawrence Berkeley National Laboratory (LB↗