Adapting quantile mapping to bias correct solar radiation data
Not Available
Bailey, Maggie D. (ORCID:0000000347017849)↗
SEARCH · Engineering Papers
Results for “Preprocessing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Designing resilient IoT and Edge Computing with federated tinyML
The rapid growth of the Internet of Things (IoT) and Edge Computing (EC) has brought significant conveniences to modern society but has also greatly expanded the cyber attack surfaces, particularly as these technologies are being increasingly integrated into critical systems such as power grids, healthcare, and smart homes. Here, to improve IoT/EC’s cybersecurity posture, we leveraged Artificial Intelligence (AI) and Machine Learning (ML) by employing tinyML to monitor voluminous IoT data for cyber threats while addressing devices’ resource constraints, and utilizing Federated Learning (FL) to share local detection knowledge across the system while preserving privacy. Building on our three-layer architecture combining tinyML and FL to enhance autonomous cyber attack detection, this paper demonstrated that the architecture improves detection accuracy, reduces resource consumption, and enables lightweight, secure IoT device monitoring. These results were validated using the public N-BaIoT dataset as well as real IoT network traffic data collected under multiple attack scenarios from our testbeds. Additionally, we introduced an enhanced FL methodology with a novel preprocessing stage, including federated feature selection and global preprocessor construction, to address IoT/EC data heterogeneity. We developed a physical IoT testbed for attack simulations and data collection, implemented a tinyML-powered detector for realistic model validation, and also built a virtual testbed for scalable evaluations of FL models across diverse network environments.
Cognitive cyber↗
Data-driven method for electric vehicle charging demand analysis: Case study in Virginia
Electric vehicle (EV) adoption in the U.S. will be accelerated by the historic $7.5 billion public investments in EV charging infrastructure. Careful analysis of EV charging demands plays a vital role in understanding the energy requirements, power grid impact, and smart charging management opportunities of EVs. To this end, this paper develops a data-driven trip-chaining-based modeling framework including five steps: Trip data acquisition and preprocessing, EV adoption modeling, travel itinerary synthesis, EV charging demand simulation and EV load profile generation. The developed analysis framework was demonstrated using real-world data for one region in Virginia, U.S. The results show that the proposed modeling framework can work effectively. For the study region in 2040, the predicted number of plug-in EVs is 470,114, resulting in a weekly charging demand of 38,078,127 kWh (55% home, 9% work, and 36% public) in September and 45,920,358 kWh (61% home, 9% work, and 30% public) in February.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Statistical characterization of experimental magnetized liner inertial fusion stagnation images using deep-learning-based fuel–background segmentation
Significant variety is observed in spherical crystal x-ray imager (SCXI) data for the stagnated fuel–liner system created in Magnetized Liner Inertial Fusion (MagLIF) experiments conducted at the Sandia National Laboratories Z-facility. As a result, image analysis tasks involving, e.g., region-of-interest selection (i.e. segmentation), background subtraction and image registration have generally required tedious manual treatment leading to increased risk of irreproducibility, lack of uncertainty quantification and smaller-scale studies using only a fraction of available data. We present a convolutional neural network (CNN)-based pipeline to automate much of the image processing workflow. This tool enabled batch preprocessing of an ensemble of N scans = 139 SCXI images across N exp = 67 different experiments for subsequent study. The pipeline begins by segmenting images into the stagnated fuel and background using a CNN trained on synthetic images generated from a geometric model of a physical three-dimensional plasma. The resulting segmentation allows for a rules-based registration. Our approach flexibly handles rarely occurring artifacts through minimal user input and avoids the need for extensive hand labelling and augmentation of our experimental dataset that would be needed to train an end-to-end pipeline. Here we also fit background pixels using low-degree polynomials, and perform a statistical assessment of the background and noise properties over the entire image database. Our results provide a guide for choices made in statistical inference models using stagnation image data and can be applied in the generation of synthetic datasets with realistic choices of noise statistics and background models used for machine learning tasks in MagLIF data analysis. We anticipate that the method may be readily extended to automate other MagLIF stagnation imaging applications.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Comparing Sensor Fusion and Multimodal Chemometric Models for Monitoring U(VI) in Complex Environments Representative of Irradiated Nuclear Fuel
Optical sensors and chemometric models were leveraged for the quantification of uranium(VI) (0–100 μg mL –1 ), europium (0–150 μg mL –1 ), samarium (0–250 μg mL –1 ), praseodymium (0–350 μg mL –1 ), neodymium (0–1000 μg mL –1 ), and HNO 3 (2–4 M) with varying corrosion product (iron, nickel, and chromium) levels using laser fluorescence, Raman scattering, and ultraviolet–visible–near-infrared absorption spectra. In this paper, an efficient approach to developing and evaluating tens of thousands of partial least-squares regression (PLSR) models, built from fused optical spectra or multimodal acquisitions, is discussed. Each PLSR model was optimized with unique preprocessing combinations, and features were selected using genetic algorithm filters. The 7-factor D-optimal design training set contained just 55 samples to minimize the number of samples. The performance of PLSR models was evaluated by using an automated latent variable selection script. PLS1 regression models tailored to each species outperformed a global PLS2 model. PLS1 models built using fused spectra data and a multimodal (i.e., analyzed separately) approach yielded similar information, resulting in percent root-mean-square error of prediction values of 0.9–5.7% for the seven factors. Further, the optical techniques and data processing strategies established in this study allow for the direct analysis of numerous species without measuring luminescence lifetimes or relying on a standard addition approach, making it optimal for near-real-time, in situ measurements. Nuclear reactor modeling helped bound training set conditions and identified elemental ratios of lanthanide fission products to characterize the burnup of irradiated nuclear fuel. Leveraging fluorescence, spectrophotometry, experimental design, and chemometrics can enable the remote quantification and characterization of complex systems with numerous species, monitor system performance, help identify the source of materials, and enable rapid high-throughput experiments in a variety of industrial processes and fundamental studies.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
Absorption Correction for Reliable Pair Distribution Functions from Low Energy X-ray Sources
This paper explores the development and testing of a simple absorption correction model for processing powder X-ray diffraction data from Debye−Scherrer geometry laboratory X-ray experiments. This may be used as a preprocessing step before using PDFGETX3 to obtain reliable pair distribution functions (PDFs). Various experimental and theoretical methods for estimating μR were explored, and the most appropriate μR values for correction were identified for different capillary diameters and X-ray beam sizes. We identify operational ranges of μR where a reasonable signal-to-noise ratio is possible after correction. A user-friendly software package, DIFFPY.LABPDFPROC, is presented that can help estimate μR and perform absorption corrections with a rapid calculation for efficient processing.
Absorption↗
Multiscale Catalytic Fast Pyrolysis of Grindelia Reveals Opportunities for Generating Low Oxygen Content Bio-Oils from Drought Tolerant Biomass
Grindelia squarrosa (curlycup gumweed) biomass possesses unique biochemistry, cell wall composition, and leaf architecture tailored for prolific growth in arid and semiarid climates. Most notably, this plant has developed high levels of extractable resins that have high effective H/C eff ratios ((mol H - 2 x mol O)/mol C), which is hypothesized to lead to low coke formation during catalytic fast pyrolysis (CFP) over the ZSM-5 catalyst. In microscale experiments with high ZSM-5 loadings (biomass-to-catalyst mass ratio (B/C) ~ 0.1), in situ CFP generated high yields of aromatic hydrocarbons (30% carbon yield) while ex situ CFP favored aliphatic hydrocarbons (25% carbon yield). The difference between the two configurations was attributed to the constant catalyst temperature during ex situ CFP. Deactivation leading to partially deoxygenated vapor products occurred rapidly until B/C ≤ 0.5 by the adsorption of organic species blocking access to acid sites inside the micropores of the catalyst. This was followed by more gradual deactivation leading to primary vapor breakthrough, which we attribute to coke formation on acid sites on the external surface of ZSM-5 crystallites. Noncatalytic fast pyrolysis of Grindelia in a bench scale reactor produced oils with oxygen content (18 wt % on dry basis) and carbon yield (33%) comparable to those of CFP of woody biomass. The CFP of Grindelia further reduced the oxygen content to 7 wt % for in situ CFP and 4 wt % for ex situ CFP at B/C of 2-3. The good deoxygenation was attributed to a combination of a high H/C eff ratio and overall better quality of the pyrolysis vapors that were passed over the ZSM-5 catalyst. The high inorganic content of the Grindelia likely catalyzed pyrolysis to remove oxygenated coke precursors. This integrated CFP study demonstrated that Grindelia could be an important feedstock for generating stabilized noncatalytic and CFP oils for downstream processing into fuels and/or extraction of high-value chemicals. The preprocessing of this feedstock will be required to remove inorganics, which cause an irreversible deactivation of ZSM-5.
BIOMASS FUELS↗
Quantifying Impacts of Biomass Pelletization on Fast Pyrolysis Using a Single-Particle Reactor, X-ray Computed Tomography, and Computational Modeling
The pore structure and density of lignocellulosic feedstocks dictate intraparticle transport phenomena and thereby play an important role in thermochemical conversion processes such as fast pyrolysis for biofuel and biochemical production. Variations in microstructure are inherent from different biomass species and can be introduced by preprocessing techniques such as cutting and pelletization. Morphological changes also occur during conversion and lead to vastly different pore structures and behavior during pyrolysis, which impact required conversion times and product distributions. The current work presents a comprehensive comparison of fast pyrolysis of neat and pelletized pine feedstocks, which includes single-particle experiments, modeling, and 3D imaging by X-ray computed tomography (XCT). The particle-scale model included anisotropic heat and mass transport in a shrinking particle with pyrolysis reactions based on the CRECK mechanism with boundary conditions informed by reactor-scale simulations of the single-particle reactor. The models were validated by measurements of the temperature and mass loss from single-particle pyrolysis experiments of neat and pelletized pine. Quantitative analysis of XCT geometries revealed that pyrolytic conversion yielded chars with increased porosity and permeability compared to the unpyrolyzed materials, along with decreased tortuosity and anisotropy. Pelletization of the pine feedstock resulted in a much denser, less permeable material, which converted slower and produced more residual char after pyrolysis compared to neat pine. The results from particle modeling revealed that accounting for the dynamic and anisotropic heat and mass transport caused by differences in pore structure is critical to achieving agreement with experimental results. Overall, this study highlights the dramatic differences in conversion behavior imparted by pelletization and the importance of capturing microstructural attributes in computational models to guide the design and optimization of pyrolysis processes for specific biomass feedstocks.
09 BIOMASS FUELS↗
Process Feasibility Analysis of Waste Biomass Valorization to Biochar and Bio-Oil via Slow and Fast Pyrolysis
The United States has abundant biomass and waste feedstock to support the nation's energy addition and affordability targets. Pyrolysis, a thermochemical conversion process, decomposes lignocellulosic feedstocks into liquid, solid, and gaseous fuels that can contribute to the domestic production of biofuels, biopower, and bioproducts. Growing private sector interest in this technology is a key motivation for this comprehensive techno-economic process modeling analysis of a respective biorefinery that includes feedstock preprocessing, slow and fast pyrolysis, and product separation to bio-oil, biochar, and syngas hydrocarbons. Results show that biochar from slow pyrolysis could achieve minimum selling prices (MSPs) of $\$$188-$\$$260/t, competitive with reported market values, while bio-oil from fast pyrolysis is estimated to yield MSPs of $\$$6.49-$\$$9.68/GGE, approximately twice conventional fuel benchmarks. Sensitivity analysis identifies feedstock cost, product yield, and scale as primary cost drivers, while scenarios involving biochar carbon credits and high value applications may substantially improve economics. Overall, these results suggest that continued innovation in feedstock logistics, process integration, and market development will be critical to achieving economically viable and scalable bioproducts.
09 BIOMASS FUELS↗
Resource-Efficient Chemistry on Quantum Computers with the Variational Quantum Eigensolver and The Double Unitary Coupled-Cluster approach
Applications of quantum simulation algorithms to obtain electronic energies of molecules on noisy intermediate-scale quantum (NISQ) devices require careful consideration of resources describing the complex electron correlation effects. In modeling second-quantized problems, the biggest challenge confronted is that the number of qubits scales linearly with the size of molecular basis. This poses a significant limitation on the size of the basis sets and the number of correlated electrons included in quantum simulations of chemical processes. To address this issue and to enable more realistic simulations on NISQ computers, we employ the double unitary coupled-cluster (DUCC) method to effectively downfold correlation effects into the reduced-size orbital space, commonly referred to as the active space. Using downfolding techniques, we demonstrate that properly constructed effective Hamiltonians can capture the effect of the whole orbital space in small-size active spaces. Combining the downfolding pre-processing technique with the Variational Quantum Eigensolver, we solve for the ground-state energy of H2 and Li2 in the cc-pVTZ basis using the DUCC-reduced active spaces. We compare these results to full configuration-interaction and high-level coupled-cluster reference calculations.
quantum computing, variational quantum solver, cou↗
Aggregation Methods for Quantifying PTM and Structural Changes in Bottom-Up Proteomics
Bottom-up proteomic workflows rely on sequential preprocessing steps, commonly including peptide-to-protein aggregation (“roll-up”), to enhance data reliability and interpretability. While roll-up is effective for protein-centered analyses, it may be suboptimal for applications focused on post-translational modifications (PTMs) or protein structural changes, such as limited proteolysis–mass spectrometry (LiP-MS). Here, we investigate how different roll-up strategies influence site-level quantification in PTM differential analysis. Moreover, we introduce a novel site-centric roll-up approach tailored for LiP-MS, which quantifies proteolytic fragments rather than solely tryptic peptides. We benchmark these methods through simulation studies, comparing their sensitivity and specificity in detecting structural and PTM-driven changes. We found that the median and mean roll-up methods outperform the sum method in both PTM and LiP proteomics, and site-level quantification in LiP outperforms peptide-level quantification. Our findings offer the first systematic, data-driven guidance for selecting roll-up techniques in site-level proteomic analyses, with implications for both PTM-focused and structural proteomics studies.
aggregation↗
Tutorial: Machine-Learning-Based CREASE-2D Analysis of 2D SAXS Profiles to Characterize Anisotropic Nanostructures in Soft Materials
We present a tutorial to guide users on how to extend the Computational Reverse Engineering Analysis of Scattering Experiments-2D (CREASE-2D) framework to interpret their experimental two-dimensional small-angle scattering (SAS) data from soft materials (e.g., polymers, peptide amphiphiles, biomolecular fibrils). Unlike most traditional SAS analysis approaches, which typically rely on azimuthally averaged onedimensional (1D) profiles, CREASE-2D utilizes the complete 2D scattering profile to reveal information about anisotropy in the structure. In past applications, CREASE has provided insights into complex structural features, including the cross-sectional shapes of assembled nanostructures and dispersity in these features, which are difficult to discern with existing analytical models. While (1D- ) CREASE has been applied to SANS and SAXS data, this tutorial shares the steps for implementing CREASE-2D using an example of a dipeptide solution system, for which we have SAXS data. We present details for these steps involved in using CREASE-2D to interpret SAXS profiles: how to preprocess SAXS data, define relevant structural features, generate three-dimensional real-space structures for specific values of these features, train a machine learning (ML) surrogate model to predict scattering profiles for given structural features, and optimize these features using genetic algorithms (GA). Then, we use these steps to interpret complex 2DSAXS data collected from dipeptide solutions that, in microscopy images, exhibit nanoscale structures that could be elliptical tubes/ flat tapes/cylinders or a combination of these cross sections. Open-source codes, computational hardware, and software requirements, as well as the strengths and limitations of this protocol, are also presented. We expect researchers working with (soft) biomaterials, peptide amphiphiles, amphiphilic polymer solutions, polymer nanocomposites, and blends of particles/polymers will find this CREASE-2D method and this tutorial of use.
CREASE↗
Composition-Preserving Extraction and Characterization of Biomass Extrinsic and Intrinsic Inorganic Compounds
The inorganic content of biomass impairs size reduction tool life and the conversion process. Conventional ash extraction relies on furnace combustion that inevitably alters the inorganic compounds due to oxidation and decomposition. This study developed composition-preserving methods for extracting and analyzing extrinsic and intrinsic inorganic compounds. Comprehensive characterization was carried out on selected biomass feedstocks, including corn stover, pine residue, and pine anatomical fractions, to reveal their inorganic species and morphology. The extrinsic inorganic compounds were found to be dominated by quartz, along with other minor minerals, such as albite, microcline, and gehlenite, and have particle sizes ranging from tens to hundreds of micrometers. Among the pine anatomical fractions, the needles contain the highest intrinsic silicon content while the bark trapped the most extrinsic minerals. By correlation of the total ash and extrinsic inorganic contents to the wear behavior, both the extrinsic and intrinsic inorganic compounds were concluded to have made significant contributions to the wear process. Finally, the results here validated a new approach to characterize inorganic compounds in biomass and provided fundamental insights for their potential impact on preprocessing tool wear.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Signatures of Biologically Driven Hemicellulose Modification Quantified by Analytical Pyrolysis Coupled with Multidimensional Gas Chromatography Mass Spectrometry
Biomass storage operation is one major source of feedstock quality variability that impacts on downstream preprocessing, feeding, handling and conversion into biofuels, chemicals and products. In field Ccorn stover biomass bales undergoes degradation and cell wall components modification resulting from microbial biological heating occurring during storage. A molecular characterization of the modification was achieved to unveil and understand the changes of structural cell wall modification in corn stover samples by conducting low temperature (400°C) pyrolysis, comparing samples that were mildly biologicallyun heated to those that had been subjected to microbial severe biological heating (~60°C). Pyrolysis of the samples originating from biomass that had been heated during storage generated small oxygenates such as furfural, 5-methylfurfural and 2-(5H)-furanone with efficiencies that were as much as ten times greater than those measured for the mildly unheated samples. Most of the compounds having enhanced efficiencies were C5 oxygenates, suggesting that the pyrolysis products may be formed from hemicellulosic precursor polymers in the corn stover. The findings suggest that microbial heating may be disrupting the cell wall structure, fragmenting the hemicellulose or cellulose chains, generating more polymer termini that have higher efficiency for generating the oxygenates at lower temperatures. Pyrolysis variability with temperature may be a beneficial strategy for improved biomass cell wall characterization, and for guiding control strategies for enhanced control over product distributions , providing insights to understand and manage the feedstock variability as well as to inform harvest and storage best management practices.
09 BIOMASS FUELS↗
Dynamic Life-Cycle Analysis of Fast Pyrolysis Biorefineries: Impacts of Feedstock Moisture Content and Particle Size
The commercialization of biorefineries has been met with significant obstacles due to the technical difficulties of handling solid biomass feedstocks, which are highly variable in physical properties and chemical compositions. An understanding of the principles of unit operations along the supply chain is key to the success of the biofuel industry. This study applies a dynamic life-cycle analysis (DLCA) methodology by developing quantitative relationships between the inputs and outputs for each unit operation based on scientific understanding of the causal effects. Using the DLCA, we assessed system sustainability of drop-in fuel production from fast pyrolysis of pine residues followed by hydro-processing. Life-cycle greenhouse gas (GHG) emissions were calculated for 4641 runs involving two key feedstock parameters: moisture content after field drying and particle size of the feed to pyrolyzer. Moisture contents varied between 25% and 35%, while particle sizes varied between 0.5 and 5 mm in these runs. The life-cycle GHG emissions of these cases vary from 22 to 40 g/MJ. Close examination of each unit operation's contribution to the total GHG emissions reveals that low moisture content after field drying produces savings in energy consumption during feedstock transportation and on-site drying. However, although small particle size leads to overall higher fuel yield, it also requires a significant amount of energy for feedstock size reduction and fuel production. The energy penalty outweighs the benefit of increased fuel yields, especially when fine particles (<1 mm) are used for pyrolysis; thus, small particle size leads to increased GHG emissions overall. The results highlight the trade-offs between the energy demand for preprocessing and the conversion yields, which can be addressed with DLCA.
09 BIOMASS FUELS↗
Defect detection in atomic-resolution images via unsupervised learning with translational invariance
Abstract Crystallographic defects can now be routinely imaged at atomic resolution with aberration-corrected scanning transmission electron microscopy (STEM) at high speed, with the potential for vast volumes of data to be acquired in relatively short times or through autonomous experiments that can continue over very long periods. Automatic detection and classification of defects in the STEM images are needed in order to handle the data in an efficient way. However, like many other tasks related to object detection and identification in artificial intelligence, it is challenging to detect and identify defects from STEM images. Furthermore, it is difficult to deal with crystal structures that have many atoms and low symmetries. Previous methods used for defect detection and classification were based on supervised learning, which requires human-labeled data. In this work, we develop an approach for defect detection with unsupervised machine learning based on a one-class support vector machine (OCSVM). We introduce two schemes of image segmentation and data preprocessing, both of which involve taking the Patterson function of each segment as inputs. We demonstrate that this method can be applied to various defects, such as point and line defects in 2D materials and twin boundaries in 3D nanocrystals.
36 MATERIALS SCIENCE↗
Forecasting of in situ electron energy loss spectroscopy
Abstract Forecasting models are a central part of many control systems, where high-consequence decisions must be made on long latency control variables. These models are particularly relevant for emerging artificial intelligence (AI)-guided instrumentation, in which prescriptive knowledge is needed to guide autonomous decision-making. Here we describe the implementation of a long short-term memory model (LSTM) for forecasting in situ electron energy loss spectroscopy (EELS) data, one of the richest analytical probes of materials and chemical systems. We describe key considerations for data collection, preprocessing, training, validation, and benchmarking, showing how this approach can yield powerful predictive insight into order-disorder phase transitions. Finally, we comment on how such a model may integrate with emerging AI-guided instrumentation for powerful high-speed experimentation.
36 MATERIALS SCIENCE↗
Cryo2StructData: A Large Labeled Cryo-EM Density Map Dataset for AI-based Modeling of Protein Structures
The advent of single-particle cryo-electron microscopy (cryo-EM) has brought forth a new era of structural biology, enabling the routine determination of large biological molecules and their complexes at atomic resolution. The high-resolution structures of biological macromolecules and their complexes significantly expedite biomedical research and drug discovery. However, automatically and accurately building atomic models from high-resolution cryo-EM density maps is still time-consuming and challenging when template-based models are unavailable. Artificial intelligence (AI) methods such as deep learning trained on limited amount of labeled cryo-EM density maps generate inaccurate atomic models. To address this issue, we created a dataset called Cryo2StructData consisting of 7,600 preprocessed cryo-EM density maps whose voxels are labelled according to their corresponding known atomic structures for training and testing AI methods to build atomic models from cryo-EM density maps. Cryo2StructData is larger than existing, publicly available datasets for training AI methods to build atomic protein structures from cryo-EM density maps. We trained and tested deep learning models on Cryo2StructData to validate its quality showing that it is ready for being used to train and test AI methods for building atomic models.
59 BASIC BIOLOGICAL SCIENCES↗