Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

Data for Yield from Iowa’s first commercial miscanthus fields: implications of spatial variability for productivity and sustainability beyond research plots

This dataset contains biomass yield measurements and associated vegetation index data collected from commercial Miscanthus × giganteus fields in eastern Iowa during the 2022–2023 growing seasons. The data support the analyses presented in the article: “Yield From Iowa's First Commercial Miscanthus Fields: Implications of Spatial Variability for Productivity and Sustainability Beyond Research Plots.” We collected 105 ground-truth biomass samples from four mature commercial fields (>4 years old) covering 92.81 ha. Samples were taken from 3 m² quadrats that were hand-harvested in alignment with commercial harvest timing. Stem biomass (excluding leaves) was weighed, moisture-corrected, and converted to dry-matter yield expressed in Mg DM ha⁻¹. Sampling locations were selected to capture spatial variability visible in aerial imagery and were recorded using RTK GPS. Each biomass observation was paired with vegetation indices derived from high-resolution PlanetScope satellite imagery (3 m resolution). Images were acquired throughout the growing season, and indices were calculated to evaluate their ability to predict end-of-season biomass yield. Statistical and machine learning approaches were used to identify key predictors, and a linear regression model based on end-of-July Green Normalized Difference Vegetation Index (GNDVI) was developed and evaluated. This repository includes the data used in that modeling workflow. Management practices, economic data, full imagery time series, and additional methodological details are described in the associated publication and are not included here. The dataset consists of three comma-separated value (CSV) files: 1. Combine_Groundtruth_Yield_VI_22_23.csv This file contains ground-truth biomass yield measurements and associated key vegetation index values collected during the 2022 and 2023 growing seasons. Rows: 105 observations Columns: Year — Year of observation (2022 or 2023) Field — Field location identifier Sample_number — Unique sample identifier GNDVI_End_Jul — Green Normalized Difference Vegetation Index calculated at end of July GNDVI_End_Aug — Green Normalized Difference Vegetation Index calculated at end of August NDRE_End_Aug — Normalized Difference Red Edge index calculated at end of August Biomass_Stem_Yield_MgDM/ha — Measured stem biomass yield (megagrams dry matter per hectare) 2. trainData_GNDVI.csv This file contains the subset of observations used to train the predictive relationship between July GNDVI and biomass yield. Rows: 76 observations Columns: Unnamed: 0 — Row index retained from the original data processing workflow GNDVI_End_Jul — GNDVI at end of July Stem_Yield_MgDM/ha — Observed stem biomass yield (Mg DM ha⁻¹) 3. testData_GNDVI.csv This file contains the test dataset used to evaluate model performance. Rows: 29 observations Columns: Unnamed: 0 — Row index retained from the original data processing workflow GNDVI_End_Jul — GNDVI at end of July Predicted_Yield_MgDM/ha — Model-predicted stem biomass yield (Mg DM ha⁻¹) Observed_Yield_MgDM/ha — Measured stem biomass yield (Mg DM ha⁻¹)

Potential yield, yield gap, in-field management, y↗

Characterization of contaminants in the Lyman-alpha forest auto-correlation with DESI

Baryon Acoustic Oscillations can be measured with sub-percent precision above redshift two with the Lyman-α (Lyα) forest auto-correlation and its cross-correlation with quasar positions. This is one of the key goals of the Dark Energy Spectroscopic Instrument (DESI) which started its main survey in May 2021. We present in this paper a study of the contaminants to the Lyα forest which are mainly caused by correlated signals introduced by the spectroscopic data processing pipeline as well as astrophysical contaminants due to foreground absorption in the intergalactic medium. Notably, an excess signal caused by the sky background subtraction noise is present in the Lyα auto-correlation in the first line-of-sight separation bin. We use synthetic data to isolate this contribution, we also characterize the effect of spectro-photometric calibration noise, and propose a simple model to account for both effects in the analysis of the Lyα forest. We then measure the auto-correlation of the quasar flux transmission fraction of low redshift quasars, where there is no Lyα forest absorption but only its contaminants. We demonstrate that we can interpret the data with a two-component model: data processing noise and triply ionized Silicon and Carbon auto-correlations. This result can be used to improve the modeling of the Lyα auto-correlation function measured with DESI.

79 ASTRONOMY AND ASTROPHYSICS↗

The Midlatitude Continental Convective Clouds Experiment (MC3E) sounding network: operations, processing and analysis

The Midlatitude Continental Convective Clouds Experiment (MC3E) took place during the spring of 2011 centered in north-central Oklahoma, USA. The main goal of this field campaign was to capture the dynamical and microphysical characteristics of precipitating convective systems in the US Central Plains. A major component of the campaign was a six-site radiosonde array designed to capture the large-scale variability of the atmospheric state with the intent of deriving model forcing data sets. Over the course of the 46-day MC3E campaign, a total of 1362 radiosondes were launched from the enhanced sonde network. This manuscript provides details on the instrumentation used as part of the sounding array, the data processing activities including quality checks and humidity bias corrections and an analysis of the impacts of bias correction and algorithm assumptions on the determination of convective levels and indices. It is found that corrections for known radiosonde humidity biases and assumptions regarding the characteristics of the surface convective parcel result in significant differences in the derived values of convective levels and indices in many soundings. In addition, the impact of including the humidity corrections and quality controls on the thermodynamic profiles that are used in the derivation of a large-scale model forcing data set are investigated. The results show a significant impact on the derived large-scale vertical velocity field illustrating the importance of addressing these humidity biases.

54 ENVIRONMENTAL SCIENCES↗

Enabling energy‐efficient manufacturing of pharmaceutical solid oral dosage forms via integrated techno‐economic analysis and advanced process modeling

Abstract The global pharmaceutical industry is a trillion‐dollar market. However, the pharmaceutical sector often lags in manufacturing innovation and automation which limits its potential to maximize energy efficiency. The integration of techno‐economic analysis (TEA) with advanced process models as part of an overarching smart manufacturing platform, can help industries create business models, which can be adapted for manufacturing to reduce energy consumption and operating costs while ensuring product quality which can further enable a more sustainable process operation. In this study, a rational design of experiment on three unit‐operations (wet granulation, drying, and milling) was performed on a batch (case 1) and continuous (case 2) pharmaceutical process to obtain experimental data. Process models for predicting product quality and energy efficiency of each of the three‐unit operations were developed. The experimental data were used to validate the models and good agreement was observed. The energy consumption of each unit operation was calculated using statistical models relating the power consumption and the process parameters. The developed process models and energy models were further integrated into a TEA framework, which quantified the energy and monetary cost of manufacturing for both batch and continuous manufacturing cases. With this integrated framework, energy costs savings of ~33% was obtained in the continuous manufacturing process (case 2) over the batch process (case 1).

Sampat, Chaitanya↗

Time-warping invariant quantum recurrent neural networks via quantum-classical adaptive gating

Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNNs), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations. This paper builds on quantum RNNs (QRNNs), a dynamic model with quantum memory, to introduce a novel class of temporal data processing quantum models that preserve invariance to time-warping transformations of the (classical) input-output sequences. The model, referred to as time warping-invariant QRNN (TWI-QRNN), augments a QRNN with a quantum–classical adaptive gating mechanism that chooses whether to apply a parameterized unitary transformation at each time step as a function of the past samples of the input sequence via a classical recurrent model. The TWI-QRNN model class is derived from first principles, and its capacity to successfully implement time-warping transformations is experimentally demonstrated on examples with classical or quantum dynamics.

97 MATHEMATICS AND COMPUTING↗

Near Real-Time Two and Three-Dimensional Electrical Geophysical Monitoring of Natural and Engineered Processes Associated with Subsurface Remediation - 16052

Legacy soil and groundwater contamination associated with nuclear weapons production is projected to be one of the largest cleanup liabilities to the U.S. Department of Energy for the foreseeable future. Remediation of subsurface contaminants is complicated by the combined effects of geochemical heterogeneity, geological heterogeneity, and the difficulty and expense of adequate borehole access. Each of these complications contributes to uncertainty in the distribution of contaminants and the performance of in situ treatments, which increases risk to human health and the environment and cost to closure in general. Electrical resistivity tomography (ERT) is a method of remotely imaging the electrical properties of the subsurface, which are governed by both geochemical and geological structure, thereby providing useful proxies for understanding contaminant distribution and the behavior in situ remediation processes. Recent advancements in both data collection and data processing capabilities are enabling ERT monitoring to be executed and provided to site operators in near real-time. To summarize the process, ERT surveys are rapidly and continuously collected during a time-sensitive operation such as an amendment injection. At the completion of each survey, data are transferred by wireless internet to offsite supercomputing resources for parallel tomographic inversion. Inversion results are then transferred back to onsite operators and/or other locations for visualization. In this paper, we demonstrate the utility of time-lapse ERT for monitoring fluid transport (both gas and liquid phase). We then demonstrate two real-time imaging applications for monitoring both natural and engineered processes at the Hanford Site within the vadose and the saturated zones. We expect this new capability for autonomous real-time imaging to enable ERT to be used as a cost-effective, short-term rapid feedback mechanism for guiding subsurface remediation injections, in addition to longer term monitoring of post-injection performance and environmental impact, all using the same electrode array and instrumentation. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Data Reduction Processes Using FPGA for MicroBooNE Liquid Argon Time Projection Chamber

MicroBooNE is a liquid Argon time projection chamber to be built at Fermilab for an accelerator-based neutrino physics experiment and as part of the R&D strategy for a large liquid argon detector at DUSEL. The waveforms of the {approx}9000 sense wires in the chamber are continuously digitized at 2 M samples/s - which results in a large volume of data coming off the TPC. We have developed a lossless data reduction scheme based on Huffman Coding and have tested the scheme on cosmic ray data taken from a small liquid Argon time projection chamber, the BO detector. For sense wire waveforms produced by cosmic ray tracks, the Huffman Coding scheme compresses the data by a factor of approximately 10. The compressed data can be fully recovered back to the original data since the compression is lossless. In addition to accelerator neutrino data, which comes with small duty cycle in sync with the accelerator beam spill, continuous digitized waveforms are to be temporarily stored in the MicroBooNE data-acquisition system for about an hour, long enough for an external alert from possible supernova events. Another scheme, Dynamic Decimation, has been developed to compress further the potential supernova data so that the storage can be implemented within a reasonable budget. In the Dynamic Decimation scheme, data are sampled at the full sampling rate in the regions-of-interest (ROI) containing waveforms of track-hits and are decimated down to lower sampling rate outside the ROI. Note that unlike in typical zerosuppression schemes, in Dynamic Decimation, the data in the pedestal region are not thrown away but kept at a lower sampling rate. An additional factor of 10 compression ratio is achieved using the Dynamic Decimation scheme on the BO detector data, making a total compression rate of approximate 100 when the Dynamic Decimation and the Huffman Coding functional blocks are cascaded. Both of the blocks are compiled in low-cost FPGA and their silicon resource usages are low.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

In-depth analysis on parallel processing patterns for high-performance Dataframes

The Data Science domain has expanded monumentally in both research and industry communities during the past decade, predominantly owing to the Big Data revolution. Artificial Intelligence (AI) and Machine Learning (ML) are bringing more complexities to data engineering applications, which are now integrated into data processing pipelines to process terabytes of data. Typically, a significant amount of time is spent on data preprocessing in these pipelines, and hence improving its efficiency directly impacts the overall pipeline performance. The community has recently embraced the concept of Dataframes as the de-facto data structure for data representation and manipulation. However, the most widely used serial Dataframes today (R, pandas) experience performance limitations while working on even moderately large data sets. We believe that there is plenty of room for improvement by taking a look at this problem from a high-performance computing point of view. In a prior publication, we presented a set of parallel processing patterns for distributed dataframe operators and the reference runtime implementation, Cylon. In this paper, we are expanding on the initial concept by introducing a cost model for evaluating the said patterns. Furthermore, we evaluate the performance of Cylon on the ORNL Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗

Computational Removal of Undesired Mass Spectral Features Possessing Repeat Units via a Kendrick Mass Filter

Polymers are a common component of chemical background which complicates data analysis and can impair interpretation. Undesired chemical background cannot always be addressed via pre-analytical methods, chromatography, or existing data processing methods. The Kendrick mass filter (KMF) is presented for the computational removal of undesired signals present in MS{sup 1} spectra. The KMF is analogous to mass defect filtering but utilizes homology information via Kendrick mass scaling in combination with chromatographic retention time and the number of observed signals. The KMF is intended to assist in situations in which current data processing methods to remove background, e.g., blank subtraction, are either not possible or effective. The major parameters affecting KMF were investigated using PEG 400 and NIST standard reference material 1950 (metabolites in human plasma). Further exploration of the KMF performance was tested using an extract of a swab known to contain polymers. An illustrative real-world example of skin analysis with polymeric signal is discussed. The KMF is also able to provide a high-level view of the compositionality of data regarding the presence of signals with repeat units and indicate the presence of different polymers. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Long-Term Statistical Process Monitoring of an Ultrafiltration Water Treatment Process

As water treatment technology has improved, the amount of available process data has substantially increased, making real-time, data-driven fault detection a reality. One shortcoming of the fault detection literature is that methods are usually evaluated by comparing their performance on hand-picked, short-term case studies, which yields no insight into long-term performance. In this work, we first evaluate multiple statistical and machine learning approaches for detrending process data. Then, we evaluate the performance of a PCA-based fault detection approach, applied to the detrended data, to monitor influent water quality, filtrate quality, and membrane fouling of an ultrafiltration membrane system for indirect potable reuse. Based on two short case studies, the adaptive lasso detrending method is selected, and the performance of the multivariate approach is evaluated over more than a year. The method is tested for different sets of three critical tuning parameters, and we find that for long-term, autonomous monitoring to be successful, these parameters should be carefully evaluated. However, in comparison with industry standards of simpler, univariate monitoring or daily pressure decay tests, multivariate monitoring produces substantial benefits in long-term testing.

ammonia↗

Integration of the FastIC front-end electronics into the Picosec MicroMegas detector

The Picosec MicroMegas collaboration aims to develop gaseous fast-timing detectors; experimentally, intrinsic time resolutions from around 50 ps to better than 20 ps are obtained, depending on the exact detector configuration. Parts of developments focus on exploring various options of fast-timing multi-channel front-end electronics, to meet the data processing demands of experimental applications. One option is the FastIC, which was developed for reading out positive and negative input polarity sensors with intrinsic amplification. In this paper, the first results from reading out the gaseous Picosec MicroMegas detector with the FastIC are presented. In laboratory studies, a basic description of the data processing chain was performed using a function generator. The results from test beam measurements are used to characterise the timing performance and the charge processing of the combination of FastIC and Picosec MicroMegas, as well as to demonstrate the possibility of a multi-channel detector readout. Although the timing-at-threshold level of the FastIC introduces a time walk of around 1.5 ns, time resolutions of around 50 ps have been achieved.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Verification and validation of developed short-term forecasting models

Recent advancements in machine learning (ML) and artificial intelligence (AI) technologies provide an opportunity for leveraging data-driven algorithms to predict future nuclear power plant (NPP) operating conditions by using recorded plant process data. Successfully implementing these models can lead to cost-reducing, conditioned-based predictive maintenance through optimized maintenance schedules and a reduction of unnecessary maintenance activities. This report discusses the verification and validation of short-term forecasting processes (i.e., data cleaning, feature selection, model optimization, and forecasting) developed in previous reports. The verification and validation (V&V) process demonstrates the expected precision and accuracy when the ML model encounters new datasets from different systems. Shapley additive explanations were used as the primary means of feature selection across these different data set. Individual models were trained for each data set, then validated through a cross-validation procedure. In this report, two different ML models were tasked to predict variables from three different plant process data sets with varying prediction horizons. The results indicate that support vector regression (SVR) outperformed long short-term memory (LSTM) neural networks in regard to each data set and each prediction horizon in this study, but further tuning and optimization could improve long short-term memory results. However, each forecasting model showed reduced performance as the prediction horizon was extended from 1 hour to 1 day ahead. Research is ongoing to evaluate the optimal input variable space, which is based on a given set of process parameters, to further improve forecasting accuracy.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Solutions for Digital Video Transmission Technology Final Report CRADA No. TC02068.0

This Project aimed at development of software for seismic data processing based on the Geotool code developed by the American company Multimax., Inc. The Geotool code was written in early 90-es for the UNIX platform. Under Project# 2821, functions of the old Geotool code were transferred into a commercial version for the Microsoft XP and Vista platform with addition of new capabilities on visualization and data processing. The developed new version of the Geotool+ was implemented using the up-to-date tool Microsoft Visual Studio 2005 and uses capabilities of the .NET platform. C++ was selected as the main programming language for the Geotool+. The two-year Project was extended by six months and funding levels increased from 600,000 to $670,000. All tasks were successfully completed and all deliverables were met for the project even though both the industrial partner and LLNL principal investigator left the project before its final report.

58 GEOSCIENCES↗

ATHENA, ARTEMIS, HEPHAESTUS: data analysis for X-ray absorption spectropscopy using IFEFFIT

A software package for the analysis of X-ray absorption spectroscopy (XAS) data is presented. This package is based on the IFEFFIT library of numerical and XAS algorithms and is written in the Perl programming language using the Perl/Tk graphics toolkit. The programs described here are: (i) ATHENA, a program for XAS data processing, (ii) ARTEMIS, a program for EXAFS data analysis using theoretical standards from FEFF and (iii) HEPHAESTUS, a collection of beamline utilities based on tables of atomic absorption data. These programs enable high-quality data analysis that is accessible to novices while still powerful enough to meet the demands of an expert practitioner. The programs run on all major computer platforms and are freely available under the terms of a free software license.

43 PARTICLE ACCELERATORS↗

Mixed core processor unit

A method and apparatus for processing data. The data is sent to a processor unit comprising a group of neural cores, a group of digital processing cores, and a routing network connecting the group of digital processing cores. The data is processed in the processor unit to generate a result.

97 MATHEMATICS AND COMPUTING↗

Using the ATLAS experiment software on heterogeneous resources

With the large dataset expected from 2030 onwards by the HL-LHC at CERN, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM CPU architecture is becoming a competitive and energy efficient alternative. Accelerators like GPUs are available in any recent HPC. In the past years ATLAS has successfully ported its full data processing and simulation software framework Athena to ARM and has invested significant effort in porting parts of the reconstruction and simulation algorithms to GPUs. We report on the successful usage of the ATLAS experiment offline and online software framework Athena on ARM and GPUs through the PanDA workflow management system at various WLCG sites. Furthermore we report on performance optimizations of the builds for ARM CPUs and the GPU integration efforts. We will discuss performance comparisons of different ARM and x86_64 architectures on WLCG resources and Cloud compute providers like GCP and AWS using ATLAS productions workflows as used in the Hep-Score23 benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Wave Detection and Tracking Within a Rotating Detonation Engine Through Object Detection

As the operational time window of experimental rotating detonation engines (RDEs) is expanded and the technology matures toward integration within gas turbines, monitoring techniques must evolve to offer computationally efficient and highly time-resolved diagnostics. In this study, computer vision object detection methodology that seeks to reduce data processing time and calculate wave velocity within drastically reduced time intervals as compared to traditional high-frame-rate RDE images analysis techniques is proposed. The adapted you-only-look-once object detection network is trained to detect individual detonation waves within single down-axis RDE images. The wave location and rotational direction detected within a frame are tracked through a series of high-speed images to calculate the frame-to-frame wave velocity with the time-step resolution of $\mathrm{20 μs}$ across a series of frames. The analysis of the annotation box size and image linearization effects is presented, demonstrating the lowest frame-to-frame velocity total uncertainty of $\mathrm{±3.8\%}$ and the highest classification speed of 9.5 frames per second using linearized images. Linearized images “unwrap” the RDE annulus pixel region to a reduced image size. Here, this new method offers great reductions in data processing times and unsteady detonation behavior insight at intervals more comparable to the timescales of detonation wave interactions via the application of machine learning to experimental RDE data.

33 ADVANCED PROPULSION SYSTEMS↗

KS4A-Omics1.0_FspDS682

Soil fungi facilitate the translocation of inorganic nutrients from soil minerals to other microorganisms and plants. This ability is particularly advantageous in impoverished soils, because fungal mycelial networks can bridge otherwise spatially disconnected and inaccessible nutrient hotspots. However, the molecular mechanisms underlying fungal mineral weathering and transport through soil remains poorly understood. Here, we addressed this knowledge gap by directly visualizing nutrient acquisition and transport through fungal hyphae in a mineral doped soil micromodel using a multimodal imaging approach. Here, we observed how a representative of common saprotrophic soil fungi, Fusarium sp. DS 682, exhibited a mechanosensory response (thigmotropism) around obstacles and through pore spaces (~12 μm) in the presence of minerals.This study establishes the significance of fungal biology and nutrient translocation mechanisms in maintaining fungal growth under water and nutrient limitations in a soil-like microenvironment, using a high-throughput multi-omic analysis approach. Data package KS4A-Omics.1.0_FspDS682 (Publication: Fungal Mineral Weathering Mechanisms Revealed Through Direct Molecular Visualization) contents reported here are the first version (1.0) and contain pre- and post-processed data using high throughput data capture technologies for multi-omic analysis and integration, this data package contains raw and post-processed experimental data for X-Ray Absorption Near Edge Structure Spectroscopy (XANES/XRF), Optical Microscopy, Proteomics, Scanning Electron Microscope (SEM), Time-of-Flight Secondary Ion Mass Spectroscopy (ToF-SIMS), X-Ray Diffraction Spectroscopy (XRD) files, and X- Ray Photoelectron Spectroscopy (XPS) using EMSL capabilities. This data package DOI contains a comprehensive collection of high-throughput multi-omics data and process metadata catalog. Support files include additional data download contents “Read Me” with dataset descriptor information and data source method application ontologies (see data dictionary section). Reported data download content is structured for compliance with reported guidelines provided by community standard initiatives and publisher stakeholder policies supporting FAIR data principles.

47 OTHER INSTRUMENTATION↗