Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Precision Polishing of Spheres Via In-Situ Process Monitoring and Machine-Learning-Based Optimization

In inertial confinement fusion (ICF) experiments seeking output gains of unity and beyond, the quality of the ablator capsule is paramount for minimizing hydrodynamic mix that quenches the central hot spot. Defects in the form of foreign particles or missing mass on the surface and within the wall of the capsule are primary offenders. High density carbon capsules made for ICF experiments on the National Ignition Facility (NIF) are precision polished to achieve the surface smoothness in the order of a few nm as well as to minimize isolated defects in the form of pits. Given the critical role of this process, we are developing smart manufacturing techniques with goal of elevating the efficiency of this process. Our approach is to use MEMS-based sensors to capture the fine vibrational signals generated during the polishing process and combine it with synchronized visual feedback as needed. Beyond using these sensors for process monitoring, we use specific deep learning methods to analyze the data and extract correlations with both the process parameters and the final performance of the polishing run. Here, we describe the multiple fronts that we have explored in this regard and the results we have gotten so far. This approach promises to have the potential to ultimately provide real-time feedback that can be used for ensuring the progress of the run as well as a means for faster optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2024

The Information Length Concept Applied to Plasma Turbulence

A methodology to study statistical properties of anomalous transport in fusion plasma is investigated. Three time traces generated by the full-f gyrokinetic code GKNET are analyzed for this purpose. The time traces consist of heat flux as a function of the radial position, which is studied in a novel manner using statistical methods. The simulation data exhibit transport processes with both medium and long correlation length along the radius. A typical example of a phenomenon with long correlation length is avalanches. In order to investigate the evolution of the turbulent state, two basic configurations are studied, one flux-driven and one gradient-driven with decaying turbulence. The information length concept in tandem with Boltzmann–Gibbs and Tsallis entropy is used in the investigation. It is found that the dynamical states in both flux-driven and gradient-driven cases are surprisingly similar, but the Tsallis entropy reveals differences between them. This indicates that the types of probability distribution function are nevertheless quite different since the higher moments are significantly different.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

A three-point velocity estimation method for two-dimensional coarse-grained imaging data

Time delay and velocity estimation methods have been widely studied subjects in the context of signal processing, with applications in many different fields of physics. The velocity of waves or coherent fluctuation structures is commonly estimated as the distance between two measurement points divided by the time lag that maximizes the cross correlation function between the measured signals, but this is demonstrated to result in erroneous estimates for two spatial dimensions. We present an improved method to accurately estimate both components of the velocity vector, relying on three non-aligned measurement points. We introduce a stochastic process describing the fluctuations as a superposition of uncorrelated pulses moving in two dimensions. Using this model, we show that the three-point velocity estimation method, using time delays calculated through cross correlations, yields the exact velocity components when all pulses have the same velocity. The two- and three-point methods are tested on synthetic data generated from realizations of such processes for which the underlying velocity components are known. The results reveal the superiority of the three-point technique. Finally, we demonstrate the applicability of the velocity estimation on gas puff imaging data of strongly intermittent plasma fluctuations due to the radial motion of coherent, blob-like structures at the boundary of the Alcator C-Mod tokamak.

Materials Science↗
DOE OSTI2025

Online learning of quadratic manifolds from streaming data for nonlinear dimensionality reduction and nonlinear model reduction

Here, this work introduces an online greedy method for constructing quadratic manifolds from streaming data, designed to enable in situ analysis of numerical simulation data on the Petabyte scale. Unlike traditional batch methods, which require all data to be available upfront and take multiple passes over the data, the proposed online greedy method incrementally updates quadratic manifolds in one pass as data points are received, eliminating the need for expensive disk input/output operations as well as storing and loading data points once they have been processed. A range of numerical examples demonstrate that the online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗
DOE OSTI2026

Observational Data for Next-Generation Climate Model Evaluation: Requirements, Considerations, and Best Practices

Climate model simulations are an important source of information about our planet’s climate system and also enable informed decision-making under different future scenarios. As a new archive of results from the next generation of climate models is anticipated to become available with the Coupled Model Intercomparison Project phase 7 (CMIP7), the need to develop efficient and robust methods to evaluate models is paramount. Observations are an integral part of model evaluation, providing a means to quantify and understand the degree to which climate models can faithfully reproduce Earth system processes. Such analysis is critical for constraining climate projections, identifying areas of focus for model development, and assisting analysts in deciphering the utility of models for specific applications. Observations of Earth system come from a diversity of sources, span different space–time domains, and are produced by different communities, and each dataset features different data structures and formats, metadata standards, and its own unique uncertainties. Uncertainties in an observational dataset may stem from gaps in temporal and spatial coverage, instrumentation errors, or assumptions in retrieval and processing methods. How then does one ensure that observational data are ready for use and utilized in the most appropriate way for robust, rapid, and routine climate model evaluation? The CMIP7 Model Benchmarking Task Team with input from the broader climate modeling, model evaluation, and observational data communities present a vision and considerations for best practices toward the optimal and appropriate use of observational data to support next-generation climate model evaluation.

Climate models↗
DOE OSTI2025

Cardinal: Seismic and Geoacoustic Array Processing

Data collected via seismic and infrasound array deployments are leveraged in the geosciences to detect and characterize a myriad of natural and anthropogenic sources. These deployments consist of numerous sensors placed in a predetermined configuration to amplify signal strength and improve the efficacy of array processing techniques used to measure signal directionality and waveform coherence. High‐fidelity feature extraction is often predicated on interstation distance as well as the frequency content and wavelength of an incident signal. Numerous array processing softwares analyze data in sequential frequency bands to obtain a more detailed characterization of a signal. However, current algorithms are limited in their ability to determine optimal array configuration for each band. We introduce an open‐source Python code, called Cardinal, to process seismic and infrasound array data in discretized time–frequency space with the option of applying an adaptive array design to determine optimal subarray configuration for each frequency band. To reduce computational time, the array processing step can be run in parallel using multithreading. Furthermore, the software has the capability to aggregate array processing results from different time–frequency pixels to produce separate sets of detections, or families, with added utility via the application of an adaptive semblance threshold, which aids in isolating signals‐of‐interest from coherent background noise. Upon appropriate configuration, Cardinal exhibits the potential to combine distinct seismic and infrasound phases into separate families.

Adaptive Array↗
DOE OSTI2025

Reuniting crystallography with real space: Ab initio structure elucidation with 4D-STEM

Structure elucidation via single-crystal methods has historically lacked experimental access to real-space information, instead relying exclusively on diffraction-space measurements of Bragg reflections. Here we exploit the dual-space imaging power of 4D scanning transmission electron microscopy to meaningfully integrate real-space information into the crystallographic workflow. We show that virtual apertures assembled by segmentation of high-angle annular dark-field images enable i) pixel-by-pixel separation of coherent Bragg signal from clusters of closely spaced nanocrystals and ii) selective extraction of integrated intensities from thinner subregions of individual specimens, facilitating retroactive tuning of multiple scattering artifacts. This strategy empowers us to simply pick and choose whichever nanoscale regions of interest generate the highest-quality diffraction patterns, allowing us to solve several independent structures of the metal-organic framework UiO-66 from specimens whose agglomerated morphology proved intractable for conventional microcrystal electron diffraction. Our method is compatible with both rotational and serial approaches to data processing, ultimately divulging the first scanning nanobeam electron diffraction structures determined by direct methods at subangstrom resolution.

Saha, Ambarneil↗
DOE OSTI2025

Improved heavy-ion PID using scintillation light detector with neural network analysis: a Monte Carlo simulation study

The photon collection efficiency of gaseous scintillator detectors varies according to the position of the impinging charged particles in the medium that generates scintillation light. Thus, when impinging particles are distributed over a large area, the intrinsic photon-number resolution of the system is affected by a large variation. This work presents and discusses a method for adjusting the total number of detected photons to account for variation in the photon collection efficiency as a function of the position of the light source within the scintillating medium. The method was developed and validated by processing data from systematic simulation studies based on GEANT4 that model the response of the Energy Loss Optical Scintillation System (ELOSS) detector. The position of the charged particle is calculated using a deep neural network algorithm. This is accomplished by analyzing the distribution of scintillation light recorded by the array of photosensors. The estimated particle position is then used to calculate the correction factor and adjust the amount of captured light to account for variations in the photon collection efficiency. The neural network algorithm provides excellent tracking capabilities, achieving sub-millimeter position resolution and an angular resolution of 12 mrad, approaching the performance of traditional tracking detectors (e.g., drift chambers). The present method can be generalized to any optical scintillation system where the photon collection efficiency depends on the position of the impinging particle.

Heavy-ion detectors↗
DOE OSTI2024

Human Liver Epithelial Cells (HuH7) Response to HCoV-229E Infection Epigenomics (ATAC-Seq) (ACS-DP4)

The purpose of this experiment was to evaluate how wild-type Human coronavirus strain 229E (HCoV-299E) infection alters chromatin accessibility in infected cells. Sample data was obtained from mock-infected cells, UV-inactivated virus treated cells, and replication competent HCoV-229E infected immortalized human liver cells (HuH7) at 24 hours post infection. Samples were processed using ATAC-seq methods for reported bar coded libraries. Sample data was acquired using an Illumina Hi-Seq 2500 sequencer system and further processed for ATAC-Seq expression analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Developing ML/AI Methods for High-Throughput Characterization of Multiple-Sensor Streams of Tokamak Dynamics for High-Speed Control (Final Report)

This project evaluated and developed new mathematical and algorithmic techniques capable of handling (in real-time) the growing amounts of data generated by modern fusion research. While existing numerical linear algebra (NLA) methods provide the backbone to classical data analysis and algorithms, these methods fundamentally do not port to distributed architectures nor do they allow low-latency data reduction for control. Motivated by the needs for modern fusion reactors, this project explored and implemented new numerical methods to characterize plasma dynamics, respond in real-time to discharge evolution, and to process massive-scale data accurately and rapidly more fully. This project links expertise in multiple-sensor diagnostics of tokamak plasma dynamics from Columbia University’s Plasma Physics Laboratory with expertise in massive-scale data reduction and extreme data control algorithms at Columbia University’s Data Science Institute. This interdisciplinary project (i) applied machine learning methods, (ii) implemented a properly-trained neural-network for very fast processing of high-speed plasma videography, and (ii) developed the applied mathematical methods, based on randomized-NLA (rNLA) routines, for data analysis, reduction, and real-time control. The Columbia University High Beta Tokamak-Extended Pulse (HBT-EP) facility provided data to test new algorithms and partnership with Columbia University's Data Sciences Institute evaluated the broader use of new algorithms for many challenging control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) v1

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) is a comprehensive data visualization and analysis application focused on working with COLTRIMS (COLd Target Recoil Ion Momentum Spectroscopy) data, which is used in atomic and molecular physics experiments. The application offers several powerful features: - Data uploading and processing capabilities for COLTRIMS files - Multiple visualization methods using UMAP (Uniform Manifold Approximation and Projection) for dimensionality reduction - Interactive selection of data points across multiple views - Feature engineering through various methods: - Manual feature selection from calculated physics parameters - Deep autoencoder for dimension reduction - Genetic programming for discovering meaningful features - Mutual information-based feature selection - Multiple clustering approaches (DBSCAN, KMeans, Agglomerative) - Quality metrics for evaluating clustering results - Export capabilities for selections and generated features

Daoud, Hazem [Lawrence Berkeley National Laborator↗
DOE OSTI2024

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM↗
DOE OSTI2024

Unsupervised multimodal fusion of in-process sensor data for advanced manufacturing process monitoring

Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments often generate vast amounts of complementary multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, fusing and interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable or impractical to obtain. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, overcoming limitations of traditional supervised machine learning methods in manufacturing contexts. Our proposed method demonstrates the ability to handle and learn encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. The unsupervised nature of our method makes it broadly applicable across various manufacturing domains, where large volumes of unlabeled sensor data are common. We evaluate the effectiveness of our approach through a series of experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to the field of smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations. The proposed method paves the way for more robust, data-driven decision-making in complex manufacturing processes.

Contrastive Learning↗
DOE OSTI2024

Out-of-Distribution Detection and Radiological Data Monitoring Using Statistical Process Control

Abstract Machine learning (ML) models often fail with data that deviates from their training distribution. This is a significant concern for ML-enabled devices as data drift may lead to unexpected performance. This work introduces a new framework for out of distribution (OOD) detection and data drift monitoring that combines ML and geometric methods with statistical process control (SPC). We investigated different design choices, including methods for extracting feature representations and drift quantification for OOD detection in individual images and as an approach for input data monitoring. We evaluated the framework for both identifying OOD images and demonstrating the ability to detect shifts in data streams over time. We demonstrated a proof-of-concept via the following tasks: 1) differentiating axial vs. non-axial CT images, 2) differentiating CXR vs. other radiographic imaging modalities, and 3) differentiating adult CXR vs. pediatric CXR. For the identification of individual OOD images, our framework achieved high sensitivity in detecting OOD inputs: 0.980 in CT, 0.984 in CXR, and 0.854 in pediatric CXR. Our framework is also adept at monitoring data streams and identifying the time a drift occurred. In our simulations tracking drift over time, it effectively detected a shift from CXR to non-CXR instantly, a transition from axial to non-axial CT within few days, and a drift from adult to pediatric CXRs within a day—all while maintaining a low false positive rate. Through additional experiments, we demonstrate the framework is modality-agnostic and independent from the underlying model structure, making it highly customizable for specific applications and broadly applicable across different imaging modalities and deployed ML models.

Zamzmi, Ghada↗
DOE OSTI2025

Leveraging data mining, active learning, and domain adaptation for efficient discovery of advanced oxygen evolution electrocatalysts

Developing advanced catalysts for acidic oxygen evolution reaction (OER) is crucial for sustainable hydrogen production. This study presents a multistage machine learning (ML) approach to streamline the discovery and optimization of complex multimetallic catalysts. Our method integrates data mining, active learning, and domain adaptation throughout the materials discovery process. Unlike traditional trial-and-error methods, this approach systematically narrows the exploration space using domain knowledge with minimized reliance on subjective intuition. Then, the active learning module efficiently refines element composition and synthesis conditions through iterative experimental feedback. The process culminated in the discovery of a promising Ru-Mn-Ca-Pr oxide catalyst. Our workflow also enhances theoretical simulations with domain adaptation strategy, providing deeper mechanistic insights aligned with experimental findings. By leveraging diverse data sources and multiple ML strategies, we demonstrate an efficient pathway for electrocatalyst discovery and optimization. This comprehensive, data-driven approach represents a paradigm shift and potentially benchmark in electrocatalysts research.

Science & Technology - Other Topics↗
DOE OSTI2024

A global urban heat island intensity dataset: Generation, comparison, and analysis

The urban heat island (UHI) effect, a phenomenon of local warming over urban areas, is the most well-known impact of urbanization on climate. Globally consistent estimates of the UHI intensity (UHII) are crucial for examining this phenomenon across time and space. However, publicly available UHII datasets are limited and have several constraints: (1) they are for clear-sky surface UHII, not all-sky surface UHII and canopy (air temperature) UHII; (2) the estimation methods often neglect anthropogenic disturbance, introducing uncertainties in the estimated UHII. To address these issues, this study proposes a new dynamic equal-area (DEA) method that can minimize the influence of various confounding factors on UHII estimates through a dynamic cyclic process. Utilizing the DEA method and leveraging various gridded temperature data, we develop a global-scale (>10,000 cities), long-term (over 20 years by month), and multi-faceted (clear-sky surface, all-sky surface, and canopy) UHII dataset. Further, based on these estimates, we provide a comprehensive analysis of the UHII and its trends in global cities. The UHII is found to be greater than zero in >80% of cities, with global annual average magnitudes around 1.0 °C (day) and 0.8 °C (night) for surface UHII, and close to 0.5 °C for canopy UHII. Furthermore, an interannual upward trend in UHII is observed in >60% of cities, with global annual average trends exceeding 0.1 °C/decade (day) and over 0.06 °C/decade (night) for surface UHII, and slightly surpassing 0.03 °C/decade for canopy UHII. Notably, there exists a positive correlation between the magnitude and trend of UHII, suggesting that cities with stronger UHII tend to experience faster growth in UHII. Additionally, discrepancies in UHII are found between different temperature data, stemming not only from distinctions in data types (surface or air temperature) but also from differences in data acquisition times (Terra or Aqua), weather conditions (clear-sky or all-sky), and processing methodologies (with or without gap filling). Overall, our proposed method, dataset, and analysis results have the potential to provide valuable insights for future urban climate studies. The UHII dataset is publicly available at https://doi.org/10.6084/m9.figshare.24821538.

54 ENVIRONMENTAL SCIENCES↗