Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Noisy data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Molecular Hydrogen in the Quiescent Disk of SW UMa

The FUSE observation has been reduced and a paper has been submitted to ApJ. The analysis has been slow because of the very noisy quality of the data, but we have derived line profile information for O VI and limits to the continuum brightness which place an interesting limit on the white dwarf temperature. The primary results are that a narrow O VI emission component seems to arise from the accretion flow onto the white dwarf itself, in agreement with cooling flow models for the X-ray spectra of low accretion rate dwarf novae. The broad component of the O VI lines is weaker than the observed C IV emission, suggesting that the UV line emission from the disk comes from photoionized plasma. A secondary result is that there is no H-2 fluorescent emission. The upper limits indicate that if molecular gas is present in the disk, it is shielded from Ly alpha photons by a layer of atomic hydrogen on the disk surface. We also derive an upper limit to the continuum level is below that observed by IUE. The limits are compatible with the lower end of the WD temperature range derived from IUE measurements, and they appear to agree with unpublished analysis of HST spectra. The grant has provided partial support for a data aide (Matt Povich) and a postdoc (Alex Lobel). It purchased a computer for M. Menou.

Raymond, John C.↗
NASA NTRS1994

Inverter Matrix for the Clementine Mission

An inverter matrix test circuit was designed for the Clementine space mission and is built into the RRELAX (Radiation and Reliability Assurance Experiment). The objective is to develop a circuit that will allow the evaluation of the CMOS FETs using a lean data set in the noisy spacecraft environment.

CMOS FETs spacecraft environment RRELAX orbit moon↗
DOE OSTI2022

Toward Robust Autotuning of Noisy Quantum dot Devices

The current autotuning approaches for quantum dot (QD) devices, while showing some success, lack an assessment of data reliability. This leads to unexpected failures when noisy or otherwise low-quality data is processed by an autonomous system. In this work, we propose a framework for robust autotuning of QD devices that combines a machine learning (ML) state classifier with a data quality control module. The data quality control module acts as a “gatekeeper” system, ensuring that only reliable data are processed by the state classifier. Lower data quality results in either device recalibration or termination. To train both ML systems, we enhance the QD simulation by incorporating synthetic noise typical of QD experiments. We confirm that the inclusion of synthetic noise in the training of the state classifier significantly improves the performance, resulting in an accuracy of 95.0(9)% when tested on experimental data. We then validate the functionality of the data quality control module by showing that the state classifier performance deteriorates with decreasing data quality, as expected. Our results establish a robust and flexible ML framework for autonomous tuning of noisy QD devices.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

On the dual advantage of placing observations through forward sensitivity analysis

The four-dimensional variational data assimilation methodology for assimilating noisy observations into a deterministic model has been the workhorse in forecasting centres for over three decades. While this method provides a computationally efficient framework for dynamic data assimilation, it is largely silent on the important question concerning the minimum number and placement of observations. To answer this question, we demonstrate the dual advantage of placing the observations where the square of the sensitivity of the model solution with respect to the unknown control variables, called forward sensitivities, attains its maximum. By following this approach, we can force the observability Gramian to be of full rank, which in turn guarantees efficient recovery of the optimal values of the control variables, which is the first of the two advantages of this strategy. We further show that the proposed strategy of placing observations has another inherent optimality: the square of the sensitivity of the optimal estimates of the control with respect to the observations (used to obtain these estimates) attains its minimum value, a second advantage that is a direct consequence of the above strategy for placing observations. Our analytical framework and numerical experiments on linear and nonlinear systems confirm the effectiveness of our proposed strategy.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1988

Radiometric performance of AVIRIS: Assessment for an arid region geologic target

Data from several AVIRIS flight lines were examined to assess instrument stability and response. Both scene and in-flight calibration data were analyzed statistically. The data clearly indicates that, although the instrument output was noisy and unstable at the time of the data acquisition, valuable spectral signatures can still be extracted and analyzed. Some first order calibration corrections can be performed by forcing internal consistency within the data. AVIRIS data are delivered in band-interleaved-by-line format, but high efficiency routines were developed which access the data as either image or spectral planes and enable effective statistical and visual examination of both AVIRIS scenes and ancillary files. Two methods were used to extract spectral information from segment 4 of the Kelso Dunes flight. Both successfully identified at least three distinct spectral signatures, but neither has positively identified a specific material.

Kieffer, Hugh H.↗
DOE OSTI2022

Weakly Supervised Event Classification Using Imperfect Real-world PMU Data with Scarce Labels

This paper studies event classification using imperfect real-world phasor measurement unit (PMU) data with scarce event types (labels). By investigating the real-world PMU data, it is observed that most real-world PMU data's event type is unknown, which makes it challenging to directly use such dataset to build event classifiers as existing classification techniques require high-quality training data with known event type (i.e., label). To address this challenge, a weakly supervised learning based event classification approach is developed, which can use noisy and low-quality PMU data for the training. First, data quality issues are fixed using data preprocessing techniques and then event features are constructed from the PMU data. Using these features, a series of labeling functions are learnt to generate initial estimates of the labels of large amounts of unlabeled PMU data. As the labeling functions are learnt using the same data with scarce labels, the label estimates from the labeling functions can be correlated, noisy, and bias. To enhance these initial estimates, a generative model is developed to characterize the dependencies among the estimated labels, based on which better labels are obtained for training event classifiers. Numerical experiments using the real-world dataset from the Western Interconnection of the U.S. power transmission grid show that the proposed weakly supervised event classifier trained using the dataset with only 5% labeled data can achieve 78.4% classification accuracy.

Liu, Yunchuan↗
NASA NTRS1972

Data smoothing and error detection based on linear interpolation

A method is presented, based on linear interpolation, for detecting and correcting bad data points in a set of data without contaminating the good data points. The method used is not concerned with the small random errors usually attributed to a noisy system. It assumes that the data points which are in error are relatively isolated from each other and that the number of such points is small compared to the total number of data points.

Guerra, V. M.↗
NASA NTRS1978

Processing techniques development, volume 3

The author has identified the following significant results. Analysis of the geometric characteristics of the aircraft synthetic aperture radar (SAR) relative to LANDSAT indicated that relatively low order polynominals would model the distortions to subpixel accuracy to bring SAR into registration for good quality imagery. Also the area analyzed was small, about 10 miles square, so this is an additional constraint. For the Air Force/ERIM data, none of the tested methods could achieve subpixel accuracy. Reasons for this is unknown; however, the noisy (high scintillation) nature of the data and attendent unrecognizability of features contribute to this error. It is concluded that the quadratic model would adequately provide distortion modeling for small areas, i.e., 10 to 20 miles square.

Landgrebe, D. A.↗
DOE OSTI2025

Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet

Modern data-driven surrogate models for weather forecasting provide accurate short-term predictions but inaccurate and nonphysical long-term forecasts. This paper investigates online weather prediction using machine learning surrogates supplemented with partial and noisy observations. We empirically demonstrate and theoretically justify that, despite the long-time instability of the surrogates and the sparsity of the observations, filtering estimates can remain accurate in the long-time horizon. As a case study, we integrate FourCastNet, a weather surrogate model, within a variational data assimilation framework using partial, noisy ERA5 data. Our results show that filtering estimates remain accurate over a year-long assimilation window and provide effective initial conditions for forecasting tasks, including extreme event prediction.

Adrian, Melissa [Univ. of Chicago, IL (United Stat↗
DOE OSTI2026

Grid Topology Discovery Algorithm Evaluation of Suitability for Utility Deployment (CRADA 606 Final Report)

This work presents the results of a field-informed demonstration aimed at evaluating the practical suitability of a topology discovery algorithm for utility environments. We demonstrated an algorithm that uses a graph-theory-informed state estimation approach for model selection. In collaboration with Survalent and Peninsula Light Co., the algorithm was applied to real feeder models and field measurements from supervisory control and data acquisition (SCADA) and advanced metering infrastructure (AMI) systems to identify the operational topology of a power distribution system. The demonstration assessed the algorithm’s performance under realistic data conditions, including sparse and noisy measurements, and examined its ability to identify the most likely network configurations. The results confirmed that the approach can effectively narrow down feasible topologies, providing operators with improved situational awareness of network status. Key lessons learned emphasize the need for systematic data validation and strategic sensor placement to enhance observability. These insights inform future deployment strategies and guide refinements for broader adoption in utility operations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Limitations of reanalysis data for wind power applications

Wind energy resource estimates commonly depend on simulated wind speed profiles generated by reanalysis or weather models due to the lack of long time series measurements with sufficient coverage at relevant heights (roughly 90 m above ground). However, modeled data, including reanalyses, can be noisy and display a wide range of biases and errors, variously attributed to terrain effects, poor coverage of assimilated inputs, and model resolution. Wind generation records, if available at high temporal and geographical resolution, can provide a proxy for wind measurements and allow for evaluation of reanalyses and weather model wind time series. We use a 7-year-long data set of hourly, plant-level generation records from over 100 wind plants across Texas to evaluate two commonly used reanalysis data sets (MERRA2 and ERA5). Additionally, we use 1-year of records (2019) to evaluate an operational, high-resolution regional weather modeling product (HRRR v3). We find that across the region, and across all modeling products, the modeled representation of wind generation (i.e., wind speeds at hub heights passed through a power curve) has relatively small mean errors when aggregated daily, but that accuracy and hourly correlation have a strong diurnal sensitivity. Accuracy and correlation systematically decline through the evening and markedly improve after sunrise. These diurnal patterns persist even in the highest resolution model tested (HRRR v3). We hypothesize the nighttime decline in accuracy is mostly due to poorly represented boundary layer conditions, perhaps related to model representation of stability, while other uncertainties (such as wake effects) play a secondary role.

17 WIND ENERGY↗
DOE OSTI2026

FunDiff: diffusion models over function spaces for physics-informed generative modeling

Recent advances in generative modeling-particularly diffusion models and flow matching-have been widely used for synthesizing discrete data such as images and videos. However, adapting these models to physical applications remains challenging, as the quantities of interest are continuous functions governed by complex physical laws. To address this, we introduce FunDiff, an efficient and robust framework for generative modeling in function spaces. FunDiff combines a latent diffusion process with a function autoencoder architecture to handle input functions with varying discretizations, generates continuous functions that can be evaluated at arbitrary locations, and seamlessly incorporate physical priors. These priors are enforced through architectural constraints or physics-informed loss functions, ensuring that generated samples satisfy fundamental physical laws. We theoretically establish minimax optimality guarantees for density estimation in function spaces, demonstrating that diffusion-based estimators achieve optimal convergence rates under suitable regularity conditions. We further demonstrate the practical effectiveness of FunDiff across diverse applications in fluid dynamics and solid mechanics. Empirical results indicate that our method can generate physically consistent samples with high fidelity to the target distribution, and exhibit robustness to noisy and low-resolution data.

Wang, Sifan [Yale University, New Haven, CT (Unite↗
NASA NTRS1988

Compression of high spectral resolution imagery

The Earth Orbiting Satellite (EOS), scheduled for launch in the mid-1990s, will include a next-generation multispectral imaging system (HIRIS) having unprecedented spatial and spectral resolution. Its high resolution, however, comes at the cost of a raw data rate which exceeds the communication channel capacity assigned to the entire EOS mission. This paper explores noisy compression algorithms which may compress multispectral data by up to 30:1 or more. Algorithm performance is measured using both traditional and mission-oriented criteria (e.g., feature classification consistency). It is shown that vector quantization, merged with suitable preprocessing techniques, is the most viable candidate.

Baker, Richard L.↗
DOE OSTI2024

An Envelope Time Synchronous Averaging for Wind Turbine Gearbox Fault Diagnosis

Vibration-based condition monitoring techniques are widely used for diagnosing faults in rotating machines. These techniques are implemented in the time domain, the frequency domain, or both. However, the composite and noisy nature of the raw data collected requires a preprocessing stage such as filtering and decomposition using in-depth processing techniques. Moreover, these methods require good frequency resolution and involve examining a broad frequency range to discern both healthy and faulty cases. In this work, we introduce a simple and fast diagnostic scheme for wind turbine gear teeth wear based on time domain analysis. The proposed method is based on the local minima interpolation of a filtered version of the vibration signal following time synchronous averaging (TSA) technique. Given tachometer signal, the TSA of the vibration data is performed using MTALAB software. Then, local minima of the filtered signal are interpolated using the Piecewise Cubic Hermite Interpolating Polynomial (PCHIP) function. The variance of the interpolated curve built a gear fault index. The derived fault index resulting of the proposed technique allows a substantial distinction between the healthy and faulty cases. Its efficiency is validated using 10 real-world datasets of vibration stemmed from a wind turbine planetary gearbox. The proposed method boasts a low computation time and ease of interpretation, specifically beneficial for gearbox fault diagnosis purposes.

fault diagnosis↗
DOE OSTI2023

High-Resolution Near-Surface Imaging at the Basin Scale Using Dark Fiber and Distributed Acoustic Sensing: Toward Site Effect Estimation in Urban Environments

Near-surface seismic structure, particularly the shear wave velocity (V s ), can strongly affect local site response, and should be accurately estimated for ground motion prediction during seismic hazard assessment. The Imperial Valley (California), occupying the southern end of the Salton Trough, is a seismically active basin with thick surficial lacustrine sedimentary deposits. In this study, we utilize ambient noise records and local earthquake events for high-resolution near-surface characterization and site effect estimation with an unlit fiber-optic telecommunication infrastructure (dark fiber) in Imperial Valley by using the distributed acoustic sensing (DAS) technique. We apply ambient noise interferometry to retrieve coherent surface waves from DAS records, and evaluate performances of three different surface wave methods on DAS ambient noise dispersion imaging. We develop a quality control workflow to improve the dispersion measurement of noisy portions of the DAS data set by using a data selection strategy. Using the joint inversion of both the fundamental mode and higher overtones of Rayleigh waves, a high resolution two-dimensional (2D) V s structure down to 70 m depth is obtained. We successfully achieve an improved V s 30 (the time-averaged shear-wave velocity in the top 30 m) model with higher spatial-resolution and reliability compared to the existing community model for the area. We also explore the potential for utilizing DAS earthquake events for site amplification estimation. The preliminary results reveal a clear anti-correlation between the approximated site response and the V s 30 profile. In conclusion, our results indicate the potential utility of DAS deployed on dark fiber for near-surface characterization in appropriate contexts.

58 GEOSCIENCES↗
DOE OSTI2020

A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy datasets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored dataset. Unfortunately, and this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as Automatic Relevance Determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial dataset, and estimates which K produces stable groups of latent features that reconstruct the initial dataset well. We then train a Multi-Layter Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solution, obtained from NMFk. In order to train the MLP classifier, a training set of 58,660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark datasets, the swimmer and MIT face data, NMFk/MLP correctly recovers the established number of hidden features. Finally, we compare the accuracy of our method to the ARD, AIC and Stability-based methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Benchmarking noise extrapolation with the OpenPulse control framework

Distilling precise estimates from noisy intermediate scale quantum (NISQ) data has recently attracted considerable attention Kandala et al., Nature (London) 567, 491 (2019). In order to augment digital qubit metrics, such as gate fidelity, we discuss analog error mitigability, i.e., the ability to accurately distill precise observable estimates, as a hybrid quantum classical computing benchmarking task. Specifically, using Rabi oscillations as a test program, we characterize single qubit error rates on IBM's Poughkeepsie superconducting quantum hardware, incorporate control-mediated noise dependence into a generalized rescaling protocol, and analyze how noise characteristics influence Richardson extrapolation-based error mitigation. Finally, our results identify regions in the space of Hamiltonian control fields and circuit depth which are most amenable to reliable noise extrapolation, as well as shed light on how low-level hardware characterization can be used as a predictive tool for uncertainty quantification in error-mitigated NISQ computations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1970

Study of sequential decoding

Decoding algorithms for data reduction and transmission through noisy space channels using sequential and hybrid computers

Jelinek, F.↗