Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Noisy data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS1976

On the computational cost of approximating and recognizing noise-perturbed straight lines and quadratic arcs in the plane

Recognition of underlying straight lines and quadratic arcs in line drawings, and approximation of very noisy data by such line/curve segments, is addressed as a subproblem of the more general problem of optimum recognition of complicated line/curve drawings. Some specific algorithms are presented with extensions and interpretations for more complicated applications, and a data generation model is developed for the problem. Data are generated as a perturbation of a single underlying straight line or an elliptic or hyperbolic arc. Recursive estimation techniques, minimization of central processing unit time, decision making with controlled error probabilities, and modeling and recognition of pictures consisting of noisy curves are dealt with. Applications envisaged include: picture data compression, contour line representation in maps, intelligent data searches, and ballistic missile decoy tracking.

Cooper, D. B.↗
DOE OSTI2025

Surrogate-driven design optimization with uncertainty constraints in Monte Carlo simulations

In multi-objective design tasks, the computational cost increases rapidly when high-fidelity simulations are used to evaluate objective functions. Surrogate models help mitigate this cost by approximating the simulation output, simplifying the design process. However, under high uncertainty, surrogate models trained on noisy data can produce inaccurate predictions, as their performance depends heavily on the quality of training data. This study investigates the impact of data uncertainty on two multi-objective design problems modelled using Monte Carlo transport simulations: a neutron moderator and an ion-to-neutron converter. For each, a grid search was performed using five different tally uncertainty levels to generate training data for neural network surrogate models. These models were then optimized using NSGA-III. The recovered Pareto-fronts were analyzed across uncertainty levels: in the moderator problem, normalized hypervolume dropped from 0.886 at 1.0% uncertainty to 0.748 at 10% uncertainty, while in the converter problem it remained near 0.50 for all cases. Average simulation times were also compared to evaluate the trade-off between accuracy and computational cost. Results show that the influence of simulation uncertainty is strongly problem-dependent. In the neutron moderator case, higher uncertainties led to exaggerated objective sensitivities and distorted Pareto-fronts, reducing normalized hypervolume. In contrast, the ion-to-neutron converter task was less affected—low-fidelity simulations produced results similar to those from high-fidelity data. These findings suggest that a fixed-fidelity approach is not optimal. Surrogate models can recover the Pareto-front under noisy conditions, and multi-fidelity studies help identify suitable uncertainty levels for each problem to balance efficiency and accuracy.

07 ISOTOPE AND RADIATION SOURCES↗
NASA NTRS1993

Classification of high dimensional multispectral image data

A method for classifying high dimensional remote sensing data is described. The technique uses a radiometric adjustment to allow a human operator to identify and label training pixels by visually comparing the remotely sensed spectra to laboratory reflectance spectra. Training pixels for material without obvious spectral features are identified by traditional means. Features which are effective for discriminating between the classes are then derived from the original radiance data and used to classify the scene. This technique is applied to Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) data taken over Cuprite, Nevada in 1992, and the results are compared to an existing geologic map. This technique performed well even with noisy data and the fact that some of the materials in the scene lack absorption features. No adjustment for the atmosphere or other scene variables was made to the data classified. While the experimental results compare favorably with an existing geologic map, the primary purpose of this research was to demonstrate the classification method, as compared to the geology of the Cuprite scene.

Hoffbeck, Joseph P.↗
NASA NTRS1986

Sequential estimation and satellite data assimilation in meteorology and oceanography

The central theme of this review article is the role that dynamics plays in estimating the state of the atmosphere and of the ocean from incomplete and noisy data. Objective analysis and inverse methods represent an attempt at relying mostly on the data and minimizing the role of dynamics in the estimation. Four-dimensional data assimilation tries to balance properly the roles of dynamical and observational information. Sequential estimation is presented as the proper framework for understanding this balance, and the Kalman filter as the ideal, optimal procedure for data assimilation. The optimal filter computes forecast error covariances of a given atmospheric or oceanic model exactly, and hence data assimilation should be closely connected with predictability studies. This connection is described, and consequences drawn for currently active areas of the atmospheric and oceanic sciences, namely, mesoscale meteorology, medium and long-range forecasting, and upper-ocean dynamics.

Ghil, M.↗
DOE OSTI2021

Quantifying experimental edge plasma evolution via multidimensional adaptive Gaussian process regression

The edge density and temperature of tokamak plasmas are strongly correlated with energy and particle confinement and their quantification is fundamental to understanding edge dynamics. These quantities exhibit behaviours ranging from sharp plasma gradients and fast transient phenomena (e.g. transitions between low and high confinement regimes) to nominal stationary phases. Analysis of experimental edge measurements therefore require robust fitting techniques to capture potentially stiff spatiotemporal evolution. Additionally, fusion plasma diagnostics inevitably involve measurement errors and data analysis requires a statistical framework to accurately quantify uncertainties. This paper outlines a generalized multidimensional adaptive Gaussian process routine capable of automatically handling noisy data and spatiotemporal correlations. We focus on the edge-pedestal region in order to underline advancements in quantifying time-dependent plasma profiles including transport barrier formation on the Alcator C-Mod tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1999

Backus Effect on a Perpendicular Errors in Harmonic Models of Real vs. Synthetic Data

Measurements of geomagnetic scalar intensity on a thin spherical shell alone are not enough to separate internal from external source fields; moreover, such scalar data are not enough for accurate modeling of the vector field from internal sources because of unmodeled fields and small data errors. Spherical harmonic models of the geomagnetic potential fitted to scalar data alone therefore suffer from well-understood Backus effect and perpendicular errors. Curiously, errors in some models of simulated 'data' are very much less than those in models of real data. We analyze select Magsat vector and scalar measurements separately to illustrate Backus effect and perpendicular errors in models of real scalar data. By using a model to synthesize 'data' at the observation points, and by adding various types of 'noise', we illustrate such errors in models of synthetic 'data'. Perpendicular errors prove quite sensitive to the maximum degree in the spherical harmonic expansion of the potential field model fitted to the scalar data. Small errors in models of synthetic 'data' are found to be an artifact of matched truncation levels. For example, consider scalar synthetic 'data' computed from a degree 14 model. A degree 14 model fitted to such synthetic 'data' yields negligible error, but amplifies 4 nT (rmss) added noise into a 60 nT error (rmss); however, a degree 12 model fitted to the noisy 'data' suffers a 492 nT error (rmms through degree 12). Geomagnetic measurements remain unaware of model truncation, so the small errors indicated by some simulations cannot be realized in practice. Errors in models fitted to scalar data alone approach 1000 nT (rmss) and several thousand nT (maximum).

Voorhies, C. V.↗
DOE OSTI2025

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗
DOE OSTI2023

An Edge Alignment-Based Orientation Selection Method for Neutron Tomography

Neutron computed tomography (nCT) is a 3D char-acterization technique used to image the internal morphology or chemical composition of samples in biology and materials sciences. A typical workflow involves placing the sample in the path of a neutron beam, acquiring projection data at a predefined set of orientations, and processing the resulting data using an analytic reconstruction algorithm. Typical nCT scans require hours to days to complete and are then processed using conventional filtered back-projection (FBP), which performs poorly with sparse views or noisy data. Hence, the main methods in order to reduce overall acquisition time are the use of an improved sampling strategy combined with the use of advanced reconstruction methods such as model-based iterative reconstruction (MBIR). In this paper, we propose an adaptive orientation selection method in which an MBIR reconstruction on previously-acquired measurements is used to define an objective function on orientations that balances a data-fitting term promoting edge alignment and a regularization term promoting orientation diversity. Using simulated and experimental data, we demonstrate that our method produces high-quality reconstructions using significantly fewer total measurements than the conventional approach.

Yang, Diyu↗
DOE OSTI2024

Characterization of Crystal Properties and Defects in CdZnTe Radiation Detectors

CdZnTe-based detectors are highly valued because of their high spectral resolution, which is an essential feature for nuclear medical imaging. However, this resolution is compromised when there are substantial defects in the CdZnTe crystals. In this study, we present a learning-based approach to determine the spatially dependent bulk properties and defects in semiconductor detectors. This characterization allows us to mitigate and compensate for the undesired effects caused by crystal impurities. We tested our model with computer-generated noise-free input data, where it showed excellent accuracy, achieving an average RMSE of 0.43% between the predicted and the ground truth crystal properties. In addition, a sensitivity analysis was performed to determine the effect of noisy data on the accuracy of the model.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1988

An adaptive identification and control scheme for large space structures

A unified identification and control scheme capable of achieving space at form performance objectives under nominal or failure conditions is described. Preliminary results are also presented, showing that the methodology offers much promise for effective robust control of large space structures. The control method is a multivariable, adaptive, output predictive controller called Model Predictive Control (MPC). MPC uses a state space model and input reference trajectories of set or tracking points to adaptively generate optimum commands. For a fixed model, MPC processes commands with great efficiency, and is also highly robust. A key feature of MPC is its ability to control either nonminimum phase or open loop unstable systems. As an output controller, MPC does not explicitly require full state feedback, as do most multivariable (e.g., Linear Quadratic) methods. Its features are very useful in LSS operations, as they allow non-collocated actuators and sensors. The identification scheme is based on canonical variate analysis (CVA) of input and output data. The CVA technique is particularly suited for the measurement and identification of structural dynamic processes - that is, unsteady transient or dynamically interacting processes such as between aerodynamics and structural deformation - from short, noisy data. CVA is structured so that the identification can be done in real or near real time, using computationally stable algorithms. Modeling LSS dynamics in 1-g laboratories has always been a major impediment not only to understanding their behavior in orbit, but also to controlling it. In cases where the theoretical model is not confirmed, current methods provide few clues concerning additional dynamical relationships that are not included in the theoretical models. CVA needs no a priori model data, or structure; all statistically significant dynamical states are determined using natural, entropy-based methods. Heretofore, a major limitation in applying adaptive controllers to LSS problems has been the requirement for accurate models. We feel that MPC coupled with the CVA identification scheme will resolve this limitation. The results are based on research performed under an SBIR grant, administered by the Air Force Office of Scientific Research.

Carroll, J. V.↗
NASA NTRS2002

Locally-Based Kernal PLS Smoothing to Non-Parametric Regression Curve Fitting

We present a novel smoothing approach to non-parametric regression curve fitting. This is based on kernel partial least squares (PLS) regression in reproducing kernel Hilbert space. It is our concern to apply the methodology for smoothing experimental data where some level of knowledge about the approximate shape, local inhomogeneities or points where the desired function changes its curvature is known a priori or can be derived based on the observed noisy data. We propose locally-based kernel PLS regression that extends the previous kernel PLS methodology by incorporating this knowledge. We compare our approach with existing smoothing splines, hybrid adaptive splines and wavelet shrinkage techniques on two generated data sets.

Rosipal, Roman↗
NASA NTRS2012

Pattern Recognition Algorithm for High-Sensitivity Odorant Detection in Unknown Environments

In a realistic odorant detection application environment, the collected sensory data is a mix of unknown chemicals with unknown concentrations and noise. The identification of the odorants among these mixtures is a challenge in data recognition. In addition, deriving their individual concentrations in the mix is also a challenge. A deterministic analytical model was developed to accurately identify odorants and calculate their concentrations in a mixture with noisy data.

Duong, Tuan A.↗
DOE OSTI2023

Addressing APC Data Sparsity in Predicting Occupancy and Delay of Transit Buses: A Multitask Learning Approach

Public transit is a vital mode of transportation in urban areas, and its efficiency is crucial for the daily commute of millions of people. To improve the reliability and predictability of transit systems, researchers have developed separate single-task learning models to predict the occupancy and delay of buses at the stop or route level. However, these models provide a narrow view of delay and occupancy at each stop and do not account for the correlation between the two. We propose a novel approach that leverages broader generalizable patterns governing delay and occupancy for improved prediction. We introduce a multitask learning toolchain that takes into account General Transit Feed Specification feeds, Automatic Passenger Counter data, and contextual temporal and spatial information. The toolchain predicts transit delay and occupancy at the stop level, improving the accuracy of the predictions of these two features of a trip given sparse and noisy data. We also show that our toolchain can adapt to fewer samples of new transit data once it has been trained on previous routes/trips as compared to state-of-the-art methods. Finally, we use actual data from Chattanooga, Tennessee, to validate our approach. We compare our approach against the state-of-the-art methods and we show that treating occupancy and delay as related problems improves the accuracy of the predictions. We show that our approach improves delay prediction significantly by as much as 4% in F1 scores while producing equivalent or better results for occupancy.

Zulqarnain, Ammar Bin↗
NASA NTRS1990

Maximum likelihood estimates of polar motion parameters

Two estimators developed by Jeffreys (1940, 1968) are described and used in conjunction with polar-motion data to determine the frequency (Fc) and quality factor (Qc) of the Chandler wobble. Data are taken from a monthly polar-motion series, satellite laser-ranging results, and optical astrometry and intercompared for use via interpolation techniques. Maximum likelihood arguments were employed to develop the estimators, and the assumption that polar motion relates to a Gaussian random process is assessed in terms of the accuracies of the estimators. The present results agree with those from Jeffreys' earlier study but are inconsistent with the later estimator; a Monte Carlo evaluation of the estimators confirms that the 1968 method is more accurate. The later estimator method shows good performance because the Fourier coefficients derived from the data have signal/noise levels that are superior to those for an individual datum. The method is shown to be valuable for general spectral-analysis problems in which isolated peaks must be analyzed from noisy data.

Wilson, Clark R.↗
NASA NTRS2013

Transfer Function Identification Using Orthogonal Fourier Transform Modeling Functions

A method for transfer function identification, including both model structure determination and parameter estimation, was developed and demonstrated. The approach uses orthogonal modeling functions generated from frequency domain data obtained by Fourier transformation of time series data. The method was applied to simulation data to identify continuous-time transfer function models and unsteady aerodynamic models. Model fit error, estimated model parameters, and the associated uncertainties were used to show the effectiveness of the method for identifying accurate transfer function models from noisy data.

Morelli, Eugene A.↗
NASA NTRS1990

Estimating the exceedance probability of rain rate by logistic regression

Recent studies have shown that the fraction of an area with rain intensity above a fixed threshold is highly correlated with the area-averaged rain rate. To estimate the fractional rainy area, a logistic regression model, which estimates the conditional probability that rain rate over an area exceeds a fixed threshold given the values of related covariates, is developed. The problem of dependency in the data in the estimation procedure is bypassed by the method of partial likelihood. Analyses of simulated scanning multichannel microwave radiometer and observed electrically scanning microwave radiometer data during the Global Atlantic Tropical Experiment period show that the use of logistic regression in pixel classification is superior to multiple regression in predicting whether rain rate at each pixel exceeds a given threshold, even in the presence of noisy data. The potential of the logistic regression technique in satellite rain rate estimation is discussed.

Chiu, Long S.↗
NASA NTRS1990

Adaptive box filters for removal of random noise from digital images

Adaptive box-filtering algorithms to remove random bit errors and to smooth noisy data have been developed. For both procedures, the standard deviation of those pixels within a local box surrounding each pixel is used. A series of two or three filters with decreasing box sizes can be run to clean up extremely noisy images and to remove bit errors near sharp edges. The second filter, for noise smoothing, is similar to the 'sigma filter' of Lee (1983). The technique effectively reduces speckle in radar images without eliminating fine details.

Eliason, Eric M.↗
DOE OSTI2026

Predicting Open Quantum Dynamics with Data-Informed Quantum-Classical Dynamics

We introduce a data-informed quantum-classical dynamics (DIQCD) approach for predicting the evolution of an open quantum system. The equation of motion in DIQCD is a Lindblad equation with a flexible, time-dependent Hamiltonian that can be optimized to fit sparse and noisy data from local observations of an extensive open quantum system. We demonstrate the accuracy and efficiency of DIQCD for both experimental and simulated quantum devices. We show that DIQCD can predict entanglement dynamics of ultracold molecules (calcium fluoride) in optical tweezer arrays. DIQCD also successfully predicts carrier mobility in organic semiconductors (rubrene) with accuracy comparable to nearly exact numerical methods.

Lindblad equation↗