SEARCH · Engineering Papers

Results for “Noisy data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Multi-fidelity Bayesian neural networks: Algorithms and applications

Here we propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity, and we apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs). These multi-fidelity BNNs consist of three neural networks: The first is a fully connected neural network, which is trained following the maximum a posteriori probability (MAP) method to fit the low-fidelity data; the second is a Bayesian neural network employed to capture the cross-correlation with uncertainty quantification between the low- and high-fidelity data; and the last one is the physics-informed neural network, which encodes the physical laws described by PDEs. For the training of the last two neural networks, we first employ the mean-field variational inference (VI) to maximize the evidence lower bound (ELBO) to obtain informative prior distributions for the hyperparameters in the BNNs, and subsequently we use the Hamiltonian Monte Carlo (HMC) method to estimate accurately the posterior distributions for the corresponding hyperparameters. We demonstrate the accuracy of the present method using synthetic data as well as real measurements. Specifically, we first approximate a one- and four-dimensional function, and then infer the reaction rates in one- and two-dimensional diffusion-reaction systems. Moreover, we infer the sea surface temperature (SST) in the Massachusetts and Cape Cod Bays using satellite images and in-situ measurements. Taken together, our results demonstrate that the present method can capture both linear and nonlinear correlation between the low- and high-fidelity data adaptively, identify unknown parameters in PDEs, and quantify uncertainties in predictions, given a few scattered noisy high-fidelity data. Finally, we demonstrate that we can effectively and efficiently reduce the uncertainties and hence enhance the prediction accuracy with an active learning approach, using as examples a specific one-dimensional function approximation and an inverse PDE problem.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2010

Performance Analysis of Apollo Navigational Starter Routine

The focus of this project is to recreate and analyze the effectiveness of the original Apollo Starter Routine (ASR) which was used to generate the state vector of the Apollo spacecraft based on a series of radiometric observations. The original Apollo navigation software is unavailable in a modern programming language and the original coding has not been preserved. This necessitates its recreation using the original software documentation. Space Shuttle navigation software does not typically use the ASR or an algorithm like it since the Shuttle s state vector is easily deduced from GPS information or other sources. However, this tactic will be ineffective when trying to determine the state vector of a craft approaching, departing or in orbit around the Moon since the GPS network faces the surface of the Earth, not outer space. The recreation of the ASR from the original documentation is therefore vital as a simulation baseline for the navigation software under development for the Constellation program. The algorithms that make up the ASR will be extracted from the original documentation and adapted for and then implemented in a modern programming language; the majority of it will be coded in Matlab. The ASR s effectiveness will then be tested using simulated tracking data. The ability of the ASR to handle realistically noisy data and the accuracy with which it generates state vectors were analyzed. The ASR proved to be robust enough to process data with range and angle noise as large as 10,000 meters and 10(exp -6) radians together and 300,000 meters and 5x10(exp -4) radians separately at Lunar distances. The ASR was able to handle marginally more noise at distances closer to the Earth where the angle noise was less significant. The ASR is capable of effectively processing 40-80 data points gathered at a rate of one per 20 seconds at close Earth orbit and up to 28-40 data points gathered at a rate of one per minute at distant Earth orbit and Lunar orbit.

Ivanov, Stoyan I.↗

DOE OSTI2025

S AP F LOWER : an automated tool for sap flow data preprocessing, gap-filling, and analysis using deep learning

Sap flow, a critical process in plant water use and ecosystem water cycles, is often measured using thermal dissipation probes (TDP) due to their ease of installation and continuous data collection. However, sap flow data frequently include noise, outliers, and gaps, creating challenges for analysis and requiring substantial manual processing. We developed S AP F LOWER , a tool that automates data preprocessing, model training, gap-filling, sapwood area scaling and modeling, and water use analysis. It integrates autocleaning, machine learning and deep learning models (e.g. random forest, Gaussian process regression, long short-term memory (LSTM), bidirectional LSTM (BiLSTM)), and efficient workflows to process sap flow data. S AP F LOWER can remove over 90% of noisy data while preserving legitimate variations and achieve high accuracy in gap-filling based on user-determined parameters. Random forest, LSTM, and BiLSTM models reduced root mean square error to 10% or less for long-term gaps. Model training and prediction can be performed efficiently within seconds. S AP F LOWER significantly enhances the efficiency and accessibility of TDP data analysis by automating complex tasks, enabling researchers without programming expertise to employ advanced techniques. Future improvements will focus on species-specific corrections for TDP and support for additional measurement methods. S AP F LOWER is openly available on GitHub (https://github.com/JiaxinWang123/SapFlower) and Zenodo (doi: 10.5281/zenodo.13665919).

ecosystem water balance↗

DOE OSTI2020

Multichannel spectral estimation in acoustics: A state-space approach

Spectral estimation is a necessary methodology to analyze the frequency content of noisy data sets especially in acoustic applications. Many spectral techniques have evolved starting with the classical Fourier transform methods based on the well-known Wiener-Khintchine relationship relating the covariance-to-spectral density as a transform pair culminating with more elegant model-based parametric techniques that apply prior knowledge of the data to produce a high-resolution spectral estimate. Multichannel spectral representations are a class of both nonparametric, as well as parametric, estimators that provide improved spectral estimates. In any case, classical nonparametric multichannel techniques can provide reasonable estimates when coupled with peak-peaking methods as long as the signal levels are reasonably high. Furthermore, parametric multichannel methods can perform quite well in low signal level environments even when applying simple peak-picking techniques. In this paper, the performance of both nonparametric (periodogram) and parametric (state-space) multichannel spectral estimation methods are investigated when applied to both synthesized noisy structural vibration data as well as data obtained from a sounding rocket flight. It is demonstrated that for the multichannel problem, state-space techniques provide improved performance, offering a parametric alternative compared to classical methods.

42 ENGINEERING↗

NASA NTRS1999

The Calculation of Fractal Dimension in the Presence of Non-Fractal Clutter

The area of information processing has grown dramatically over the last 50 years. In the areas of image processing and information storage the technology requirements have far outpaced the ability of the community to meet demands. The need for faster recognition algorithms and more efficient storage of large quantities of data has forced the user to accept less than lossless retrieval of that data for analysis. In addition to clutter that is not the object of interest in the data set, often the throughput requirements forces the user to accept "noisy" data and to tolerate the clutter inherent in that data. It has been shown that some of this clutter, both the intentional clutter (clouds, trees, etc) as well as the noise introduced on the data by processing requirements can be modeled as fractal or fractal-like. Traditional methods using Fourier deconvolution on these sources of noise in frequency space leads to loss of signal and can, in many cases, completely eliminate the target of interest. The parameters that characterize fractal-like noise (predominately the fractal dimension) have been investigated and a technique to reduce or eliminate noise from real scenes has been developed. Examples of clutter reduced images are presented.

Herren, Kenneth A.↗

DOE OSTI2022

Physics-Based Method for Generating Fully Synthetic IV Curve Training Datasets for Machine Learning Classification of PV Failures

Classification machine learning models require high-quality labeled datasets for training. Among the most useful datasets for photovoltaic array fault detection and diagnosis are module or string current-voltage (IV) curves. Unfortunately, such datasets are rarely collected due to the cost of high fidelity monitoring, and the data that is available is generally not ideal, often consisting of unbalanced classes, noisy data due to environmental conditions, and few samples. In this paper, we propose an alternate approach that utilizes physics-based simulations of string-level IV curves as a fully synthetic training corpus that is independent of the test dataset. In our example, the training corpus consists of baseline (no fault), partial soiling, and cell crack system modes. The training corpus is used to train a 1D convolutional neural network (CNN) for failure classification. The approach is validated by comparing the model’s ability to classify failures detected on a real, measured IV curve testing corpus obtained from laboratory and field experiments. Results obtained using a fully synthetic training dataset achieve identical accuracy to those obtained with use of a measured training dataset. When evaluating the measured data’s test split, a 100% accuracy was found both when using simulations or measured data as the training corpus. When evaluating all of the measured data, a 96% accuracy was found when using a fully synthetic training dataset. The use of physics-based modeling results as a training corpus for failure detection and classification has many advantages for implementation as each PV system is configured differently, and it would be nearly impossible to train using labeled measured data.

Hopwood, Michael W. (ORCID:0000000161901767)↗

DOE OSTI2023

Data-Centric AI and the Open Energy Data Initiative (OEDI)

This presentation emphasizes the critical importance of data-centric AI. The limitations of model-centric AI when dealing with poor or insufficient data are highlighted, and it is illustrated how training models on inaccurate or noisy data leads to suboptimal results. This talk advocates for a hybrid approach that combines a focus on data quality and model parameters to achieve optimal results. The Open Energy Data Initiative (OEDI) is introduced as a valuable resource for obtaining high-quality energy-related datasets, hosting nearly 2,000 publicly accessible datasets, including 99 solar-related datasets, totaling over 2.7 petabytes of data. OEDI's data lakes enable users to query and work with data without extensive transfers. In conclusion, the significance of data-centric AI and adherence to data curation best practices is emphasized, positioning OEDI as a prime source of high-quality data for AI and machine learning in the renewable energy sector.

AI↗

NASA NTRS1985

Future possibilities in objective analysis and data assimilation for atmospheric dynamics

The role that dynamics plays in estimating the state of the atmosphere from incomplete and noisy data is reviewed. Objective analysis represents an attempt at relying mostly on the data and minimizing the role of dynamics in the estimation. Data assimilation tries to balance properly the roles of dynamical and observational information. Sequential estimation is presented as the proper framework for understanding this balance, and the Kalman filter as the ideal, optimal procedure for data assimilation. The optimal filter computes forecast error covariances of a given atmospheric model exactly, and hence data assimilation should be closely connected with predictability studies. This connection is described, and consequences drawn for currently active areas of the atmospheric and related sciences, namely, mesoscale meteorology, long range forecasting, and upper ocean dynamics. Possibilities offered by judicious data assimilation in understanding barotropic adjustment, a phenomenon that appears to play a crucial role in atmospheric behavior on the scale of weeks to months, and hence in long range forecasting are addressed.

Ghil, M.↗

DOE OSTI2024

NeuroSEM: A hybrid framework for simulating multiphysics problems by coupling PINNs and spectral elements

Multiphysics problems that are characterized by complex interactions among fluid dynamics, heat transfer, structural mechanics, and electromagnetics, are inherently challenging due to their coupled nature. While experimental data on certain state variables may be available, integrating these data with numerical solvers remains a significant challenge. Physics-informed neural networks (PINNs) have shown promising results in various engineering disciplines, particularly in handling noisy data and solving inverse problems in partial differential equations (PDEs). However, their effectiveness in forecasting nonlinear phenomena in multiphysics regimes, particularly involving turbulence, is yet to be fully established. Here, this study introduces NeuroSEM, a hybrid framework integrating PINNs with the highfidelity Spectral Element Method (SEM) solver, Nektar++. NeuroSEM leverages the strengths of both PINNs and SEM, providing robust solutions for multiphysics problems. PINNs are trained to assimilate data and model physical phenomena in specific subdomains, which are then integrated into the Nektar++ solver. We demonstrate the efficiency and accuracy of NeuroSEM for thermal convection in cavity flow and flow past a cylinder. The framework effectively handles data assimilation by addressing those subdomains and state variables where the data is available. We applied NeuroSEM to the Rayleigh-B´enard convection system, including cases with missing thermal boundary conditions and noisy datasets. Finally, we applied the proposed NeuroSEM framework to real particle image velocimetry (PIV) data to capture flow patterns characterized by horseshoe vortical structures. Our results indicate that NeuroSEM accurately models the physical phenomena and assimilates the data within the specified subdomains. The framework’s plug-and-play nature facilitates its extension to other multiphysics or multiscale problems. Furthermore, NeuroSEM is optimized for efficient execution on emerging integrated GPU-CPU architectures. This hybrid approach enhances the accuracy and efficiency of simulations, making it a powerful tool for tackling complex engineering challenges in various scientific domains.

42 ENGINEERING↗

NASA NTRS1970

Simple data-smoothing and noise-suppression technique

Algorithm, based on the Borel method of summing divergent sequences, is used for smoothing noisy data where knowledge of frequency content is not required. Technique's effectiveness is demonstrated by a series of graphs.

Duty, R. L.↗

DOE OSTI2022

Accounting for uncertainty sources in pyrometric measurements: Variables affecting Surface Temperature measurements using the LLNL Streak Spectrometers

Here we present a detailed analysis of the error sensitivities of the LLNL temperature analysis package used in analyzing streak spectrometer data. These sensitivities include noise, temperature, wavelength, emissivity, missing bands, and missing data points. The analysis package was tested for its ability to accurately reconstruct randomly generated noisy data and found to perform within specifications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1986

Sequential estimation and satellite data assimilation in meteorology and oceanography

The role of dynamics in estimating the state of the atmosphere and ocean from incomplete and noisy data is discussed and the classical applications of four-dimensional data assimilation to large-scale atmospheric dynamics are presented. It is concluded that sequential updating of a forecast model with continuously incoming conventional and remote-sensing data is the most natural way of extracting the maximum amount of information from the imperfectly known dynamics, on the one hand, and the inaccurate and incomplete observations, on the other.

Ghil, M.↗

NASA NTRS1989

Identification and interpretation of patterns in rocket engine data: Artificial intelligence and neural network approaches

This paper describes an expert system which is designed to perform automatic data analysis, identify anomalous events, and determine the characteristic features of these events. We have employed both artificial intelligence and neural net approaches in the design of this expert system. The artificial intelligence approach is useful because it provides (1) the use of human experts' knowledge of sensor behavior and faulty engine conditions in interpreting data; (2) the use of engine design knowledge and physical sensor locations in establishing relationships among the events of multiple sensors; (3) the use of stored analysis of past data of faulty engine conditions; and (4) the use of knowledge-based reasoning in distinguishing sensor failure from actual faults. The neural network approach appears promising because neural nets (1) can be trained on extremely noisy data and produce classifications which are more robust under noisy conditions than other classification techniques; (2) avoid the necessity of noise removal by digital filtering and therefore avoid the need to make assumptions about frequency bands or other signal characteristics of anomalous behavior; (3) can, in effect, generate their own feature detectors based on the characteristics of the sensor data used in training; and (4) are inherently parallel and therefore are potentially implementable in special-purpose parallel hardware.

Ali, Moonis↗

NASA NTRS2002

Abstracts for the 2002 Meeting of the AIAA Working Group on Dynamic Space Simulation

Undersampling is sometimes used in order to save memory and reducing digitizing speed requirements for pyroshock data. Many interpolation techniques have been used to estimate the missing peaks due undersampling. The more exotic techniques may not work well when real data that contains noise is used as an input. Various peak interpolation techniques will be evaluated using noisy data in this presentation.

Simpson, Alda D.↗

DOE OSTI2024

Machine learning-driven descriptions of protein dynamics at solid-liquid interfaces

This chapter has described how ML has enabled quantitative analysis of HS-AFM data to discover the physical phenomena governing protein dynamics and ordering at solid-liquid interfaces. The research detailed in this chapter modeled the rotation models of protein nanorods, the discovery of which would otherwise not be possible. By tracking the trajectories of individual protein rods from frame to frame, it was possible to model Brownian type motion and behaviors and Levy-flight dynamics that had not previously been shown. We also described the application of the Python package AtomAI, which has been developed specifically to analyze and extract physical phenomena, providing exemplar code for training an ensemble of deep neural networks to produce the semantic segmentation of AFM data and functions for encoding and decoding local environments. We last described a combinatorial approach to analyze very noisy data with a densely covered substrate where the emergence of order for the protein liquid crystals could be elucidated. By combining the methods from Case 1 and 2, it was possible to obtain the center of mass and angle for each rod in the images and track the assembly of the rods over time into a 2D liquid crystal array on the surface of mica.

protein dynamics, solid-liquid interfaces, atomic ↗

DOE OSTI2023

Fourier-informed knot placement schemes for B-spline approximation

Fitting B-splines to scientific data is especially challenging when the given data contain noise, jumps, or corners. Here, we describe how periodic data sets with these features can be efficiently approximated with B-splines by analyzing the Fourier spectrum of the data. Our method uses a collection of spectral filters to compute high-order derivatives, smoothed versions of noisy data, and the locations of jump discontinuities. Further, these quantities are then combined to choose knots that capture the qualitative features of the data, leading to accurate B-spline approximations with few knots. The method we introduce is direct and does not require any intermediate B-spline fitting before choosing the final knot distribution. Aside from fast Fourier transforms to transfer to and from Fourier space, the method runs in linear time with very little communication. We assess performance on several test cases in one and two dimensions, including data sets with jump discontinuities and noise. These tests show the method fits discontinuous data without spurious oscillations and remains effective in the presence of noise.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2011

EEG Artifact Removal Using a Wavelet Neural Network

!n this paper we developed a wavelet neural network. (WNN) algorithm for Electroencephalogram (EEG) artifact removal without electrooculographic (EOG) recordings. The algorithm combines the universal approximation characteristics of neural network and the time/frequency property of wavelet. We. compared the WNN algorithm with .the ICA technique ,and a wavelet thresholding method, which was realized by using the Stein's unbiased risk estimate (SURE) with an adaptive gradient-based optimal threshold. Experimental results on a driving test data set show that WNN can remove EEG artifacts effectively without diminishing useful EEG information even for very noisy data.

Nguyen, Hoang-Anh T.↗

NASA NTRS2024

Noise Reduction for Solar-Induced Fluorescence Retrievals Using Machine Learning and Principal Component Analysis: Simulations and Applications to GOME-2 Satellite Retrievals

We use a spectral-based approach that employs principal component analysis along with a relatively shallow artificial neural network (NN) to substantially reduce noise and other artifacts in terrestrial chlorophyll solar-induced fluorescence (SIF) retrievals. SIF is a very small emission at red and far-red wavelengths that is difficult to measure and is highly sensitive to random errors and systematic artifacts. Our approach relies upon an assumption that a trained NN can effectively reconstruct the total SIF signal from a relatively small number of leading principal components of the satellite-observed far-red radiance spectra without using information from the trailing modes that contain most of the random errors. We test the approach with simulated reflectance spectra produced with a full atmospheric and surface radiative transfer model using different observing and geophysical parameters and various noise levels. Resulting noisy and noise-reduced retrieved SIF values are compared with true values to assess performance. We then apply our noise reduction approach to SIF derived from two different satellite spectrometers. For evaluation, since the truth in this case is unknown, we compare SIF retrievals from two independent sensors with each other. We also compare the noise-reduced SIF temporal variations with those from an independent gross primary product (GPP) product that should display similar variations. Results show that our noise reduction approach improves capture of SIF seasonal and inter-annual variability. Our approach should be applicable to many noisy data products derived from spectral measurements. Our methodology does not replace the original retrieval algorithms; rather the original noisy retrievals are needed as the target for the NN training process.

noise reduction, solar-induced fluorescence, SIF, ↗