Handling noisy data in sparse model identification using subsampling and co-teaching
Not Available
Abdullah, Fahim↗
SEARCH · Engineering Papers
Results for “Noisy data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Emulators for scarce and noisy data: Application to auxiliary field diffusion Monte Carlo for the deuteron
Not Available
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Electronic specific heat capacities and entropies from density matrix quantum Monte Carlo using Gaussian process regression to find gradients of noisy data
In this work, we present a machine learning approach to calculating electronic specific heat capacities for a variety of benchmark molecular systems. Our models are based on data from density matrix quantum Monte Carlo, which is a stochastic method that can calculate the electronic energy at finite temperature. As these energies typically have noise, numerical derivatives of the energy can be challenging to find reliably. In order to circumvent this problem, we use Gaussian process regression to model the energy and use analytical derivatives to produce the specific heat capacity. From there, we also calculate the entropy by numerical integration. We compare our results to cubic splines and finite differences in a variety of molecules in which Hamiltonians can be diagonalized exactly with full configuration interaction. We finally apply this method to look at larger molecules where exact diagonalization is not possible and make comparisons with more approximate ways to calculate the specific heat capacity and entropy.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Emulators for Scarce and Noisy Data: Application to Auxiliary-Field Diffusion Monte Carlo for Neutron Matter
Understanding the equation of state (EOS) of pure neutron matter is necessary for interpreting multimessenger observations of neutron stars. Reliable data analyses of these observations require well-quantified uncertainties for the EOS input, ideally propagating uncertainties from nuclear interactions directly to the EOS. This, however, requires calculations of the EOS for a prohibitively larger number of nuclear Hamiltonians, solving the nuclear many-body problem for each one. Quantum Monte Carlo methods, such as auxiliary-field diffusion Monte Carlo (AFDMC), provide precise and accurate results for the neutron matter EOS, but they are very computationally expensive, making them unsuitable for the fast evaluations necessary for uncertainty propagation. Here, we employ parametric matrix models to develop fast emulators for AFDMC calculations of neutron matter and use them to directly propagate uncertainties of coupling constants in the Hamiltonian to the EOS. As these uncertainties include estimates of the effective field theory truncation uncertainty, this approach provides robust uncertainty estimates for use in astrophysical data analyses. In conclusion, this Letter will enable novel applications such as using astrophysical observations to put constraints on coupling constants for nuclear interactions.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Active Operator Inference for Learning Low-Dimensional Dynamical-System Models from Noisy Data
Not provided.
Mathematics↗
Probabilistic Approaches to Transfer Learning for Sparse and Noisy Data Environments.
Abstract not provided.
Khalil, Mohammad↗
From 2D to 3D: Recovering Turbulent Density Dispersions from Noisy Data
Explore the source record for details and available documents.
Astronomy and AstroPhysics↗
Analytic continuation of noisy data using Adams Bashforth residual neural network
We propose a data-driven learning framework for the analytic continuation problem in numerical quantum many-body physics. Designing an accurate and efficient framework for the analytic continuation of imaginary time using computational data is a grand challenge that has hindered meaningful links with experimental data. The standard Maximum Entropy (MaxEnt)-based method is limited by the quality of the computational data and the availability of prior information. Also, the MaxEnt is not able to solve the inversion problem under high level of noise in the data. Here we introduce a novel learning model for the analytic continuation problem using a Adams-Bashforth residual neural network (AB-ResNet). Additionally, the advantage of this deep learning network is that it is model independent and, therefore, does not require prior information concerning the quantity of interest given by the spectral function. More importantly, the ResNet-based model achieves higher accuracy than MaxEnt for data with higher level of noise. Finally, numerical examples show that the developed AB-ResNet is able to recover the spectral function with accuracy comparable to MaxEnt where the noise level is relatively small.
97 MATHEMATICS AND COMPUTING↗
Vector splines on the sphere with application to the estimation of vorticity and divergence from discrete, noisy data
Vector smoothing splines on the sphere are defined. Theoretical properties are briefly alluded to. The appropriate Hilbert space norms used in a specific meteorological application are described and justified via a duality theorem. Numerical procedures for computing the splines as well as the cross validation estimate of two smoothing parameters are given. A Monte Carlo study is described which suggests the accuracy with which upper air vorticity and divergence can be estimated using measured wind vectors from the North American radiosonde network.
Wahba, G.↗
Inference of model sparsity in nonlinear dynamics using noisy data.
Abstract not provided.
Sandhu, Rimple↗
Inference Of Model Sparsity In Nonlinear Dynamics Using Noisy Data.
Abstract not provided.
Sandhu, Rimple↗
Clustering of noisy image data using an adaptive neuro-fuzzy system
Identification of outliers or noise in a real data set is often quite difficult. A recently developed adaptive fuzzy leader clustering (AFLC) algorithm has been modified to separate the outliers from real data sets while finding the clusters within the data sets. The capability of this modified AFLC algorithm to identify the outliers in a number of real data sets indicates the potential strength of this algorithm in correct classification of noisy real data.
Pemmaraju, Surya↗
Development of Steady-State and Dynamic Mass and Energy Constrained Neural Networks for Distributed Chemical Systems Using Noisy Transient Data
The paper presents the development of algorithms for mass and energy constrained neural network models that can exactly conserve the overall mass and energy of distributed chemical process systems, even though the noisy transient data used for optimal model training violate the same. In contrast to approximately satisfying mass and energy balance constraints of a system by soft penalization of objective function, algorithms have been developed for solving equality-constrained nonlinear optimization problems, thus providing the guarantee of exactly satisfying the system mass and energy conservation laws. For developing dynamic mass-energy constrained network models for distributed systems, hybrid series and parallel dynamic-static neural networks have been leveraged. The developed algorithms for solving both the training and forward problems are validated using both steady-state and dynamic data in the presence of various noise characteristics. The developed data-driven algorithms are flexible to exactly satisfy mass and energy balance constraints for dynamic chemical processes if the system holdup information is available. The proposed network structures and algorithms are applied to the development of data-driven lumped and distributed models of an adiabatic superheater/reheater system, a nonisothermal continuous stirred tank reactor, as well as an electrically heated plug-flow reactor system where one form of energy gets transformed to another. It has been observed that the mass-energy constrained neural networks yield a root mean squared error of <1% with respect to the system truth for the case studies evaluated in this work.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Labeling sequential data from noisy annotations
Crowdsourcing algorithms often work under the assumption that the data samples are independent. Recent work has shown that data dependence, such as temporal correlations in sequential data, can be leveraged to improve the label quality. Existing methods that exploit this special structure rely on third-order statistics of the annotator outputs to ensure the identifiability of key latent parameters, which are costly to acquire. This work proposes an approach for integrating crowdsourced annotations under the Dawid-Skene/Hidden Markov Model (DS-HMM) for sequential data based on second-order statistics, which naturally enjoys a lower sample complexity. An effective algorithm is proposed to tackle the challenging optimization problem associated with the proposed estimator. Numerical experiments showcase the effectiveness of the data labeling paradigm.
Marrinan, Timothy P.↗
Online real-time learning of dynamical systems from noisy streaming data
Abstract Recent advancements in sensing and communication facilitate obtaining high-frequency real-time data from various physical systems like power networks, climate systems, biological networks, etc. However, since the data are recorded by physical sensors, it is natural that the obtained data is corrupted by measurement noise. In this paper, we present a novel algorithm for online real-time learning of dynamical systems from noisy time-series data, which employs the Robust Koopman operator framework to mitigate the effect of measurement noise. The proposed algorithm has three main advantages: (a) it allows for online real-time monitoring of a dynamical system; (b) it obtains a linear representation of the underlying dynamical system, thus enabling the user to use linear systems theory for analysis and control of the system; (c) it is computationally fast and less intensive than the popular extended dynamic mode decomposition (EDMD) algorithm. We illustrate the efficiency of the proposed algorithm by applying it to identify the Van der Pol oscillator, the chaotic attractor of the Henon map, the IEEE 68 bus system, and a ring network of Van der Pol oscillators.
97 MATHEMATICS AND COMPUTING↗
Source shape estimation for neutron imaging systems using convolutional neural networks
Neutron imaging systems are important diagnostic tools for characterizing the physics of inertial confinement fusion reactions at the National Ignition Facility (NIF). In particular, neutron images give diagnostic information on the size, symmetry, and shape of the fusion hot spot and surrounding cold fuel. Images are formed via collection of neutron flux from the source using a system of aperture arrays and scintillator-based detectors. Currently, reconstruction of fusion source geometry from the collected neutron images is accomplished by solving a computationally intensive maximum likelihood estimation problem via expectation maximization. In contrast, it is often useful to have simple representations of the overall source geometry that can be computed quickly. In this work, we develop convolutional neural networks (CNNs) to reconstruct the outer contours of simple source geometries. We compare the performance of the CNN for penumbral and pinhole data and provide experimental demonstrations of our methods on both non-noisy and noisy data.
Machine learning, neutron imaging, source reconstr↗
Review of Presentation by Carter Wolf
The presentation was informative on the different techniques used to study the electronic dynamics of materials on the femtosecond scale, such as ARPES and TR ARPES. The presentation also discussed in depth code analysis, and the procedure used to remove noise from noisy data via machine learning procedures. The presentation first focused on an overview of the ARPES technique, and how TR-ARPES differs from it. The presentation then explored computational techniques used for fitting ARPES and TR-ARPES data, using the pyARPES framework. After that, the presentation discussed machine learning approaches for denoising noisy data such as Noise2Noise and Noise2Self. The main issue of denoising existing ARPES data and the necessity of machine learning approaches due to the lack of clean ARPES data were very clearly articulated as a major part of the project. The implementation, training and refinement and optimization were clearly detailed, along with a full documentation of attempts that worked and attempts that did not, thus thoroughly and clearly showing the research process. Experimental techniques regarding ARPES were also briefly documented.
36 MATERIALS SCIENCE↗