Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Noisy data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Probabilistic Autoencoder for Type Ia Supernova Spectral Time Series

We construct a physically parameterized probabilistic autoencoder (PAE) to learn the intrinsic diversity of Type Ia supernovae (SNe Ia) from a sparse set of spectral time series. The PAE is a two-stage generative model, composed of an autoencoder that is interpreted probabilistically after training using a normalizing flow. We demonstrate that the PAE learns a low-dimensional latent space that captures the nonlinear range of features that exists within the population and can accurately model the spectral evolution of SNe Ia across the full range of wavelength and observation times directly from the data. By introducing a correlation penalty term and multistage training setup alongside our physically parameterized network, we show that intrinsic and extrinsic modes of variability can be separated during training, removing the need for the additional models to perform magnitude standardization. We then use our PAE in a number of downstream tasks on SNe Ia for increasingly precise cosmological analyses, including the automatic detection of SN outliers, the generation of samples consistent with the data distribution, and solving the inverse problem in the presence of noisy and incomplete data to constrain cosmological distance measurements. We find that the optimal number of intrinsic model parameters appears to be three, in line with previous studies, and show that we can standardize our test sample of SNe Ia with an rms of 0.091 ± 0.010 mag, which corresponds to 0.074 ± 0.010 mag if peculiar velocity contributions are removed.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

B-DeepONet: An enhanced Bayesian DeepONet for solving noisy parametric PDEs using accelerated replica exchange SGLD

Here, the Deep Operator Network (DeepONet) is a neural network architecture used to approximate operators, including the solution operator of parametric PDEs. DeepONets have shown remarkable approximation ability. However, the performance of DeepONets deteriorates when the training data is polluted with noise, a scenario that occurs in practice. To handle noisy data, we propose a Bayesian DeepONet based on replica exchange Langevin diffusion (reLD). Replica exchange uses two particles. The first particle trains a DeepONet to exploit the loss landscape and make predictions. The other particle trains a different DeepONet to explore the loss landscape and escape local minima via swapping. Compared to DeepONets trained with state-of-the-art gradient-based algorithms (e.g., Adam), the proposed Bayesian DeepONet greatly improves the training convergence for noisy scenarios and accurately estimates the uncertainty. To further reduce the high computational cost of the reLD training of DeepONets, we propose (1) an accelerated training framework that exploits the DeepONet's architecture to reduce its computational cost up to 25% without compromising performance and (2) a transfer learning strategy that accelerates training DeepONets for PDEs with different parameter values. Finally, we illustrate the effectiveness of the proposed Bayesian DeepONet using four parametric PDE problems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Comparing Ice Jam Hindcasting Models with Tree Scar Data

Hindcasting models can use historic ice jam observations and hydroclimatic data to identify conditions that form ice jams.However, historic ice jam records are often sparse or incomplete. New sources of historic ice jam data could improve hindcasting models, leading to better ice jam forecasting and flood warning systems. Because ice jams damage riparian trees, marker rings associated with historic scars include information about ice jam frequency and severity. This study examined marker rings from 56 trees along the Muskegon River to supplement the historic ice jam data on this system. The study team compared tree ring data to results from hindcasting models, which were independently validated with newspaper reports on 1,500 separate days. Logistic regression converted the marker ring data into annual ice jam probabilities. Ice jam dates from the dendrochronology data were too noisy to train or validate a hindcasting model. However, the marker ring data did confirm that ice jams on the Muskegon are nonstationary. Ice jams are significantly more likely now than they were before 1966.The marker rings also helped to distinguish between false-negatives and nondetects in the hindcasting model.

Stanford Gibson↗
NASA NTRS2012

Sampling Technique for Robust Odorant Detection Based on MIT RealNose Data

This technique enhances the detection capability of the autonomous Real-Nose system from MIT to detect odorants and their concentrations in noisy and transient environments. The lowcost, portable system with low power consumption will operate at high speed and is suited for unmanned and remotely operated long-life applications. A deterministic mathematical model was developed to detect odorants and calculate their concentration in noisy environments. Real data from MIT's NanoNose was examined, from which a signal conditioning technique was proposed to enable robust odorant detection for the RealNose system. Its sensitivity can reach to sub-part-per-billion (sub-ppb). A Space Invariant Independent Component Analysis (SPICA) algorithm was developed to deal with non-linear mixing that is an over-complete case, and it is used as a preprocessing step to recover the original odorant sources for detection. This approach, combined with the Cascade Error Projection (CEP) Neural Network algorithm, was used to perform odorant identification. Signal conditioning is used to identify potential processing windows to enable robust detection for autonomous systems. So far, the software has been developed and evaluated with current data sets provided by the MIT team. However, continuous data streams are made available where even the occurrence of a new odorant is unannounced and needs to be noticed by the system autonomously before its unambiguous detection. The challenge for the software is to be able to separate the potential valid signal from the odorant and from the noisy transition region when the odorant is just introduced.

Duong, Tuan A.↗
NASA NTRS1998

An Innovative Method for Low Cost, Autonomous Navigation for Low Earth Orbit Satellites

An innovative approach to autonomous attitude, trajectory, and rate estimation is presented for low earth orbit (LEO) satellites which relies on magnetometers and sun sensors. These two sensors are reliable, inexpensive, and are used routinely in LEO missions for attitude determination and control. An extended Kalman filter (EKF) is developed from two existing systems, one which uses an EKF to estimate attitude and trajectory using magnetometer and gyro data and a second pseudo-linear filter which estimates rotation rate using magnetometer and sun sensor data. The theoretical background of the combined system is presented along with test results from noisy, simulated sensor data.

Deutschmann, Julie↗
NASA NTRS1981

MAGSAT anomaly profiles of the eastern Indian Ocean

Ground tracks from SEASAT were used in an effort to develop qualititative relationships between the gravity field and MAGSAT magnetic anomalies in the eastern Indian Ocean. Investigation of data quality led to analyses of the average value (over 80 vector data points, or approximately 36 km intervals) and of the standard deviation of this average, as a mean of identifying noisy portions of the data. It was discovered that the plots of the average value minus the individual (measured) point value are most useful for identifying noisy areas and data spikes. Spectrum analysis using edited (spikes removed) data show that the noise floor is less than 1 nT and the slope of the spectrum in the region of wavelengths between 1200 km and 250 km is approximately -3. Consequently the estimated resolution limit improved from approximately 360 km to approximately 250 km.

Sailor, R. V.↗
NASA NTRS2013

Clarifications on the "Comparison Between SMOS, VUA, ASCAT, and ECMWF Soil Moisture Products Over Four Watersheds in U.S."

In a recent paper, Leroux et al. compared three satellite soil moisture data sets (SMOS, AMSR-E, and ASCAT) and ECMWF forecast soil moisture data to in situ measurements over four watersheds located in the United States. Their conclusions stated that SMOS soil moisture retrievals represent "an improvement [in RMSE] by a factor of 2-3 compared with the other products" and that the ASCAT soil moisture data are "very noisy and unstable." In this clarification, the analysis of Leroux et al. is repeated using a newer version of the ASCAT data and additional metrics are provided. It is shown that the ASCAT retrievals are skillful, although they show some unexpected behavior during summer for two of the watersheds. It is also noted that the improvement of SMOS by a factor of 2-3 mentioned by Leroux et al. is driven by differences in bias and only applies relative to AMSR-E and the ECWMF data in the now obsolete version investigated by Leroux et al.

Scattering↗
DOE OSTI2021

Detection of Outliers in LiDAR Data Acquired by Multiple Platforms over Sorghum and Maize

High-resolution point cloud data acquired with a laser scanner from any platform contain random noise and outliers. Therefore, outlier detection in LiDAR data is often necessary prior to analysis. Applications in agriculture are particularly challenging, as there is typically no prior knowledge of the statistical distribution of points, plant complexity, and local point densities, which are crop-dependent. The goals of this study were first to investigate approaches to minimize the impact of outliers on LiDAR acquired over agricultural row crops, and specifically for sorghum and maize breeding experiments, by an unmanned aerial vehicle (UAV) and a wheel-based ground platform; second, to evaluate the impact of existing outliers in the datasets on leaf area index (LAI) prediction using LiDAR data. Two methods were investigated to detect and remove the outliers from the plant datasets. The first was based on surface fitting to noisy point cloud data via normal and curvature estimation in a local neighborhood. The second utilized the PointCleanNet deep learning framework. Both methods were applied to individual plants and field-based datasets. To evaluate the method, an F-score was calculated for synthetic data in the controlled conditions, and LAI, the variable being predicted, was computed both before and after outlier removal for both scenarios. Results indicate that the deep learning method for outlier detection is more robust than the geometric approach to changes in point densities, level of noise, and shapes. The prediction of LAI was also improved for the wheel-based vehicle data based on the coefficient of determination (R2) and the root mean squared error (RMSE) of the residuals before and after the removal of outliers.

36 MATERIALS SCIENCE↗
NASA NTRS1989

An adaptive data-smoothing routine

An adaptive noise reduction algorithm that can be implemented on a microcomputer is developed. Smoothing polynomials are used where the polynomial coefficients are chosen such that the mean-square-error between the noisy and smoothed data is minimized. This approach is equivalent to the implementation of a low-pass finite impulse response filter. The noise reduction depends on the order of the smoothing polynomial. A whiteness test on the error sequence is incorporated to search for the optimal smoothing. Expansion coefficients may be computed via the fast Fourier transform, and the resulting smoothing process is the equivalent of the implementation of an adaptive ideal low-pass filter. Results are obtained for an analytical signal with added white Gaussian noise. The routine may be applied to any smooth signal with additive random noise.

Taylor, Clayborne D.↗
NASA NTRS1983

Signal processing of anthropometric data

The Anthropometric Measurements Laboratory has accumulated a large body of data from a number of previous experiments. The data is very noisy, therefore it requires the application of some signal processing schemes. Moreover, it was not regarded as time series measurements but as positional information; hence, the data is stored as coordinate points as defined by the motion of the human body. The accumulated data defines two groups or classes. Some of the data was collected from an experiment designed to measure the flexibility of the limbs, referred to as radial movement. The remaining data was collected from experiments designed to determine the surface of the reach envelope. An interactive signal processing package was designed and implemented. Since the data does not include time this package does not include a time series element. Presently the results is restricted to processing data obtained from those experiments designed to measure flexibility.

Zimmermann, W. J.↗
NASA NTRS1989

Synopsis of early field test results from the gravity gradiometer survey system

Although the amount of data yielded by the initial airborne and surface tests was modest, it was sufficient to demonstrate that the full gravity gradient tensor was successfully measured from moving platforms both in the air and on the surface. The measurements were effectively continuous with spatial along-track resolution limited only by choice of integration lengths taken to reduce noise. The airborne data were less noisy (800 E squared/Hz typical) than were the Gravity Gradiometer Survey System (GGSS) measurements taken at the surface (5000 E squared/Hz typical). Single tracks of surface gravity disturbances recovered from airborne data were accurate to 3 to 4 mgal in each component of gravity when compared to 5 x 5 mean gravity anomalies over a 90 km track. Multitrack processing yielded 2 to 3 mgal when compared to 5 x 5 mean anomalies. Deflection of the vertical recovery over a distance of 150 km was about one arcsecond.

Brzezowski, S.↗
DOE OSTI2026

REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization

Applications of Implicit Neural Representations (INRs) have emerged as a promising deep learning approach for compactly representing large volumetric datasets. These models can act as surrogates for volume data, enabling efficient storage and on-demand reconstruction via model predictions. However, conventional deterministic INRs only provide value predictions without insights into the model’s prediction uncertainty or the impact of inherent noisiness in the data. This limitation can lead to unreliable data interpretation and visualization due to prediction inaccuracies in the reconstructed volume. Identifying erroneous results extracted from model-predicted data may be infeasible, as raw data may be unavailable due to its large size. To address this challenge, we introduce REV-INR, Regularized Evidential Implicit Neural Representation, which learns to predict data values accurately along with the associated coordinate-level data uncertainty and model uncertainty using only a single forward pass of the trained REV-INR during inference. By comprehensively comparing and contrasting REV-INR with existing well-established deep uncertainty estimation methods, we show that REV-INR achieves the best volume reconstruction quality with robust data (aleatoric) and model (epistemic) uncertainty estimates using the fastest inference time. Consequently, we demonstrate that REV-INR facilitates assessment of the reliability and trustworthiness of the extracted isosurfaces and volume visualization results, enabling analyses to be solely driven by model-predicted data.

Saklani, Shanu [Indian Institute of Technology, Ka↗
DOE OSTI2025

High-throughput spin-bath characterization of spin defects in semiconductors

Detailed knowledge of the local environments of spin defects in semiconductors, such as nitrogenvacancy (NV) centers in diamond or divacancies in silicon carbide, is crucial for optimizing control and entanglement protocols in quantum sensing and information applications. However, at present a direct experimental characterization of individual defect environments is not scalable, as conventional spin-bath measurements are time consuming and difficult to automate. Achieving high-throughput characterization requires short experiments to probe the spin bath. However, with fewer and noisier measurements, the inverse problem of recovering spin-bath properties from measured data becomes ill posed, with multiple spin baths having a high likelihood of yielding the same data. In this work, we present a set of computational tools to resolve the ill-posed inverse problem of recovering the atomic positions and hyperfine couplings of random nuclei surrounding spin defects from sparse, noisy experimental coherence data, which can be obtained in hours. Here, we use a trans-dimensional Bayesian approach that incorporates ab initio data to yield full posterior distributions over nuclear spin environments, enabling robust recovery from limited data. We also provide practical tools and guidelines to determine the limits of detectability for hyperfine couplings under specific dynamical decoupling sequences and sampling conditions. In addition, we demonstrate how the tools developed here, in combination with ab initio simulations of spin baths, can guide the design of efficient experimental protocols for application-specific high-throughput screening. To showcase the utility of our approach, we apply it to design fast dynamical decoupling experiments to characterize the spin baths often individual NV centers in diamond. While the primary focus is on accelerating spin-bath characterization of spin defects, this Bayesian approach also lays the foundation for digital-twin studies of spin defects, where a virtual model of the spin-defect system evolves in real time with ongoing experimental measurements. Together, the set of tools we designed and applied paves the way for scalable deployment of spin defects in semiconductors for quantum sensing and information applications.

Bayesian methods↗
DOE OSTI2025

Erratum: Centrality-Dependent Modification of Jet-Production Rates in Deuteron-Gold Collisions at $\sqrt{𝑠_{𝑁⁢𝑁}}$ = 200 GeV [Phys. Rev. Lett. 116, 122301 (2016)]

We previously reported jet-production rates in 𝑝 + 𝑝 and 𝑑 + Au collisions at $\sqrt{𝑠_{𝑁⁢𝑁}}$ = 200 GeV recorded in 2008 with the PHENIX detector at the Relativistic Heavy Ion Collider. Subsequently, we found that electronic noise in the electromagnetic calorimeter was not completely removed by the tower-timing cuts used in the analysis. We have conducted an independent re-analysis of the data that eliminated noisy towers by removing them completely from the analysis. These cuts remove any contributions from instrumental noise, but reduce the overall acceptance for electromagnetic clusters by ≈16% and hence the overall statistical power of the data. Finally, this re-analysis found that instrumental noise in the electromagnetic calorimeter led to an artificial increase in the jet yield for both the 𝑝 + 𝑝 and 𝑑 + Au collisions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Pin-pointing Node Failures in HPC Systems

Automated fault prediction and diagnosis in HPC systems needs to be efficient for better system resilience. With increasing scalability required for exascale, accurate fault prediction aiding in quick remedy is hard. With changing supercomputer architectures, distilling fault data from the noisy raw logs requires substantial efforts. Predicting node failures in such voluminous system logs is challenging. To this end, we investigate an interesting way to pin-point node failures in such supercomputing systems. Our study on Cray system data with automated machine learning tools suggests that specific patterns of event messages on node unavailability can be indicator to node failures. This data extraction coupled with system and job data correlation helps in devising a methodology to predict node failures and their location over a specific time frame. This work aims to enable broader applicability for a generic fault prediction framework.

Roman, E↗
DOE OSTI2024

G2Aero Database of Airfoils - Curated Airfoils

This dataset contains a curated set of 19,164 airfoil shapes from various applications and the data-driven design space of separable shape tensors (PGA space), which can be used as a parameter space for machine-learning applications focused on airfoil shapes. We constructed the airfoil dataset in two main stages. First, we identified 13 baseline airfoils from the NREL 5MW and IEA 15MW reference wind turbines. We reparameterized these shapes using least-squares fits of 8-order CST parametrizations, which involve 18 coefficients. By uniformly perturbing all 18 CST coefficients by +/-20% around each baseline airfoil, we generated 1,000 unique airfoils. Each airfoil was sampled with 1,001 shape landmarks whose x-coordinates followed a cosine distribution along the chord. This process resulted in a total of 13,000 airfoil shapes, each with 1,001 landmarks. In the second phase, we gathered additional airfoils from the extensive BigFoil database, which consolidates data from sources such as the University of Illinois Urbana-Champaign (UIUC) airfoil database, the JavaFoil database, the NACA-TR-824 database, and others. We undertook a thorough pre-processing step to filter out shapes with sparse, noisy, or incomplete data. We also removed airfoils with sharp leading edge and those exceeding our threshold for trailing edge thickness. Additionally, we thinned out the collection of NACA airfoils-- parametric sweeps of NACA airfoils with increasing thickness and camber present in BigFoil database-- by selecting every fourth step in the parameter sweeps. Finally, we regularized the airfoils by reparametrizing them with an 8-order CST parametrization (with 1,001 shape landmarks with x coordinated following cosine distribution along the chord) and removing airfoils with high reconstruction errors. This data pre-processing resulted in a set of 6,164 airfoils. In total, our curated airfoil dataset comprises 19,164 airfoils, each with 1,001 landmarks, and is stored in the curated_airfoils.npz file. Using this curated airfoil dataset, we utilized the separable shape tensors framework to develop a data-driven parameterization of airfoils based on principal geodesic analysis (PGA) of separable shape tensors. This PGA space is provided in PGAspace.npz file.

airfoils↗
NASA NTRS1993

Test-Signal Generator For SNR Calibration

Assembly of commercial and custom-made electronic equipment designed to generate noisy intermediate-frequency or baseband received phase-modulation data-communication signals with accurately known signal-to-noise ratios. Signals used to perform signal-to-noise-ratio calibrations and other tests of responses of data-communication receivers to noisy incoming signals. Underlying principle applicable to generation of test signals for other advanced data-communication receivers.

Gutierrez-Luaces, Benito O.↗