Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data corruption”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

NEON AOP Survey of Upper East River CO Watersheds: LAZ Files, LiDAR Surface Elevation, Terrain Elevation, and Canopy Height Rasters

Lawrence Berkeley National Laboratory (LBNL) contracted the National Ecological Observatory Network Airborne Observation Platform (NEON AOP) to observe watersheds of interest surrounding Crested Butte, CO with remotely sensed data, including LiDAR. The flight box design encompassed the watersheds, surveying a total area of 334 km2 across 72 lines. The instrument used was an Optech Gemini, with a pulse density of 2-9 pulses m-2 across the study area (see final report document for detailed information (Goulden and Musinsky 2020). These LiDAR data are the primary data that were provided by NEON, and include both LiDAR source data and derived surface products. The source data are contained in 'crbu_laz.zip' and 'classified_point_cloud.zip. 'classified_point_cloud.zip' represents an additional processing level on 'crbu_laz.zip'. The derived products are 'dsm_mosaic_min_phase_me.tif', 'dtm_mosaic_min_phase_me.tif', and 'tch_mosaic_min_phase_me.tif'. These files are raster mosaics of a digital surface elevation model (DSM), digital terrain elevation model (DTM), and a canopy height model (CHM) at 1 m resolution with a height threshold of > 2 m, respectively, in geotiff (.tif) format. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231. [UPDATE] On 06/26/2026 the files 'crbu_laz.zip' and 'classified_point_cloud.zip' were replaced with original output files from NEON to address zip file corruption and naming errors. No changes were made to the underlying data. (See Methods Step 2 for details).

2018 NEON and 2025 CHESS Campaigns↗

Leipzig University W-Band Cloud Radar, Gothic (Colorado), SAIL Campaign Second Winter (15.11.2022 - 05.06.2023)

The instrument is a polarimetric W-Band simultaneous transmission simultaneous reception (STSR) Doppler Cloud Radar manufactured by Radiometer Physics GmbH (RPG, instrument type RPG-FMCW-94-DP). It was deployed about 500m away from the ARM M1 facility in Gothic (CO) near the Ore House between Nov 15, 2022 and Jun 5, 2023 at an altitude of 2913 m. The Doppler cloud radar was mounted on a prototype of a cold temperature scanner manufactured by RPG. The dataset contains common radar variables like equivalent radar reflectivity factor, mean Doppler velocity, Doppler spectrum width as well as common polarimetric radar variables like differential phase shift (PhiDP), specific differential phase shift (KDP), correlation coefficient (RHV) and differential reflectivity (ZDR). Between Nov 15, 2022, and Feb 7, 2023 the radar was operated at 40° constant elevation and 151° azimuth (towards the KAZR at the AMF1). Between Feb 7 and Mar 15, 2023 the following scan cycle was repeated in 15min intervals: zenith pointing measurements, RHI scans from 0° to 90° and 90° to 0° elevation at 151° azimuth, constant elevation measurements at 40° elevation and 151° azimuth and PPI scans at 85° elevation for wind retrievals (one PPI per hour). Between Mar 15 18:18 UTC and Apr 8, 2023 11:04 UTC, the scanner was stuck at 72° elevation, which resulted in corrupted files for this period. For that period, the files containing data are uploaded but should be treated with care. Between Apr 8 and Apr 14, 2023 the scanning pattern described above was performed again. Afterwards, except for May 17 and May 18, 2023 where scanning patterns as described above were performed, measurements continued at 40° elevation and 151° azimuth until May 22, 2023 09:00 UTC. The scanning pattern was then started again, but without hourly PPI measurements until May 31, 2023. Between May 31 and Jun 5, 2023 solely zenith-pointing observations were performed. The data provided here are the RPG Level 1 (L1) files, Doppler spectra Level 0 (L0) files are available upon request.

54 ENVIRONMENTAL SCIENCES↗

STUDIES IN ASTRONOMICAL TIME SERIES ANALYSIS. VI. BAYESIAN BLOCK REPRESENTATIONS

This paper addresses the problem of detecting and characterizing local variability in time series and other forms of sequential data. The goal is to identify and characterize statistically significant variations, at the same time suppressing the inevitable corrupting observational errors. We present a simple nonparametric modeling technique and an algorithm implementing it-an improved and generalized version of Bayesian Blocks-that finds the optimal segmentation of the data in the observation interval. The structure of the algorithm allows it to be used in either a real-time trigger mode, or a retrospective mode. Maximum likelihood or marginal posterior functions to measure model fitness are presented for events, binned counts, and measurements at arbitrary times with known error distributions. Problems addressed include those connected with data gaps, variable exposure, extension to piecewise linear and piecewise exponential representations, multivariate time series data, analysis of variance, data on the circle, other data modes, and dispersed data. Simulations provide evidence that the detection efficiency for weak signals is close to a theoretical asymptotic limit derived by Arias-Castro et al. In the spirit of Reproducible Research all of the code and data necessary to reproduce all of the figures in this paper are included as supplementary material.

79 ASTRONOMY AND ASTROPHYSICS↗

IDENTIFYING IONIZED REGIONS IN NOISY REDSHIFTED 21 cm DATA SETS

One of the most promising approaches for studying reionization is to use the redshifted 21 cm line. Early generations of redshifted 21 cm surveys will not, however, have the sensitivity to make detailed maps of the reionization process, and will instead focus on statistical measurements. Here, we show that it may nonetheless be possible to directly identify ionized regions in upcoming data sets by applying suitable filters to the noisy data. The locations of prominent minima in the filtered data correspond well with the positions of ionized regions. In particular, we corrupt semi-numeric simulations of the redshifted 21 cm signal during reionization with thermal noise at the level expected for a 500 antenna tile version of the Murchison Widefield Array (MWA), and mimic the degrading effects of foreground cleaning. Using a matched filter technique, we find that the MWA should be able to directly identify ionized regions despite the large thermal noise. In a plausible fiducial model in which {approx}20% of the volume of the universe is neutral at z {approx} 7, we find that a 500-tile MWA may directly identify as many as {approx}150 ionized regions in a 6 MHz portion of its survey volume and roughly determine the size of each of these regions. This may, in turn, allow interesting multi-wavelength follow-up observations, comparing galaxy properties inside and outside of ionized regions. We discuss how the optimal configuration of radio antenna tiles for detecting ionized regions with a matched filter technique differs from the optimal design for measuring power spectra. These considerations have potentially important implications for the design of future redshifted 21 cm surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Bi-sparsity pursuit: A paradigm for robust subspace recovery

The success of sparse models in computer vision and machine learning is due to the fact that, high dimensional data is distributed in a union of low dimensional subspaces in many real-world applications. The underlying structure may, however, be adversely affected by sparse errors. In this paper, we propose a bi-sparse model as a framework to analyze this problem, and provide a novel algorithm to recover the union of subspaces in the presence of sparse corruptions. We further show the effectiveness of our method by experiments on real-world vision data.

42 ENGINEERING↗

BI-sparsity pursuit for robust subspace recovery

Here, the success of sparse models in computer vision and machine learning in many real-world applications, may be attributed in large part, to the fact that many high dimensional data are distributed in a union of low dimensional subspaces. The underlying structure may, however, be adversely affected by sparse errors, thus inducing additional complexity in recovering it. In this paper, we propose a bi-sparse model as a framework to investigate and analyze this problem, and provide as a result , a novel algorithm to recover the union of subspaces in presence of sparse corruptions. We additionally demonstrate the effectiveness of our method by experiments on real-world vision data.

47 OTHER INSTRUMENTATION↗

Fitting local, low-dimensional parameterizations of optical turbulence modeled from optimal transport velocity vectors

This work exploits a connection between optimal transport theory and the physics of image propagation to yield a locally low-dimensional model of turbulence-corrupted imagery. Optimal transport produces an invertible, pixel-wise linear trajectories to approximate the globally nonlinear turbulence between a clean and turbulence corrupted image pair. We use the low-dimensional model to fit subsets of the optimal transport vector fields and stitch the local models into a surrogate for the global map to be used for image cleaning. Experiments are performed on laboratory generated data of beam propagation using different values of the Fried parameter (a scale measuring turbulence coherence) as well as a toy data set. The results suggest this is a fruitful direction, and first step, towards using multiple realizations of turbulence corrupted images to learn a blind surrogate for the optimal transport vector field for image cleaning.

47 OTHER INSTRUMENTATION↗

Attack-Resilient Weighted $\ell_{1}$ Observer with Prior Pruning

Security related questions for Cyber Physical Systems (CPS) have attracted much research attention in searching for novel methods for attack-resilient control and/or estimation. Specifically, false data injection attacks (FDIAs) have been shown to be capable of bypassing bad data detection (BDD), while arbitrarily compromising the integrity of state estimators and robust controller even with very sparse measurements corruption. Moreover, based on the inherent sparsity of pragmatic attack signals, ℓ1 -minimization scheme has been used extensively to improve the design of attack-resilient estimators. For this, the theoretical maximum for the percentage of compromised nodes that can be accommodated has been shown to be 50%. In order to guarantee correct state recoveries for larger percentage of attacked nodes, researchers have begun to incorporate prior information into the underlying resilient observer design framework. For the most pragmatic cases, this prior information is often obtained through some data-driven machine learning process. Existing results have shown strong positive correlation between the tolerated attack percentages and the precision of the prior information. In this paper, we present a pruning method to improve the precision of the prior information, given corresponding stochastic uncertainty characteristics of the underlying machine learning model. Then a weighted ℓ1 -minimization is proposed based on the pruned prior. The theoretical and simulation results show that the pruning method significantly improves the observer performance for much larger attack percentages, even when moderately accurate machine learning model used.

Resilient observer, Cyber-physical systems, prunin↗

Sensitivity of back‐projection algorithm (BPA) synthetic aperture radar (SAR) image formation to initial position, velocity, and attitude navigation errors

Abstract The Back‐Projection Algorithm (BPA) is a time‐domain‐matched filtering technique to form synthetic aperture radar (SAR) images. To produce high‐quality BPA images, precise navigation data for the radar platform must be known. Errors in position, velocity, or attitude result in improperly formed images that are corrupted by shifting and blurring. The contribution of this paper is the development of analytical expressions that characterise the relationship between navigation errors and image formation errors from an inertial navigation point of view, where trajectory estimation errors in position, velocity, and attitude propagate through time and cause compounding errors in the vehicle state vector. These analytical expressions are verified via simulated image formation and real‐data image formation.

Lindstrom, Colton↗

Development of Software Defined Radio Protocol for Acoustic Communication on Pipes

Nuclear Science Engineering Division at Argonne has been developing a new method of transmitting information via acoustic waves on pipes in a nuclear facility. This method provides the capability to communicate information from within the containment building under the conditions of total power outage following a severe accident. Compared to traditional RF communications, acoustic communications allows for signal transmission across thick concrete barriers, and better information security due to spatial confinement of the signal. Data of interest consists of sensor readings, voice, and images. Transmission of images is most challenging because corruption of the header string might render the reconstructed image unreadable.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Robustness of topological persistence in knowledge distillation for wearable sensor data

Topological data analysis (TDA) has shown great success in various applications involving wearable sensor data. However, there are difficulties in leveraging topological features in machine learning and wearable sensors because of the large time consumption and computational resources required to extract the features. To address this problem, knowledge distillation (KD) is utilized to generate a small model and accommodate topological features with persistence image (PI) representations from the raw time series data. Deploying topological knowledge in KD enables the student to achieve better performance compared to the one trained solely on raw time series data. However, it is not yet known if there are coherent characteristics for topological features in PI, which can aid in improving the performance during KD. In this paper, we investigate the suitability and challenges of utilizing topological features in KD for wearable sensor data, thereby contributing to the advancement of the field. Our study explores the impact of transferred topological features by comparing the Teacher-to-Student framework with Multiple Teachers-to-Student where teachers utilize both time series data and persistence images obtained by TDA as inputs. Additionally, we conduct a rigorous examination of topological knowledge effects by testing under various corruptions, knowledge types, and learning strategies in the context of human activity recognition tasks. Our analysis of topological features in KD presents the optimal strategy for incorporating these features. This study includes datasets of varying scales, window lengths, and activity classes, providing a comprehensive evaluation. Our results demonstrate that leveraging topological features in KD to enhance performance across databases.

97 MATHEMATICS AND COMPUTING↗

Analysis of the Impact of Radiation on Cameras and Robotic Perception - 19242

Robotic systems are increasingly used for inspection, maintenance, and disaster assistance for nuclear power plants. These robots utilize increasingly high levels of autonomy to operate in environments that are unsafe for humans to operate the robot in or require more precise control over the robot than a human can offer. In these cases, the robot must include perception and computation components to autonomously sense and navigate through the environment to fulfill its mission. This difficult task is complicated by radiation interfering with sensors' operation. The impact of this interference is largely unexplored, as existing works on radiation interference on sensors focus on estimating the incident radiation, rather than how it impacts a robotic system using the sensor. In this work, we seek to quantify and describe the impact of radiation on cameras, and how the radiation affects robotic performance. In this paper we study cameras that are suitable for small, lightweight unmanned aerial vehicles (UAVs) that are increasingly deployed to fly in confined areas in emergencies, where the surrounding environment is unsafe for workers to operate. These small UAVs lack the payload capacity to carry significant radiation shielding, so any onboard sensors will be affected by radiation. In the case of cameras, this radiation appears as noise in the camera imagery. This noise will interfere with the UAV's operation, by obscuring obstacles and interfering with localization and optical flow algorithms that detect detailed features in the environment. To date, there is a lack of information on how the radiation noise in the camera image impacts the overall robot system performance Existing work focuses on estimating the amount of incident radiation by measuring the corruption in the images. However, these works suffer several shortcomings for robotic systems. First, a robot must be able to successfully navigate its environment during a successful mission, and merely quantifying the amount of radiation noise in an image does not adequately explain how it may interfere with higher level navigation and perception tasks. Second, radiation estimation procedures rely on focusing the camera on a known and static background. This is unsuitable for robotic exploration tasks, where the robot frequently moves around an unknown environment that may be dynamically changing. Our work contains two contributions that address these shortcomings. First, we conduct a detailed analysis of the noise radiation causes in cameras by exposing cameras that are commonly used on UAVs [e.g., small form factor complementary metal-oxide-semiconductor (CMOS) cameras] to varying levels of radiation that may be present in emergency scenarios. From this exposure, we quantify not only how much noise is present, but quantify its appearance. By quantifying the appearance and amount of noise in a camera image, researchers can design mitigation strategies for the corruption, so that their robotic systems can operate in these challenging environments. Second, we demonstrate how to leverage this experimental data in robotic simulations. Using data from the first set of experiments allows us to test and verify the robots in simulation. Using simulators allows us to verify the robots will perform as expected during critical times without needing expensive test equipment or damaging valuable robot prototypes. Other researchers using robots for exploration of irradiated environments will benefit from our research by incorporating similar noise models in their research. 1 Southwest Research Institute{sup R} is a registered trademark of Southwest Research Institute in the United States and/or other countries. This will increase the reliability of robotic systems prior to deployment in emergency situations, which will in turn increase their effectiveness and speed the development of robots that would otherwise need expensive and time-consuming test procedures to validate. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Transactive Energy System Deployment Over Insecure Communication Links

Here, in this paper, the privacy and security issues associated with the transactive energy system (TES) deployment over insecure communication links are addressed. In particular, it is ensured that 1) individual agents’ bidding information is kept private throughout hierarchical market-based interactions; and 2) any extraneous data injection attack can be quickly and easily detected. An implementation framework is proposed to enable the cryptography-based enhancement of privacy and security for the deployment of any general hierarchical systems including TESs. Under the proposed framework, a unified cryptography-based approach is developed to achieve both privacy and security simultaneously. Specifically, privacy preservation is realized by an enhanced Paillier encryption scheme, where a block design is proposed to significantly improve computational efficiency. Attack detection is further achieved by an enhanced Paillier digital signature scheme, where a stamp-concatenation mechanism is proposed to enable detection of data replace and reorder attacks. Simulation results verify the effectiveness of the proposed cyber-resilient design for transactive energy systems. Note to Practitioners—This paper is motivated by addressing the issues of cyber resiliency for practically deploying transactive energy system (TES) but it is also applicable to the problem of enhancing the privacy and security for any general hierarchical control systems. TES is an emerging control approach that engages energy suppliers and customers through market operations and uses the price to optimally allocate energy resources. While it has been shown to be promising for power system applications, the underlying market-based interactions raise significant concerns of privacy (data leakage) and security (data tampering). However, existing TES works only focus on the coordination mechanism instead of privacy and security issues. This paper proposes a new cryptography-based TES design for practical deployment. Specifically, to protect privacy, individual supply and demand amounts to be exchanged are all encrypted in a particular way such that the original amounts cannot be inferred from the encrypted amounts, while the desired computation for setting the market clearing price can be carried out over the encrypted amounts, thus generating an encrypted result which, when decrypted, matches that of the same computation over the original amounts. To achieve security, for each exchanged data, its sender generates a particular digital signature which is exchanged together with the data. This enables the receiver to automatically detect the integrity by checking whether a mathematical relationship holds for the pair of data and signature. In our future research, we will investigate more challenging scenarios where some suppliers and customers themselves could be corrupted and purposely submit distorted amounts.

97 MATHEMATICS AND COMPUTING↗

A unifying Bayesian framework for merging X-ray diffraction data

Novel X-ray methods are transforming the study of the functional dynamics of biomolecules. Key to this revolution is detection of often subtle conformational changes from diffraction data. Diffraction data contain patterns of bright spots known as reflections. To compute the electron density of a molecule, the intensity of each reflection must be estimated, and redundant observations reduced to consensus intensities. Systematic effects, however, lead to the measurement of equivalent reflections on different scales, corrupting observation of changes in electron density. Here, we present a modern Bayesian solution to this problem, which uses deep learning and variational inference to simultaneously rescale and merge reflection observations. We successfully apply this method to monochromatic and polychromatic single-crystal diffraction data, as well as serial femtosecond crystallography data. We find that this approach is applicable to the analysis of many types of diffraction experiments, while accurately and sensitively detecting subtle dynamics and anomalous scattering.

59 BASIC BIOLOGICAL SCIENCES↗

Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems

Here, we present a probabilistic deep learning methodology that enables the construction of predictive data-driven surrogates for stochastic systems. Leveraging recent advances in variational inference with implicit distributions, we put forth a statistical inference framework that enables the end-to-end training of surrogate models on paired input–output observations that may be stochastic in nature, originate from different information sources of variable fidelity, or be corrupted by complex noise processes. The resulting surrogates can accommodate high-dimensional inputs and outputs and are able to return predictions with quantified uncertainty. Furthermore, the effectiveness our approach is demonstrated through a series of canonical studies, including the regression of noisy data, multi-fidelity modeling of stochastic processes, and uncertainty propagation in high-dimensional dynamical systems.

97 MATHEMATICS AND COMPUTING↗

FOREGROUND MODEL AND ANTENNA CALIBRATION ERRORS IN THE MEASUREMENT OF THE SKY-AVERAGED λ21 cm SIGNAL AT z∼ 20

The most promising near-term observable of the cosmic dark age prior to widespread reionization (z ∼ 15-200) is the sky-averaged λ21 cm background arising from hydrogen in the intergalactic medium. Though an individual antenna could in principle detect the line signature, data analysis must separate foregrounds that are orders of magnitude brighter than the λ21 cm background (but that are anticipated to vary monotonically and gradually with frequency, e.g., they are considered {sup s}pectrally smooth{sup )}. Using more physically motivated models for foregrounds than in previous studies, we show that the intrinsic spectral smoothness of the foregrounds is likely not a concern, and that data analysis for an ideal antenna should be able to detect the λ21 cm signal after subtracting a ∼fifth-order polynomial in log ν. However, we find that the foreground signal is corrupted by the angular and frequency-dependent response of a real antenna. The frequency dependence complicates modeling of foregrounds commonly based on the assumption of spectral smoothness. Our calculations focus on the Large-aperture Experiment to detect the Dark Age, which combines both radiometric and interferometric measurements. We show that statistical uncertainty remaining after fitting antenna gain patterns to interferometric measurements is not anticipated to compromise extraction of the λ21 cm signal for a range of cosmological models after fitting a seventh-order polynomial to radiometric data. Our results generalize to most efforts to measure the sky-averaged spectrum.

79 ASTRONOMY AND ASTROPHYSICS↗

Bayesian operator inference for data-driven reduced-order modeling

This work proposes a Bayesian inference method for the reduced-order modeling of time-dependent systems. Informed by the structure of the governing equations, the task of learning a reduced-order model from data is posed as a Bayesian inverse problem with Gaussian prior and likelihood. The resulting posterior distribution characterizes the operators defining the reduced-order model, hence the predictions subsequently issued by the reduced-order model are endowed with uncertainty. The statistical moments of these predictions are estimated via a Monte Carlo sampling of the posterior distribution. Since the reduced models are fast to solve, this sampling is computationally efficient. Furthermore, the proposed Bayesian framework provides a statistical interpretation of the regularization term that is present in the deterministic operator inference problem, and the empirical Bayes approach of maximum marginal likelihood suggests a selection algorithm for the regularization hyperparameters. The proposed method is demonstrated on two examples: the compressible Euler equations with noise-corrupted observations, and a single-injector combustion process.

97 MATHEMATICS AND COMPUTING↗

Determining biomolecular structures near room temperature using X-ray crystallography: concepts, methods and future optimization

For roughly two decades, cryocrystallography has been the overwhelmingly dominant method for determining high-resolution biomolecular structures. Competition from single-particle cryo-electron microscopy and micro-electron diffraction, increased interest in functionally relevant information that may be missing or corrupted in structures determined at cryogenic temperature, and interest in time-resolved studies of the biomolecular response to chemical and optical stimuli have driven renewed interest in data collection at room temperature and, more generally, at temperatures from the protein–solvent glass transition near 200 K to ∼350 K. Fischer has recently reviewed practical methods for room-temperature data collection and analysis [Fischer (2021), Q. Rev. Biophys. 54 , e1]. Here, the key advantages and physical principles of, and methods for, crystallographic data collection at noncryogenic temperatures and some factors relevant to interpreting the resulting data are discussed. For room-temperature data collection to realize its potential within the structural biology toolkit, streamlined and standardized methods for delivering crystals prepared in the home laboratory to the synchrotron and for automated handling and data collection, similar to those for cryocrystallography, should be implemented.

59 BASIC BIOLOGICAL SCIENCES↗