Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “inpainting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Physics-informed semantic inpainting: Application to geostatistical modeling

A fundamental problem in geostatistical modeling is to infer the heterogeneous geological field based on limited measurements and some prior spatial statistics. Semantic inpainting, a technique for image processing using deep generative models, has been recently applied for this purpose, demonstrating its effectiveness in dealing with complex spatial patterns. However, the original semantic inpainting framework incorporates only information from direct measurements, while in geostatistics indirect measurements are often plentiful. In this work, to overcome this limitation, we propose a physics-informed semantic inpainting framework, employing the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) and jointly incorporating the direct and indirect measurements by exploiting the underlying physical laws. Our simulation results for a high-dimensional problem with 512 dimensions show that in the new method, the physical conservation laws are satisfied and contribute in enhancing the inpainting performance compared to using only the direct measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Coarse-to-fine Task-driven Inpainting for Geoscience Images

The processing and recognition of geoscience images have wide applications. Most of existing researches focus on understanding the high-quality geoscience images by assuming that all the images are clear. However, in many real-world cases, the geoscience images might contain occlusions during the image acquisition. This problem actually implies the image inpainting problem in computer vision and multimedia. As far as we know, all the existing image inpainting algorithms learn to repair the occluded regions for a better visualization quality, they are excellent for natural images but not good enough for geoscience images, and they never consider the following geoscience task when developing inpainting methods. Here, this paper aims to repair the occluded regions for a better geoscience task performance and advanced visualization quality simultaneously, without changing the current deployed deep learning based geoscience models. Because of the complex context of geoscience images, we propose a coarse-to-fine encoder-decoder network with the help of designed coarse-to-fine adversarial context discriminators to reconstruct the occluded image regions. Due to the limited data of geoscience images, we propose a MaskMix based data augmentation method, which augments inpainting masks instead of augmenting original images, to exploit the limited geoscience image data. The experimental results on three public geoscience datasets for remote sensing scene recognition, cross-view geolocation and semantic segmentation tasks respectively show the effectiveness and accuracy of the proposed method. The code is available at: https://github.com/HMS97/Task-driven-Inpainting.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Crystal structure prediction with host-guided inpainting generation and foundation potentials

Unconditional crystal structure generation with diffusion models faces challenges in identifying symmetric crystals as the unit cell size increases. Here, we present the crystal host-guided generation (CHGGen) framework to address this challenge through conditional generation using an inpainting method, which optimizes a fraction of atomic positions within a predefined and symmetrized host structure to improve the success rate for symmetric structure generation. By integrating inpainting structure generation with a foundation potential for structure optimization, we demonstrate the method on the ZnS–P 2 S 5 and Li–Si chemical systems, where the inpainting method generates a higher fraction of symmetric structures than unconditional generation. The practical significance of CHGGen extends to enabling the structural modification of crystal structures, particularly for systems with partial occupancy or intercalation chemistry. The inpainting method also allows for seamless integration with other generative models, providing a versatile framework for accelerating materials discovery.

Zhong, Peichen [University of California, Berkeley↗
DOE OSTI2014

SU-E-I-08: An Inpaint-Based Interpolation Technique to Recover Blocked Information for Cone Beam CT with a Synchronized Moving Grid (SMOG)

Purpose: Synchronized moving grid (SMOG) is a promising technique to reduce scatter and ghost artifacts in cone beam computed tomography (CBCT). However, the grid blocks part of image information in each projection, and multiple projections at the same gantry angle have to been taken to obtain full information. Because of the continuity of a patient's anatomy in the projection, the blocked information may be estimated by interpolation. This study aims to evaluate an inpainting-based interpolation approach to recover the missing information for CBCT reconstruction. Method: We used a simple region-based inpainting approach to interpolate the missing information. For a pixel to be interpolated, we divided the nearby regions having image information into 6 sub-regions: up-left, up-middle, up-right, down-left, down-middle, and down-right, each with 9 pixels. The average pixel value of each sub-region was calculated. These average values, along with the pixel location, were used to determine the interpolated pixel value. We compared our approach with the Criminisi Exemplar (CE) and total variation (TV) based inpainting techniques. Projection images of Catphan and a head phantom were used for the comparison. The SMOG was simulated by erasing the information (filling with “0”) of the areas in each projection corresponding to the grid. Results: For the Catphan, the processing time was 178, 45 and 0.98 minutes for CE, TV and our approach, respectively. The signal to noise ratio (SNR) was 19.4, 18.5 and 26.4 db, correspondingly. For the head phantom, the processing time was 222, 45 and 0.93 minutes for CE, TV and our approach, respectively. The SNR was 24.6, 20.2 and 26.2db correspondingly. Conclusion: We have developed a simple inpainting based interpolation approach, which can recover some of the image information for the SMOG-based CBCT imaging. This study is supported by NIH/NCI grant 1R01CA166948-01.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2025

Introducing a multiscale feature integration network for inpainting with applications to enhanced CMB map reconstruction

We introduce a novel neural network, SkyReconNet, which combines the expanded receptive fields of dilated convolutional layers along with standard convolutions, to capture both the global and local features for reconstructing the missing information in an image. We implement our network to inpaint the masked regions in a full-sky cosmic microwave background (CMB) map. Inpainting CMB maps is a particularly formidable challenge when dealing with extensive and irregular masks, such as galactic masks which can obscure substantial fractions of the sky. The hybrid design of SkyReconNet leverages the strengths of standard and dilated convolutions to accurately predict CMB fluctuations in the masked regions by effectively utilizing the information from surrounding unmasked areas. During training, the network optimizes its weights by minimizing a composite loss function that combines the structural similarity index measure (SSIM) and mean squared error (MSE). SSIM preserves the essential structural features of the CMB, ensuring an accurate and coherent reconstruction of the missing CMB fluctuations, while MSE minimizes the pixelwise deviations, thus enhancing the overall accuracy of the predictions. The predicted CMB maps and their corresponding angular power spectra align closely with the targets, achieving the performance limited only by the fundamental uncertainty of cosmic variance. The network’s generic architecture enables application to other physics-based challenges involving data with missing or defective pixels, systematic artifacts, etc. In conclusion, our results demonstrate its effectiveness in addressing the challenges posed by large irregular masks, offering a significant inpainting tool not only for CMB analyses but also for image-based experiments across disciplines where such data imperfections are prevalent.

Cosmic microwave background↗
DOE OSTI2023

Load Profile Inpainting for Missing Load Data Restoration and Baseline Estimation

This paper introduces a Generative Adversarial Nets (GAN) based, Load Profile Inpainting Network (Load-PIN) for restoring missing load data segments and estimating the baseline for a demand response event. The inputs are time series load data before and after the inpainting period together with explanatory variables (e.g., weather data). Here, we propose a Generator structure consisting of a coarse network and a fine-tuning network. The coarse network provides an initial estimation of the data segment in the inpainting period. The fine-tuning network consists of self-attention blocks and gated convolution layers for adjusting the initial estimations. Loss functions are specially designed for the fine-tuning and the discriminator networks to enhance both the point-to-point accuracy and realisticness of the results. We test the Load-PIN on three real-world data sets for two applications: patching missing data and deriving baselines of conservation voltage reduction (CVR) events. We benchmark the performance of Load-PIN with five existing deep-learning methods. Our simulation results show that, compared with the state-of-the-art methods, Load-PIN can handle varying-length missing data events and achieve 15-30% accuracy improvement.

14 SOLAR ENERGY↗
DOE OSTI2021

Inpainting radar missing data regions with deep learning

Abstract. Missing and low-quality data regions are a frequent problem for weather radars. They stem from a variety of sources: beam blockage, instrument failure, near-ground blind zones, and many others. Filling in missing data regions is often useful for estimating local atmospheric properties and the application of high-level data processing schemes without the need for preprocessing and error-handling steps – feature detection and tracking, for instance. Interpolation schemes are typically used for this task, though they tend to produce unrealistically spatially smoothed results that are not representative of the atmospheric turbulence and variability that are usually resolved by weather radars. Recently, generative adversarial networks (GANs) have achieved impressive results in the area of photo inpainting. Here, they are demonstrated as a tool for infilling radar missing data regions. These neural networks are capable of extending large-scale cloud and precipitation features that border missing data regions into the regions while hallucinating plausible small-scale variability. In other words, they can inpaint missing data with accurate large-scale features and plausible local small-scale features. This method is demonstrated on a scanning C-band and vertically pointing Ka-band radar that were deployed as part of the Cloud Aerosol and Complex Terrain Interactions (CACTI) field campaign. Three missing data scenarios are explored: infilling low-level blind zones and short outage periods for the Ka-band radar and infilling beam blockage areas for the C-band radar. Two deep-learning-based approaches are tested, a convolutional neural network (CNN) and a GAN that optimize pixel-level error or combined pixel-level error and adversarial loss respectively. Both deep-learning approaches significantly outperform traditional inpainting schemes under several pixel-level and perceptual quality metrics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering

The implementation is proposed of image inpainting techniques for the reconstruction of gaps in experimental X-ray scattering data. The proposed methods use deep learning neural network architectures, such as convolutional autoencoders, tunable U-Nets, partial convolution neural networks and mixed-scale dense networks, to reconstruct the missing information in experimental scattering images. In particular, the recovered pixel intensities are evaluated against their corresponding ground-truth values using the mean absolute error and the correlation coefficient metrics. The results demonstrate that the proposed methods achieve better performance than traditional inpainting algorithms such as biharmonic functions. Overall, tunable U-Net and mixed-scale dense network architectures achieved the best reconstruction performance among all the tested algorithms, with correlation coefficient scores greater than 0.9980.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Nonlocal Feature-Driven Exemplar-Based Approach for Image Inpainting

Here, we present a nonlocal variational image completion technique which admits simultaneous inpainting of multiple structures and textures in a unified framework. The recovery of geometric structures is achieved by using general convolution operators as a measure of behavior within an image. These are combined with a nonlocal exemplar-based approach to exploit the self-similarity of an image in the selected feature domains and to ensure the inpainting of textures. We also introduce an anisotropic patch distance metric to allow for better control of the feature selection within an image and present a nonlocal energy functional based on this metric. Finally, we derive an optimization algorithm for the proposed variational model and examine its validity experimentally with various test images.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Cryo-STEM Tomography with Inpainting

Traditionally, microscopists have worked with the Nyquist-Shannon theory of sampling, which states that to be able to reconstruct the image fully it needs to be sampled at a rate of at least twice the highest spatial frequency in the image. This sampling rate assumes that the image is sampled at regular intervals and that every pixel contains information that is crucial for the image (it even assumes that noise is important). Images in general, and especially low dose S/TEM images, contain significantly less information than can be encoded by a grid of pixels (which is why image compression works). Mathematically speaking, the image data has a low dimensional or sparse representation. Through the application of compressive sensing methods [1,2,3] this representation can be found using pre-designed measurements that are usually random for implementation simplicity. These measurements and the compressive sensing reconstruction algorithms have the added benefit of reducing noise. This reconstruction approach can be extended into higher dimensions, whereby the random sampling in each 2-D image can be extended into: a sequence of tomographic projections (i.e. tilt images); a sequence of video frames (i.e. incorporating temporal resolution and dynamics); spectral resolution (i.e. energy filtering an image to see the distribution of elements); and ptychography (i.e. sampling a full diffraction image at each location in a 2-D grid across the sample). This approach has been employed experimentally for materials science samples requiring low-dose imaging [2], and can be readily applied to biological samples. Figure 1 shows the resolution possible in a complex biological system, mouse pancreatic islet beta cells [4], when tomogram slices are reconstructed using subsampling. Reducing the number of pixels (1/6 pix and 1/3*1/3) shows minimal degradation compared to the reconstructions using all pixels (all data and 1/3 tilt). Although subsampling 1/6 of the tilts (1/6 of overall dose) degrades the reconstruction to the point that the cellular structures cannot be identified. Using 1/3 of both the pixels and the tilts provides a high quality image at 1/9 the overall dose even for this most basic and rapid demonstration of the CS methods. Figure 2 demonstrates the theoretical tomogram reconstruction quality (vertical axis) as undersampling (horizontal axis) is increased; we examined subsampling pixels and tilt-angles individually and a combined approach in which both pixels and tilts are sub-sampled. Note that subsampling pixels maintains high quality reconstructions (solid lines). Using the inpainting algorithm to obtain tomograms can automatically reduce the dose applied to the system by an order of magnitude. Perhaps the best way to understand the impact is to consider that by using inpainting (and with minimal hardware changes), a sample that can normally withstand a dose of ~10 e/Å2 can potentially be imaged with an “equivalent quality” to a dose level of 103 e/Å2. To put this in perspective, this is approaching the dose level used for the most advanced images, in terms of spatial resolution, for inorganic systems. While there are issues for biological specimens beyond dose (structural complexity being the most important one), this sampling approach allows the methods that are traditionally used for materials science to be applied to biological systems [5]. References: [1] A Stevens, H Yang, L Carin et al. Microscopy 63(1), (2014), pp. 41. [2] L Kovarik, A Stevens, A Liyu et al. Appl. Phys. Lett. 109, 164102 (2016) [3] A Stevens, L Kovarik, P Abellan et al. Adv. Structural and Chemical Imaging 1(10), (2015), pp. 1. [4] MD Guay, W Czaja, MA Aronova et al. Scientific Reports 6, 27614 (2016) [5] Supported by the Chemical Imaging, Signature Discovery, and Analytics in Motion Initiatives at PNNL. PNNL is operated by Battelle Memorial Inst. for the US DOE; contract DE-AC05-76RL01830.

Stevens, Andrew↗
DOE OSTI2020

Inpainting Galactic Foreground Intensity and Polarization Maps Using Convolutional Neural Networks

The Deep Convolutional Neural Networks (DCNNs) have been a popular tool for image generation and restoration. In this work, we applied DCNNs to the problem of inpainting non-Gaussian astrophysical signal, in the context of Galactic diffuse emissions at the millimetric and submillimetric regimes, specifically Synchrotron and Thermal Dust emissions. Both signals are affected by contamination at small angular scales due to extragalactic radio sources (the former) and dusty star-forming galaxies (the latter). Furthermore, we compare the performance of the standard diffusive inpainting with that of two novel methodologies relying on DCNNs, namely Generative Adversarial Networks and Deep-Prior. We show that the methods based on the DCNNs are able to reproduce the statistical properties of the ground-truth signal more consistently with a higher confidence level.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Modified algorithms for image inpainting in Fourier transform domain

One of the aims of image inpainting is recovering an image some of which Fourier transform coefficients are lost. In this area, the algorithm of iterative coupled transform domain (ICTDI) has been given by Li and Zeng (SIAM J Imaging Sci 9:24–51, 2016). In this paper, we present some modified algorithms of ICTDI and prove their convergence. In fact, we consider the effect of spectrum and phase angle of the Fourier transform, separately. Therefore, in comparison with ICTDI, one more regularization parameter is generated, and hence, we have more degree of freedom, and therefore, in general, we expect a more appropriate solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

CMB-PAInT: An inpainting tool for the cosmic microwave background

Abstract The presence of astrophysical emissions in microwave observations forces us to perform component separation to extract the Cosmic Microwave Background (CMB) signal. However, even in the most optimistic cases, there are still strongly contaminated regions, such as the Galactic plane or those with emission from extragalactic point sources, which require the use of a mask. Since many CMB analyses, especially the ones working in harmonic space, need the whole sky map, it is crucial to develop a reliable inpainting algorithm that replaces the values of the excluded pixels by others statistically compatible with the rest of the sky. This is especially important when working withQandUsky maps in order to obtainE- andB-mode maps which are free fromE-to-Bleakage. In this work we study a method based on Gaussian Constrained Realizations (GCR), that can deal with both intensity and polarization. Several tests have been performed to asses the validation of the method, including the study of the one-dimensional probability distribution function (1-PDF),E- andB-mode map reconstruction, and power spectra estimation. We have considered two scenarios for the input simulation: one case with only CMB signal and a second one including also Planck PR4 semi-realistic noise. Even if we are limited to low resolution maps, N side = 64 ifT,QandUare considered, we believe that this is a useful approach to be applied to future missions such as LiteBIRD, where the target are the largest scales.

Astronomy & Astrophysics↗
DOE OSTI2014

A novel adaptive discrete cosine transform-domain filter for gap-inpainting of high resolution PET scanners

Purpose: Several positron emission tomography (PET) scanners with special detector block arrangements have been developed in recent years to improve the resolution of PET images. However, the discontinuous detector blocks cause gaps in the sinogram. This study proposes an adaptive discrete cosine transform-based (aDCT) filter for gap-inpainting. Methods: The gap-corrupted sinogram was morphologically closed and subsequently converted to the DCT domain. A certain number of the largest coefficients in the DCT spectrum were identified to determine the low-frequency preservation region. The weighting factors for the remaining coefficients were determined by an exponential weighting function. The aDCT filter was constructed and applied to two digital phantoms and a simulated phantom introduced with various levels of noise. Results: For the Shepp-Logan head phantom, the aDCT filter filled the gaps effectively. For the Jaszczak phantom, no secondary artifacts were induced after aDCT filtering. The percent mean square error and mean structure similarity of the aDCT filter were superior to those of the DCT2 filter at all noise levels. For the simulated striatal dopamine innervation study, the aDCT filter recovered the shape of the striatum and restored the striatum to reference activity ratios to the ideal value. Conclusions: The proposed aDCT filter can recover the missing gap data in the sinogram and improve the image quality and quantitative accuracy of PET images.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2024

Development of a Full-Scale Connected U-Net for Reflectivity Inpainting in Spaceborne Radar Blind Zones

CloudSat’s Cloud Profiling Radar is a valuable tool for remotely monitoring high-latitude snowfall, but its ability to observe hydrometeor activity near the Earth’s surface is limited by a radar blind zone caused by ground clutter contamination. This study presents the development of a deeply supervised U-Net-style convolutional neural network to predict cold season reflectivity profiles within the blind zone at two Arctic locations. The network learns to predict the presence and intensity of near-surface hydrometeors by coupling latent features encoded in blind zone-aloft clouds with additional context from collocated atmospheric state variables (i.e., temperature, specific humidity, and wind speed). Results show that the U-Net predictions outperform traditional linear extrapolation methods, with low mean absolute error, a 38% higher Sørensen–Dice coefficient, and vertical reflectivity distributions 60% closer to observed values. The U-Net is also able to detect the presence of near-surface cloud with a critical success index (CSI) of 72% and cases of shallow cumuliform snowfall and virga with 18% higher CSI values compared to linear methods. An explainability analysis shows that reflectivity information throughout the scene, especially at cloud edges and at the 1.2-km blind zone threshold, along with atmospheric state variables near the tropopause, are the most significant contributors to model skill. This surface-trained generative inpainting technique has the potential to enhance current and future remote sensing precipitation missions by providing a better understanding of the nonlinear relationship between blind zone reflectivity values and the surrounding atmospheric state.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Data-driven high-fidelity 2D microstructure reconstruction via non-local patch-based image inpainting

Microstructure reconstruction problems are usually limited to the representation with finitely many number of phases, e.g. binary and ternary. However, images of microstructure obtained through experimental, for example, using microscope, are often represented as a RGB or grayscale image. Because the phase-based representation is discrete, more rigid, and provides less flexibility in modeling the microstructure, as compared to RGB or grayscale image, there is a loss of information in the conversion. In this paper, a microstructure reconstruction method, which produces images at the fidelity of experimental microscopy, i.e. RGB or grayscale image, is proposed without introducing any physics-based microstructure descriptor. Furthermore, the image texture is preserved and the microstructure image is represented with continuous variables (as in RGB or grayscale images), instead of binary or categorical variables, which results in a high-fidelity image of microstructure reconstruction. The advantage of the proposed method is its quality of reconstruction, which can be applied to any other binary or multiphase 2D microstructure. The proposed method can be thought of as a subsampling approach to expand the microstructure dataset, while preserving its image texture. Moreover, the size of the reconstructed image is more flexible, compared to other machine learning microstructure reconstruction method, where the size must be fixed beforehand. In addition, the proposed method is capable of joining the microstructure images taken at different locations to reconstruct a larger microstructure image. A significant advantage of the proposed method is to remedy the data scarcity problem in materials science, where experimental data is scare and hard to obtain. The proposed method can also be applied to generate statistically equivalent microstructures, which has a strong implication in microstructure-related uncertainty quantification applications. The proposed microstructure reconstruction method is demonstrated with the UltraHigh Carbon Steel micrograph DataBase (UHCSDB).

97 MATHEMATICS AND COMPUTING↗