Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sky image processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Convolutional neural networks for intra-hour solar forecasting based on sky image sequences

Accurate and timely solar forecasts play an increasingly critical role in power systems. Compared to longer forecasting timescales, very short-term solar forecasting has lagged behind in both research and practice. In this paper, we propose deep convolutional neural networks (CNNs) to provide operational intra-hour (10-minute-ahead to 60-minute-ahead) solar forecasts. We develop two CNN structures inspired by a widely-used CNN architecture. The CNNs are tailored to our solar forecasting regression tasks and rely solely on sky image sequences. Case studies based on six years of data (over 150,000 data points) demonstrate that the best CNN model has forecast skill scores of 20%-39% over the naive persistence of cloudiness benchmark, even at these very short timescales. The CNNs also have consistently superior performance when compared to shallow machine learning models with meteorological predictors, where the improvement averages around 7%. The sensitivity analyses show that the sky image length, resolution, and weather conditions have impacts on the deep learning model accuracy. In our intra-hour problem with specific setups, two sky images with a 10-minute 128 x 128 resolution yield the most accurate forecasts. Current limitations, future work, and deployment challenges and solutions are also discussed.

14 SOLAR ENERGY↗

Automated construction of clear-sky dictionary from all-sky imager data

All-sky imagers (ASIs) have significant promise as scalable sensors for short-term solar irradiance forecasting. Many of the current computational techniques that use ASIs for this purpose rely on collections of clear-sky images indexed by time of day, solar angle, or both, called clear-sky dictionaries (CSDs). These CSDs act as baselines against which images can be compared to locate and classify clouds within the image frame. CSDs are often compiled by hand, where individuals visually inspect collections of images one at a time to find clear-sky images. This process is not scalable, and it is prone to error. This paper proposes an automated, nonparametric alternative that uses the principles of digital image processing to find clear-sky images within a set of images taken over several days. We use ground-truth measurements of the clearness index to assess the performance of our method, and we show that the images it selects accurately correspond to clear-sky images. We also compare our proposal, which is nonparametric, with a state-of-the-art parametric method. The numerical results indicate that the performance of the method proposed here is superior.

14 SOLAR ENERGY↗

Automating sky object classification in astronomical survey images

We describe the application of machine classification techniques to the development of an automated tool for the reduction of a large scientific data set. The 2nd Palomer Observatory Sky Survey is nearly completed. This survey provides comprehensive coverage of the northern celestial hemisphere in the form of photographic plates. The plates are being transformed into digitized images whose quality will probably not be surpassed in the next ten to twenty years. The images are expected to contain on the order of 10(exp 7) galaxies and 10(exp 8) stars. Astronomers wish to determine which of these sky objects belong to various classes of galaxies and stars. The size of this data set precludes manual analysis. Our approach is to develop a software system which integrates the functions of independently developed techniques for image processing and data classification. Digitized sky images are passed through image processing routines to identify sky objects and to extract a set of features for each object. These routines are used to help select a useful set of attributes for classifying sky objects. Then GID3* and O-BTree, two inductive learning techniques, learn classification decision trees from examples. These classifiers will be used to process the rest of the data. This paper gives an overview of the machine learning techniques used, describes the details of our specific application, and reports the initial encouraging results. The results indicate that our approach is well-suited to the problem. The primary benefits of the approach are increased data reduction throughput and consistency of classification. The classification rules which are the product of the inductive learning techniques will form an object, examinable basis for classifying sky objects. A final, not to be underestimated benefit is that astronomers will be freed from the tedium of an intensely visual task to pursue more challenging analysis and interpretation problems based on automatically cataloged data.

Fayyad, Usama M.↗

The Swift-BAT Hard X-ray Transient Monitor

The Burst Alert Telescope (BAT) on the Swift satellite is a large field of view instrument that continually monitors the sky to provide the gamma-ray burst trigger for Swift. An average of more than 70% of the sky is observed on a daily basis. The survey mode data is processed on two sets on time scales: from one minute to one day as part of the transient monitor program, and from one spacecraft pointing (approx.20 minutes) to the full mission duration for the hard X-ray survey program. The transient monitor has recently become public through the web site http:// swift.gsfc.nasa.gov/docs/swift/results/transients/. Sky images are processed to detect astrophysical sources in the 15-50 keV energy band and the detected flux or upper limit is calculated for >100 sources on time scales up to one day. Light curves are updated each time that new BAT data becomes available (approx.10 times daily). In addition, the monitor is sensitive to an outburst from a new or unknown source. Sensitivity as a function of time scale for catalog and unknown sources will be presented. The daily exposure for a typical source is approx.1500-3000 seconds, with a 1-sigma sensitivity of approx.4 mCrab. 90% of the sources are sampled at least every 16 days, but many sources are sampled daily. It is expected that the Swift-BAT transient monitor will become an important resource for the high energy astrophysics community.

Krimm, Hans↗

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

Development of a Random-Forest Cloud-Regime Classification Model Based on Surface Radiation and Cloud Products

Various methods have been developed to characterize cloud type, otherwise referred to as cloud regime. These include manual sky observations, combining radiative and cloud vertical properties observed from satellite, surface-based remote sensing, and digital processing of sky imagers. While each method has inherent advantages and disadvantages, none of these cloud-typing methods actually includes measurements of surface shortwave or longwave radiative fluxes. Here, a method that relies upon detailed, surface-based radiation and cloud measurements and derived data products to train a random-forest machine-learning cloud classification model is introduced. Measurements from five years of data from the ARM Southern Great Plains site were compiled to train and independently evaluate the model classification performance. A cloud-type accuracy of approximately 80% using the random-forest classifier reveals that the model is well suited to predict climatological cloud properties. Furthermore, an analysis of the cloud-type misclassifications is performed. While physical cloud types may be misreported, the shortwave radiative signatures are similar between misclassified cloud types. From this, we assert that the cloud-regime model has the capacity to successfully differentiate clouds with comparable cloud–radiative interactions. Therefore, we conclude that the model can provide useful cloud-property information for fundamental cloud studies, inform renewable energy studies, and be a tool for numerical model evaluation and parameterization improvement, among many other applications.

54 ENVIRONMENTAL SCIENCES↗

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images: Preprint

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

Legacy Survey of Space and Time Data Preview 1: visit_image dataset type

The Legacy Survey of Space and Time Data Preview 1 (DP1) is the first release of data from the NSF-DOE Vera C. Rubin Observatory. It consists of raw and calibrated single-epoch images, co-adds, difference images, detection catalogs, and other derived data products. DP1 is based on 1792 science-grade optical/near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera, LSSTComCam, on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile during the first on-sky commissioning campaign in late 2024. DP1 covers a total of approximately 15 sq. deg. over seven roughly equally-sized non-contiguous fields, each independently observed in six broad photometric bands, ugrizy, spanning a range of stellar densities and latitudes and overlapping with external reference datasets. This dataset is a subset of the full data release consisting of the visit_image dataset type. These are individual processed and calibrated sky images obtained from a single observation with a single filter. This release contains 15,972 datasets of this type.

79 ASTRONOMY AND ASTROPHYSICS↗

Conference on Applications of Digital Image Processing to Astronomy, Pasadena, Calif., August 20-22, 1980, Proceedings

The astronomic applications of non-military digital image processing are covered in this conference volume. Systems like CCD's, interactive data analysis facilities, stellar speckle interferometry, sky flux subsystems, guide star systems and various image processing systems are described. Techniques in photometry including filtering, automatic photometry, and image restoration are examined. Digital spectral analyses of galaxies, supernova remnants, stars and other celestial bodies are discussed together with algorithms developed to calibrate, clean up, enhance, and quantitatively analyze data. The techniques of image processing permit astronomers to make much more efficient use of their data for both subjective and quantitative analyses. Future missions, such as the Space Telescope, representing a vast data base are briefly covered.

Elliott, D. A.↗

Axial ratios and orientations for 100 Galactic globular star clusters

Axial ratios and major-axis orientations determined from image processing of Palomar and SRC Sky Survey material are tabulated for a sample of 100 Galactic globular clusters. From a subsample of 99 clusters, the mean axial ratio is found to be 0.93 + or - 0.01. Given that only 32 percent are flatter than b/a less than 0.9, and 5 percent flatter than 0.8, the clusters are found to be quite spherical. As predicted by King (1961 and 1966), tidal interactions by the Galaxy on the cluster are not the means by which the observed ellipticities are produced. Rather, Galactic absorption effects may have a moderate but not dominant effect on the clusters' observed ellipticities, as suggested by van den Bergh (1984). The alternate mechanisms of cluster rotation or anisotropic internal velocity distributions, or both, are considered to be more likely. The observed distribution of axial ratios is statistically indistinguishable from that of a similar-size sample of rotationally flattened clusters, which have axial-ratio values lying in the range of 0.75 less than b/a less than 0.83, and the major axes of which are oriented randomly in space.

White, Raymond E.↗

Removing imaging systematics from galaxy clustering measurements with Obiwan: application to the SDSS-IV extended Baryon Oscillation Spectroscopic Survey emission-line galaxy sample

This article presents the application of a new tool, Obiwan, which uses image simulations to determine the selection function of a galaxy redshift survey and calculate three-dimensional (3D) clustering statistics. Obiwan relies on a forward model of the process by which images of the night sky are transformed into a 3D large-scale structure catalogue, and offers several advantages over more traditional map-based techniques – such as operating on individual exposures and adopting a maximum likelihood approach. The photometric pipeline automatically detects and models galaxies and then generates a catalogue of such galaxies with detailed information for each one of them, including their location, redshift, and so on. Systematic biases in the imaging data are therefore imparted into the catalogues and must be accounted for in any scientific analysis of their information content. Obiwan simulates this process for samples selected from the Legacy Surveys imaging data. This imaging data will be used to select target samples for the next-generation Dark Energy Spectroscopic Instrument (DESI) experiment. Here, we apply Obiwan to a portion of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey emission-line galaxies (ELGs). Systematic biases in the data are clearly identified and removed. We compare the 3D clustering results to those obtained by the map-based approach applied to the complete eBOSS Data Release 16 (DR16) sample. We find the results are consistent, thereby validating the eBOSS DR16 ELG catalogues, which is used to obtain cosmological results.

79 ASTRONOMY AND ASTROPHYSICS↗

MaNGA galaxy properties – II. A detailed comparison of observed and simulated spiral galaxy scaling relations

ABSTRACT We present a catalogue of dynamical properties for 2368 late-type galaxies from the MaNGA survey. The latter complements the catalogue of photometric properties for the same sample based on deep optical dark energy sky instrument legacy imaging survey (DESI) photometry processed with autoprof . Rotation curves (RCs), extracted by model-fitting H α velocity maps from the MaNGA Data Analysis Pipeline, extend out to 1.4 (1.9) Re for the primary (secondary) MaNGA samples, respectively. The RCs and ancillary MaNGA Pipe3D data products were used to construct various fundamental galaxy scaling relations that are also compared uniformly with similar relations from numerical investigation of a hundred astrophysical object (NIHAO) zoom-in simulations. Simulated NIHAO galaxies were found to broadly reproduce the observed MaNGA galaxy population for $\log (M_*/{\rm {\rm M}_{\odot })\, \gt\, 8.5}$. Some discrepancies remain, such as those pertaining to central stellar densities and the diversity of RCs due to strong feedback schemes. Also presented are spatially resolved scatters for the velocity-size-stellar mass (VRM*) structural relations using MaNGA and NIHAO samples. The scatter for these relations in the galaxian interiors is a consequence of the diversity of inner RC shapes, while scatter in the outskirts is dictated by the large range of stellar surface densities, which is itself driven by sporadic star formation. The detailed, spatially resolved scatter analysis highlights the complex interplay between local and global astrophysical processes and provides a strong constraint to numerical simulations.

Astronomy & Astrophysics↗

MEM application to IRAS CPC images

A method for applying the Maximum Entropy Method (MEM) to Chopped Photometric Channel (CPC) IRAS additional observations is illustrated. The original CPC data suffered from problems with repeatability which MEM is able to cope with by use of a noise image, produced from the results of separate data scans of objects. The process produces images of small areas of sky with circular Gaussian beams of approximately 30 in. full width half maximum resolution at 50 and 100 microns. Comparison is made to previous reconstructions made in the far-infrared as well as morphologies of objects at other wavelengths. Some projects with this dataset are discussed.

Marston, A. P.↗

Legacy Survey of Space and Time Data Preview 1: template_coadd dataset type

The Legacy Survey of Space and Time Data Preview 1 (DP1) is the first release of data from the NSF-DOE Vera C. Rubin Observatory. It consists of raw and calibrated single-epoch images, co-adds, difference images, detection catalogs, and other derived data products. DP1 is based on 1792 science-grade optical/near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera, LSSTComCam, on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile during the first on-sky commissioning campaign in late 2024. DP1 covers a total of approximately 15 sq. deg. over seven roughly equally-sized non-contiguous fields, each independently observed in six broad photometric bands, ugrizy, spanning a range of stellar densities and latitudes and overlapping with external reference datasets. This dataset is a subset of the full data release consisting of the template_coadd dataset type. These are the combination of processed images with the best seeing, for a patch of sky and for each of the six LSST filters. Used to create difference images. This release contains 2,730 datasets of this type.

79 ASTRONOMY AND ASTROPHYSICS↗

Legacy Survey of Space and Time Data Preview 1: deep_coadd dataset type

The Legacy Survey of Space and Time Data Preview 1 (DP1) is the first release of data from the NSF-DOE Vera C. Rubin Observatory. It consists of raw and calibrated single-epoch images, co-adds, difference images, detection catalogs, and other derived data products. DP1 is based on 1792 science-grade optical/near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera, LSSTComCam, on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile during the first on-sky commissioning campaign in late 2024. DP1 covers a total of approximately 15 sq. deg. over seven roughly equally-sized non-contiguous fields, each independently observed in six broad photometric bands, ugrizy, spanning a range of stellar densities and latitudes and overlapping with external reference datasets. This dataset is a subset of the full data release consisting of the deep_coadd dataset type. These are the combination of multiple processed, calibrated, and background- subtracted images, for a patch of sky, for each of the six filters. This release contains 2,644 datasets of this type.

79 ASTRONOMY AND ASTROPHYSICS↗

SKYSURF: Constraints on Zodiacal Light and Extragalactic Background Light through Panchromatic HST All-sky Surface-brightness Measurements. I. Survey Overview and Methods

We give an overview and describe the rationale, methods, and testing of the Hubble Space Telescope (HST) Archival Legacy project "SKYSURF." SKYSURF uses HST's unique capability as an absolute photometer to measure the ∼0.2–1.7 μm sky-surface brightness (sky-SB) from 249,861 WFPC2, ACS, and WFC3 exposures in ∼1400 independent HST fields. SKYSURF's panchromatic data set is designed to constrain the discrete and diffuse UV to near-IR sky components: Zodiacal Light (ZL), Kuiper Belt Objects (KBOs), Diffuse Galactic Light (DGL), and the discrete plus diffuse Extragalactic Background Light (EBL). We outline SKYSURF's methods to: (1) measure sky-SB levels between detected objects; (2) measure the discrete EBL, most of which comes from AB≃17–22 mag galaxies; and (3) estimate how much truly diffuse light may exist. Simulations of HST WFC3/IR images with known sky values and gradients, realistic cosmic ray (CR) distributions, and star plus galaxy counts were processed with nine different algorithms to measure the "Lowest Estimated Sky-SB" (LES) in each image between the discrete objects. The best algorithms recover the LES values within 0.2% when there are no image gradients, and within 0.2%–0.4% when there are 5%–10% gradients. We provide a proof of concept of our methods from the WFC3/IR F125W images, where any residual diffuse light that HST sees in excess of zodiacal model predictions does not depend on the total object flux that each image contains. This enables us to present our first SKYSURF results on diffuse light in Carleton et al.

Hubble Space Telescope↗

Using machine learning techniques to automate sky survey catalog generation

We describe the application of machine classification techniques to the development of an automated tool for the reduction of a large scientific data set. The 2nd Palomar Observatory Sky Survey provides comprehensive photographic coverage of the northern celestial hemisphere. The photographic plates are being digitized into images containing on the order of 10(exp 7) galaxies and 10(exp 8) stars. Since the size of this data set precludes manual analysis and classification of objects, our approach is to develop a software system which integrates independently developed techniques for image processing and data classification. Image processing routines are applied to identify and measure features of sky objects. Selected features are used to determine the classification of each object. GID3* and O-BTree, two inductive learning techniques, are used to automatically learn classification decision trees from examples. We describe the techniques used, the details of our specific application, and the initial encouraging results which indicate that our approach is well-suited to the problem. The benefits of the approach are increased data reduction throughput, consistency of classification, and the automated derivation of classification rules that will form an objective, examinable basis for classifying sky objects. Furthermore, astronomers will be freed from the tedium of an intensely visual task to pursue more challenging analysis and interpretation problems given automatically cataloged data.

Fayyad, Usama M.↗

Mitigating Shear-dependent Object Detection Biases with Metacalibration

M ETACALIBRATION is a new technique for measuring weak gravitational lensing shear that is unbiased for isolated galaxy images. Here we test M ETACALIBRATION with overlapping, or "blended" galaxy images. Using standard M ETACALIBRATION , we find a few percent shear measurement bias for galaxy densities relevant for current surveys, and that this bias increases with increasing galaxy number density. We show that this bias is not due to blending itself, but rather to shear-dependent object detection. If object detection is shear-independent, no deblending of images is needed, in principle. We demonstrate that detection biases are accurately removed when including object detection in the M ETACALIBRATION process, a technique we call M ETADETECTION . This process involves applying an artificial shear to images of small regions of sky and performing detection on the sheared images, as well as measurements that are used to calculate a shear response. We demonstrate that the method can accurately recover weak shear signals even in highly blended scenes. In the M ETACALIBRATION process, the space between objects is sheared coherently, which does not perfectly match the real universe in which some, but not all, galaxy images are sheared coherently. We find that even for the worst case scenario, in which the space between objects is completely unsheared, the resulting shear bias is at most a few tenths of a percent for future surveys. We discuss additional technical challenges that must be met in order to implement M ETADETECTION for real surveys.

79 ASTRONOMY AND ASTROPHYSICS↗