Engineering Papers⌕ Search

Engineering topics

Connolly, Andrew J.

Publications and source records attributed to Connolly, Andrew J..

LSE-163: Data Products Definition Document

This document describes the data products and processing services to be delivered by the NSF-DOE Vera C. Rubin Observatory whilst performing the Legacy Survey of Space and Time (LSST). LSST will deliver three levels of data products and services. Prompt data products are computed and released within 24 hours of observation, and include images, difference images, catalogs of sources and objects detected in difference images, and catalogs of Solar System objects. Their primary purpose is to enable rapid follow-up of time-domain events. Data Release data products are computed during annual processing campaigns, and include well-calibrated single-epoch images, deep coadds, and catalogs of objects, sources, and forced sources, enabling static sky and precision time-domain science. The Science Platform will allow for the creation of User Generated data products and will enable science cases that greatly benefit from co-location of user processing and/or data within the Rubin Observatory Data Access Center. LSST will also devote 10% of observing time to programs with special cadence. Their data products will be created using the same software and hardware as Prompt and Data Release products. All data products will be made available using user-friendly databases and web services.

79 ASTRONOMY AND ASTROPHYSICS↗

Machine Learning and Cosmology

Methods based on machine learning have recently made substantial inroads in many corners of cosmology. Through this process, new computational tools, new perspectives on data collection, model development, analysis, and discovery, as well as new communities and educational pathways have emerged. Despite rapid progress, substantial potential at the intersection of cosmology and machine learning remains untapped. In this white paper, we summarize current and ongoing developments relating to the application of machine learning within cosmology and provide a set of recommendations aimed at maximizing the scientific impact of these burgeoning tools over the coming decade through both technical development as well as the fostering of emerging communities.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DESC DC2 Data Release Note

In preparation for cosmological analyses of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), the LSST Dark Energy Science Collaboration (LSST DESC) has created a 300 deg$^2$ simulated survey as part of an effort called Data Challenge 2 (DC2). The DC2 simulated sky survey, in six optical bands with observations following a reference LSST observing cadence, was processed with the LSST Science Pipelines (19.0.0). In this Note, we describe the public data release of the resulting object catalogs for the coadded images of five years of simulated observations along with associated truth catalogs. We include a brief description of the major features of the available data sets. To enable convenient access to the data products, we have developed a web portal connected to Globus data services. We describe how to access the data and provide example Jupyter Notebooks in Python to aid first interactions with the data. We welcome feedback and questions about the data release via a GitHub repository.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

LDM-151: Data Management Science Pipelines Design

The LSST Science Requirements Document (the LSST SRD) specifies a set of data product guidelines, designed to support science goals envisioned to be enabled by the LSST observing program. Following these guidelines, the details of these data products have been described in the LSST Data Products Definition Document (DPDD), and captured in a formal flow-down from the SRD via the LSST System Requirements (LSR), Observatory System Specifications (OSS), to the Data Management System Requirements (DMSR). The LSST Data Management subsystem's responsibilities include the design, implementation, deployment and execution of software pipelines necessary to generate these data products. This document describes the design of the scientific aspects of those pipelines.

79 ASTRONOMY AND ASTROPHYSICS↗

Learning Spectral Templates for Photometric Redshift Estimation from Broadband Photometry

Estimating redshifts from broadband photometry is often limited by how accurately we can map the colors of galaxies to an underlying spectral template. Current techniques utilize spectrophotometric samples of galaxies or spectra derived from spectral synthesis models. Both of these approaches have their limitations: either the sample sizes are small and often not representative of the diversity of galaxy colors, or the model colors can be biased (often as a function of wavelength), which introduces systematics in the derived redshifts. In this paper, we learn the underlying spectral energy distributions from an ensemble of ∼100 K galaxies with measured redshifts and colors. We show that we are able to reconstruct emission and absorption lines at a significantly higher resolution than the broadband filters used to measure the photometry for a sample of 20 spectral templates. We find that our training algorithm reduces the fraction of outliers in the derived photometric redshifts by up to 28%, bias up to 91%, and scatter up to 25%, when compared to estimates using a standard set of spectral templates. We discuss the current limitations of this approach and its applicability for recovering the underlying properties of galaxies. Our derived templates and the code used to produce these results are publicly available in a dedicated Github repository: https://github.com/dirac-institute/photoz-template-learning.

79 ASTRONOMY AND ASTROPHYSICS↗

Dimensionality Reduction of SDSS Spectra with Variational Autoencoders

High-resolution galaxy spectra contain much information about galactic physics, but the high dimensionality of these spectra makes it difficult to fully utilize the information they contain. We apply variational autoencoders (VAEs), a nonlinear dimensionality reduction technique, to a sample of spectra from the Sloan Digital Sky Survey (SDSS). In contrast to principal component analysis (PCA), a widely used technique, VAEs can capture nonlinear relationships between latent parameters and the data. We find that a VAE can reconstruct the SDSS spectra well with only six latent parameters, outperforming PCA with the same number of components. Different galaxy classes are naturally separated in this latent space, without class labels having been given to the VAE. The VAE latent space is interpretable because the VAE can be used to make synthetic spectra at any point in latent space. For example, making synthetic spectra along tracks in latent space yields sequences of realistic spectra that interpolate between two different types of galaxies. Using the latent space to find outliers may yield interesting spectra: in our small sample, we immediately find unusual data artifacts and stars misclassified as galaxies. In this exploratory work, we show that VAEs create compact, interpretable latent spaces that capture nonlinear features of the data. While a VAE takes substantial time to train (≈1 day for 48,000 spectra), once trained, VAEs can enable the fast exploration of large astronomical data sets.

79 ASTRONOMY AND ASTROPHYSICS↗

Photometric Redshifts with the LSST. II. The Impact of Near-infrared and Near-ultraviolet Photometry

Accurate photometric redshift (photo-z) estimates are essential to the cosmological science goals of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). In this work, we use simulated photometry for mock galaxy catalogs to explore how LSST photo-z estimates can be improved by the addition of near-infrared (NIR) and/or ultraviolet (UV) photometry from the Euclid, Wide-Field InfrarRed Survey Telescope (WFIRST), and/or Cosmological Advanced Survey Telescope for Optical and ultraviolet Research (CASTOR) space telescopes. Generally, we find that deeper optical photometry can reduce the standard deviation of the photo-z estimates more than adding NIR or UV filters but that additional filters are the only way to significantly lower the fraction of galaxies with catastrophically under- or overestimated photo-z. For Euclid, we find that the addition of JH 5σ photometric detections can reduce the standard deviation for galaxies with z > 1 (z > 0.3) by ∼20% (∼10%), and the fraction of outliers by ∼40% (∼25%). For WFIRST, we show how the addition of deep YJHK photometry could reduce the standard deviation by ≳50% at z > 1.5 and drastically reduce the fraction of outliers to just ∼2% overall. For CASTOR, we find that the addition of its UV- and u-band photometry could reduce the standard deviation by ∼30% and the fraction of outliers by ∼50% for galaxies with z < 0.5. We also evaluate the photo-z results within sky areas that overlap with both the NIR and UV surveys and when spectroscopic training sets built from the surveys’ small-area deep fields are used.

79 ASTRONOMY AND ASTROPHYSICS↗

Applying Information Theory to Design Optimal Filters for Photometric Redshifts

In this paper we apply ideas from information theory to create a method for the design of optimal filters for photometric redshift estimation. We show the method applied to a series of simple example filters in order to motivate an intuition for how photometric redshift estimators respond to the properties of photometric passbands. We then design a realistic set of six filters covering optical wavelengths that optimize photometric redshifts for z <= 2.3 and i < 25.3. We create a simulated catalog for these optimal filters and use our filters with a photometric redshift estimation code to show that we can improve the standard deviation of the photometric redshift error by 7.1% overall and improve outliers 9.9% over the standard filters proposed for the Large Synoptic Survey Telescope (LSST). We compare features of our optimal filters to the LSST and find that the LSST filters incorporate key features for optimal photometric redshift estimation. Finally, we describe how information theory can be applied to a range of optimization problems in astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗

Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era

The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt --- e.g., via the onset of artificial intelligence --- which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our technical and collaborative frameworks to promote efficient algorithmic development and take advantage of opportunities for scientific discovery in the petabyte era. We discuss challenges for discovery in large and complex data sets; challenges and requirements for the next stage of development of statistical methodologies and algorithmic tool sets; how we might change our paradigms of collaboration and education; and the ethical implications of scientists' contributions to widely applicable algorithms and computational modeling. We start with six distinct recommendations that are supported by the commentary following them. This white paper is related to a larger corpus of effort that has taken place within and around the Petabytes to Science Workshops (https://petabytestoscience.github.io/).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Sub-band Image Reconstruction Using Differential Chromatic Refraction

Refraction by the atmosphere causes the positions of sources to depend on the airmass through which an observation was taken. This shift is dependent on the underlying spectral energy of the source and the filter or bandpass through which it is observed. Wavelength-dependent refraction within a single passband is often referred to as differential chromatic refraction (DCR). With a new generation of astronomical surveys undertaking repeated observations of the same part of the sky over a range of different airmasses and parallactic angles, DCR should be a detectable and measurable astrometric signal. In this paper we introduce a novel procedure that takes this astrometric signal and uses it to infer the underlying spectral energy distribution of a source; we solve for multiple latent images at specific wavelengths via a generalized deconvolution procedure built on robust statistics. We demonstrate the utility of such an approach for estimating a partially deconvolved image, at higher spectral resolution than the input images, for surveys such as the Large Synoptic Survey Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗

Robust Period Estimation Using Mutual Information for Multiband Light Curves in the Synoptic Survey Era

The Large Synoptic Survey Telescope (LSST) will produce an unprecedented amount of light curves using six optical bands. Robust and efficient methods that can aggregate data from multidimensional sparsely sampled time-series are needed. Here, we present a new method for light curve period estimation based on quadratic mutual information (QMI). The presented method does not assume a particular model for the light curve nor its underlying probability density and it is robust to non-Gaussian noise and outliers. By combining the QMI from several bands the true period can be estimated even when no single-band QMI yields the period. Period recovery performance as a function of average magnitude and sample size is measured using 30,000 synthetic multiband light curves of RR Lyrae and Cepheid variables generated by the LSST Operations and Catalog simulators. The results show that aggregating information from several bands is highly beneficial in LSST sparsely sampled time-series, obtaining an absolute increase in period recovery rate up to 50%. We also show that the QMI is more robust to noise and light curve length (sample size) than the multiband generalizations of the Lomb–Scargle and AoV periodograms, recovering the true period in 10%–30% more cases than its competitors. A python package with efficient Cython implementations of the QMI and other methods is provided.

79 ASTRONOMY AND ASTROPHYSICS↗

Scientific Synergy between LSST and Euclid

We report that Euclid and the Large Synoptic Survey Telescope (LSST) are poised to dramatically change the astronomy landscape early in the next decade. The combination of high-cadence, deep, wide-field optical photometry from LSST with high-resolution, wide-field optical photometry, and near-infrared photometry and spectroscopy from Euclid will be powerful for addressing a wide range of astrophysical questions. We explore Euclid/LSST synergy, ignoring the political issues associated with data access to focus on the scientific, technical, and financial benefits of coordination. We focus primarily on dark energy cosmology, but also discuss galaxy evolution, transient objects, solar system science, and galaxy cluster studies. We concentrate on synergies that require coordination in cadence or survey overlap, or would benefit from pixel-level co-processing that is beyond the scope of what is currently planned, rather than scientific programs that could be accomplished only at the catalog level without coordination in data processing or survey strategies. Also, we provide two quantitative examples of scientific synergies: the decrease in photo-z errors (benefiting many science cases) when high-resolution Euclid data are used for LSST photo-z determination, and the resulting increase in weak-lensing signal-to-noise ratio from smaller photo-z errors. We briefly discuss other areas of coordination, including high-performance computing resources and calibration data. Finally, we address concerns about the loss of independence and potential cross-checks between the two missions and the potential consequences of not collaborating.

79 ASTRONOMY AND ASTROPHYSICS↗

VDES J2325-5229 a z = 2.7 gravitationally lensed quasar discovered using morphology-independent supervised machine learning

In this paper, we present the discovery and preliminary characterization of a gravitationally lensed quasar with a source redshift z s = 2.74 and image separation of 2.9 arcsec lensed by a foreground z l = 0.40 elliptical galaxy. Since optical observations of gravitationally lensed quasars show the lens system as a superposition of multiple point sources and a foreground lensing galaxy, we have developed a morphology-independent multi-wavelength approach to the photometric selection of lensed quasar candidates based on Gaussian Mixture Models (GMM) supervised machine learning. Using this technique and gi multicolour photometric observations from the Dark Energy Survey (DES), near-IR JK photometry from the VISTA Hemisphere Survey (VHS) and WISE mid-IR photometry, we have identified a candidate system with two catalogue components with i AB = 18.61 and i AB = 20.44 comprising an elliptical galaxy and two blue point sources. Spectroscopic follow-up with NTT and the use of an archival AAT spectrum show that the point sources can be identified as a lensed quasar with an emission line redshift of z = 2.739 ± 0.003 and a foreground early-type galaxy with z = 0.400 ± 0.002. We model the system as a single isothermal ellipsoid and find the Einstein radius θ E ~ 1.47 arcsec, enclosed mass M enc ~ 4 × 10 11 M ⊙ and a time delay of ~52 d. Finally, the relatively wide separation, month scale time delay duration and high redshift make this an ideal system for constraining the expansion rate beyond a redshift of 1.

79 ASTRONOMY AND ASTROPHYSICS↗

An optical to IR sky brightness model for the LSST

To optimize the observing strategy of a large survey such as the LSST, one needs an accurate model of the night sky emission spectrum across a range of atmospheric conditions and from the near-UV to the near-IR. We have used the ESO SkyCalc Sky Model Calculator to construct a library of template spectra for the Chilean night sky. The ESO model includes emission from the upper and lower atmosphere, scattered starlight, scattered moonlight, and zodiacal light. We have then extended the ESO templates with an empirical fit to the twilight sky emission as measured by a Canon all-sky camera installed at the LSST site. With the ESO templates and our twilight model we can quickly interpolate to any arbitrary sky position and date and return the full sky spectrum or surface brightness magnitudes in the LSST filter system. Comparing our model to all-sky observations, we find typical residual RMS values of ±0.2-0.3 magnitudes per square arcsecond.

79 ASTRONOMY AND ASTROPHYSICS↗

Determining frequentist confidence limits using a directed parameter space search

We consider the problem of inferring constraints on a high-dimensional parameter space with a computationally expensive likelihood function. We propose a machine learning algorithm that maps out the Frequentist confidence limit on parameter space by intelligently targeting likelihood evaluations so as to quickly and accurately characterize the likelihood surface in both low- and high-likelihood regions. We compare our algorithm to Bayesian credible limits derived by the well-tested Markov Chain Monte Carlo (MCMC) algorithm using both multi-modal toy likelihood functions and the seven yr Wilkinson Microwave Anisotropy Probe cosmic microwave background likelihood function. We find that our algorithm correctly identifies the location, general size, and general shape of high-likelihood regions in parameter space while being more robust against multi-modality than MCMC.

79 ASTRONOMY AND ASTROPHYSICS↗

Variability-based active galactic nucleus selection using image subtraction in the SDSS and LSST era

With upcoming all-sky surveys such as LSST poised to generate a deep digital movie of the optical sky, variability-based active galactic nucleus (AGN) selection will enable the construction of highly complete catalogs with minimum contamination. In this study, we generate g-band difference images and construct light curves (LCs) for QSO/AGN candidates listed in Sloan Digital Sky Survey Stripe 82 public catalogs compiled from different methods, including spectroscopy, optical colors, variability, and X-ray detection. Image differencing excels at identifying variable sources embedded in complex or blended emission regions such as Type II AGNs and other low-luminosity AGNs that may be omitted from traditional photometric or spectroscopic catalogs. To separate QSOs/AGNs from other sources using our difference image LCs, we explore several LC statistics and parameterize optical variability by the characteristic damping timescale (τ) and variability amplitude. By virtue of distinguishable variability parameters of AGNs, we are able to select them with high completeness of 93.4% and efficiency (i.e., purity) of 71.3%. Based on optical variability, we also select highly variable blazar candidates, whose infrared colors are consistent with known blazars. One-third of them are also radio detected. With the X-ray selected AGN candidates, we probe the optical variability of X-ray detected optically extended sources using their difference image LCs for the first time. A combination of optical variability and X-ray detection enables us to select various types of host-dominated AGNs. Contrary to the AGN unification model prediction, two Type II AGN candidates (out of six) show detectable variability on long-term timescales like typical Type I AGNs. This study will provide a baseline for future optical variability studies of extended sources.

79 ASTRONOMY AND ASTROPHYSICS↗

THE MILKY WAY TOMOGRAPHY WITH SLOAN DIGITAL SKY SURVEY. IV. DISSECTING DUST

We use Sloan Digital Sky Survey (SDSS) photometry of 73 million stars to simultaneously constrain best-fit main-sequence stellar spectral energy distribution (SED) and amount of dust extinction along the line of sight toward each star. Using a subsample of 23 million stars with Two Micron All Sky Survey (2MASS) photometry, whose addition enables more robust results, we show that SDSS photometry alone is sufficient to break degeneracies between intrinsic stellar color and dust amount when the shape of extinction curve is fixed. When using both SDSS and 2MASS photometry, the ratio of the total to selective absorption, R{sub V} , can be determined with an uncertainty of about 0.1 for most stars in high-extinction regions. These fits enable detailed studies of the dust properties and its spatial distribution, and of the stellar spatial distribution at low Galactic latitudes (|b| < 30 Degree-Sign ). Our results are in good agreement with the extinction normalization given by the Schlegel et al. (SFD) dust maps at high northern Galactic latitudes, but indicate that the SFD extinction map appears to be consistently overestimated by about 20% in the southern sky, in agreement with recent study by Schlafly et al. The constraints on the shape of the dust extinction curve across the SDSS and 2MASS bandpasses disfavor the reddening law of O'Donnell, but support the models by Fitzpatrick and Cardelli et al. For the latter, we find a ratio of the total to selective absorption to be R{sub V} = 3.0 {+-} 0.1(random){+-}0.1 (systematic) over most of the high-latitude sky. At low Galactic latitudes (|b| < 5 Degree-Sign ), we demonstrate that the SFD map cannot be reliably used to correct for extinction because most stars are embedded in dust, rather than behind it, as is the case at high Galactic latitudes. We analyze three-dimensional maps of the best-fit R{sub V} and find that R{sub V} = 3.1 cannot be ruled out in any of the 10 SEGUE stripes at a precision level of {approx}0.1-0.2. Our best estimate for the intrinsic scatter of R{sub V} in the regions probed by SEGUE stripes is {approx}0.2. We introduce a method for efficient selection of candidate red giant stars in the disk, dubbed 'dusty parallax relation', which utilizes a correlation between distance and the extinction along the line of sight. We make these best-fit parameters, as well as all the input SDSS and 2MASS data, publicly available in a user-friendly format. These data can be used for studies of stellar number density distribution, the distribution of dust properties, for selecting sources whose SED differs from SEDs for high-latitude main-sequence stars, and for estimating distances to dust clouds and, in turn, to molecular gas clouds.

79 ASTRONOMY AND ASTROPHYSICS↗

SPECTROSCOPIC DETERMINATION OF THE LOW-REDSHIFT TYPE Ia SUPERNOVA RATE FROM THE SLOAN DIGITAL SKY SURVEY

Supernova rates (SNRs) are directly coupled to high-mass stellar birth and evolution. As such, they are one of the few direct measures of the history of cosmic stellar evolution. In this paper, we describe a probabilistic technique for identifying supernovae within spectroscopic samples of galaxies. We present a study of 52 Type Ia supernovae ranging in age from -14 days to +40 days extracted from a parent sample of {approx}350,000 spectra from the SDSS DR5. We find an SNR of 0.472{sup +0.048}{sub -0.039}(Systematic){sup +0.081}{sub -0.071}(Statistical)SNu at a redshift of (z) = 0.1. This value is higher than other values at low redshift at the 1{sigma} level, but is consistent at the 3{sigma} level. In this paper, we demonstrate the potential for the described approach to detect supernovae in future spectroscopic surveys.

79 ASTRONOMY AND ASTROPHYSICS↗