Engineering topics
Rangel, Esteban
Publications and source records attributed to Rangel, Esteban.
2022 AI Testbed Expeditions Report
By exploiting the coherent properties of a light source, coherent diffraction imaging (CDI) is able to obtain the sample image at a nanoscale resolution using the measured diffraction pattern. Bragg Coherent Diffraction Imaging (BCDI) has become valuable for recovering the displacement and strain field of crystals, providing a valuable tool in material science and solid-state physics. X-ray ptychography is another emerging CDI technique that can produce a high-resolution image of the extended sample and has become popular in many research areas (e.g., materials science, biology, electronics, and optics characterization). CDI including BCDI and ptychography has become an established technique in Synchrotron Facilities including the Advanced Photon Source (APS) and will greatly benefit from the 100x coherent flux increase of the upcoming APS Upgrade (APSU). The current image formation process in CDI employs iterative phase retrieval algorithms, which is a time-consuming and computationally expensive process. Especially after APSU, the traditional iterative methods will not be able to match the experimental data acquisition speed. We employ deep learning (DL) approach to replace the iterative approaches, therefore allowing hundreds of times faster recovery of the object. We developed AutoPhaseNN, a DL-based approach which learns to solve the inverse problem without labeled data. Taking 3D BCDI as a representative technique, AutoPhaseNN has been demonstrated to be one hundred times faster than traditional iterative phase retrieval methods while providing comparable image quality. The current network is trained with 64 x 64 x 64 data size, to achieve higher resolution imaging, we will need to scale the network to input and train/infer 3D arrays of size 256 x 256 x 256 (today) and of size 2560x2560x2560 (APSU). However, the scalability of the network is restricted due to the memory-intensive training process. To perform the training for a 256 x 256 x 256 data size, the required memory exceeds the capacity of the current machine. In this project, we explore using Sambanova system to train the network for the direct data inversion for CDI.
Modelling the Lyman-α forest with Eulerian and SPH hydrodynamical methods
ABSTRACT We compare two state-of-the-art numerical codes to study the overall accuracy in modelling the intergalactic medium and reproducing Lyman-α forest observables for DESI and high-resolution data sets. The codes employ different approaches to solving both gravity and modelling the gas hydrodynamics. The first code, Nyx, solves the Poisson equation using the Particle-Mesh (PM) method and the Euler equations using a finite-volume method. The second code, CRK-HACC , uses a Tree-PM method to solve for gravity, and an improved Lagrangian smoothed particle hydrodynamics (SPH) technique, where fluid elements are modelled with particles, to treat the intergalactic gas. We compare the convergence behaviour of the codes in flux statistics as well as the degree to which the codes agree in the converged limit. We find good agreement overall with differences being less than observational uncertainties, and a particularly notable ≲1 per cent agreement in the 1D flux power spectrum. This agreement was achieved by applying a tessellation methodology for reconstructing the density in CRK-HACC instead of using an SPH kernel as is standard practice. We show that use of the SPH kernel can lead to significant and unnecessary biases in flux statistics; this is especially prominent at high redshifts, z ∼ 5, as the Lyman-α forest mostly comes from lower-density regions that are intrinsically poorly sampled by SPH particles.
Farpoint: A High-resolution Cosmology Simulation at the Gigaparsec Scale
Abstract In this paper we introduce the Farpoint simulation, the latest member of the Hardware/Hybrid Accelerated Cosmology Code (HACC) gravity-only simulation family. The domain covers a volume of (1000 h −1 Mpc) 3 and evolves close to two trillion particles, corresponding to a mass resolution of m p ∼ 4.6 × 10 7 h −1 M ⊙ . These specifications enable comprehensive investigations of the galaxy–halo connection, capturing halos down to small masses. Further, the large volume resolves scales typical of modern surveys with good statistical coverage of high-mass halos. The simulation was carried out on the GPU-accelerated system Summit, one of the fastest supercomputers currently available. We provide specifics about the Farpoint run and present an initial set of results. The high mass resolution facilitates precise measurements of important global statistics, such as the halo concentration–mass relation and the correlation function down to small scales. Selected subsets of the simulation data products are publicly available via the HACC Simulation Data Portal.
The Last Journey. II. SMACC—Subhalo Mass-loss Analysis Using Core Catalogs
Here we introduce Subhalo Mass-loss Analysis using Core Catalogs (SMACC). SMACC adds a mass model to substructure merger trees based on halo "core tracking." Our approach avoids the need for running expensive subhalo finding algorithms and instead uses subhalo mass-loss modeling to assign masses to halo cores. We present details of the SMACC methodology and demonstrate its excellent performance in describing halo substructure and its evolution. Validation of the approach is carried out using cosmological simulations at significantly different resolutions. We apply SMACC to the 1.24-trillion-particle Last Journey simulation and construct core catalogs with the additional mass information. These catalogs can be readily used as input to semianalytic models or subhalo abundance matching approaches to determine approximate galaxy distributions, as well as for in-depth studies of small-scale structure evolution.
The Last Journey. I. An Extreme-scale Simulation on the Mira Supercomputer
The Last Journey is a large-volume, gravity-only, cosmological N-body simulation evolving more than 1.24 trillion particles in a periodic box with a side length of 5.025 Gpc. It was implemented using the HACC simulation and analysis framework on the BG/Q system Mira. The cosmological parameters are chosen to be consistent with the results from the Planck satellite. A range of analysis tools have been run in situ to enable a diverse set of science projects and, at the same time, keep the resulting data amount manageable. Analysis outputs have been generated starting at redshift z ~ 10 to allow for construction of synthetic galaxy catalogs using a semianalytic modeling approach in postprocessing. As part of our in situ analysis pipeline, we employ a new method for tracking halo substructures, introducing the concept of subhalo cores. The production of multiwavelength synthetic sky maps is facilitated by generating particle light cones in situ, also beginning at z ~ 10. We provide an overview of the simulation setup and generated data products; a first set of analysis results is presented. A subset of the data is publicly available.
CosmoDC2: A Synthetic Sky Catalog for Dark Energy Science with LSST
This paper introduces cosmoDC2, a large synthetic galaxy catalog designed to support precision dark energy science with the Large Synoptic Survey Telescope (LSST). CosmoDC2 is the starting point for the second data challenge (DC2) carried out by the LSST Dark Energy Science Collaboration (LSST DESC). The catalog is based on a trillion-particle, (4.225 Gpc) 3 box cosmological N-body simulation, the Outer Rim run. It covers 440 deg 2 of sky area to a redshift of z = 3 and matches expected number densities from contemporary surveys to a magnitude depth of 28 in the r band. Each galaxy is characterized by a multitude of galaxy properties including stellar mass, morphology, spectral energy distributions, broadband filter magnitudes, host halo information, and weak lensing shear. The size and complexity of cosmoDC2 requires an efficient catalog generation methodology; our approach is based on a new hybrid technique that combines data-based empirical approaches with semianalytic galaxy modeling. A wide range of observation-based validation tests has been implemented to ensure that cosmoDC2 enables the science goals of the planned LSST DESC DC2 analyses. This paper also represents the official release of the cosmoDC2 data set, including an efficient reader that facilitates interaction with the data.
The Outer Rim Simulation: A Path to Many-core Supercomputers
Herein we describe the Outer Rim cosmological simulation, one of the largest high-resolution N-body simulations performed to date, aimed at promoting science to be carried out with large-scale structure surveys. The simulation covers a volume of (4.225 Gpc) 3 and evolves more than one trillion particles. It was executed on Mira, a BlueGene/Q system at the Argonne Leadership Computing Facility. We discuss some of the computational challenges posed by a system such as Mira, a many-core supercomputer, and how the simulation code, Hardware/Hybrid Accelerated Cosmology Code, has been designed to overcome these challenges. We have carried out a large range of analyses on the simulation data and we report on the results as well as the data products that have been generated. The full data set generated by the simulation totals more than 5 PB of data, making curation and handling of the data a large challenge in itself. The simulation results have been used to generate synthetic catalogs for large-scale structure surveys, including Dark Energy Spectroscopic Instrument and eBOSS, as well as experiments on the cosmic microwave background. A detailed catalog for the data challenges of the Large Synoptic Survey Telescope Dark Energy Science Collaboration has been created as well. We publicly release some of the Outer Rim halo catalogs, downsampled particle information, and lightcone data.
Building Halo Merger Trees from the Q Continuum Simulation
Cosmological N-body simulations rank among the most computationally intensive efforts today. A key challenge is the analysis of structure, substructure, and the merger history for many billions of compact particle clusters, called halos. Effectively representing the merging history of halos is essential for many galaxy formation models used to generate synthetic sky catalogs, an important application of modern cosmological simulations. Generating realistic mock catalogs requires computing the halo formation history from simulations with large volumes and billions of halos over many time steps, taking hundreds of terabytes of analysis data. We present fast parallel algorithms for producing halo merger trees and tracking halo substructure from a single-level, density-based clustering algorithm. Merger trees are created from analyzing the halo-particle membership function in adjacent snapshots, and substructure is identified by tracking the "cores" of merging halos - sets of particles near the halo center. Core tracking is performed after creating merger trees and uses the relationships found during tree construction to associate substructures with hosts. The algorithms are implemented with MPI and evaluated on a Cray XK7 supercomputer using up to 16,384 processes on data from HACC, a modern cosmological simulation framework. We present results for creating merger trees from 101 analysis snapshots taken from the Q Continuum, a large volume, high mass resolution, cosmological simulation evolving half a trillion particles.
Parallel DTFE Surface Density Field Reconstruction
We improve the interpolation accuracy and efficiency of the Delaunay tessellation field estimator (DTFE) for surface density field reconstruction by proposing an algorithm that takes advantage of the adaptive triangular mesh for line-of-sight integration. The costly computation of an intermediate 3D grid is completely avoided by our method and only optimally chosen interpolation points are computed, thus, the overall computational cost is significantly reduced. The algorithm is implemented as a parallel shared-memory kernel for large-scale grid rendered field reconstructions in our distributed-memory framework designed for N-body gravitational lensing simulations in large volumes. We also introduce a load balancing scheme to optimize the efficiency of processing a large number of field reconstructions. Our results show our kernel outperforms existing software packages for volume weighted density field reconstruction, achieving ~ 10x speedup, and our load balancing algorithm gains an additional ~ 3.6x speedup at scales with ~ 16k processes.
Efficient construction of mock catalogs for baryon acoustic oscillation surveys
Precision measurements of the large scale structure of the Universe require large numbers of high fidelity mock catalogs to accurately assess, and account for, the presence of systematic effects. We introduce and test a scheme for generating mock catalogs rapidly using suitably derated N-body simulations. Our aim is to reproduce the large scale structure and the gross properties of dark matter halos with high accuracy, while sacrificing the details of the halo's internal structure. By adjusting global and local time-steps in an N-body code, we demonstrate that we recover halo masses to better than 0.5% and the power spectrum to better than 1% both in real and redshift space for k =1 h Mpc{sup −1}, while requiring a factor of 4 less CPU time. We also calibrate the redshift spacing of outputs required to generate simulated light cones. We find that outputs separated by Δ z =0.05 allow us to interpolate particle positions and velocities to reproduce the real and redshift space power spectra to better than 1% (out to k =1 h Mpc{sup −1}). We apply these ideas to generate a suite of simulations spanning a range of cosmologies, motivated by the Baryon Oscillation Spectroscopic Survey (BOSS) but broadly applicable to future large scale structure surveys including eBOSS and DESI. As an initial demonstration of the utility of such simulations, we calibrate the shift in the baryonic acoustic oscillation peak position as a function of galaxy bias with higher precision than has been possible so far. This paper also serves to document the simulations, which we make publicly available.