Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A comparison of effective field theory models of redshift space galaxy power spectra for DESI 2024 and future surveys

In preparation for the next generation of galaxy redshift surveys, and in particular the year-one data release from the Dark Energy Spectroscopic Instrument (DESI), we investigate the consistency of a variety of effective field theory models that describe the galaxy-galaxy power spectra in redshift space into the quasi-linear regime using 1-loop perturbation theory. These models are employed in the pipelines velocileptors, PyBird, and Folpsν. While these models have been validated independently, a detailed comparison with consistent choices has not been attempted. After briefly discussing the theoretical differences between the models we describe how to provide a more apples-to-apples comparison between them. We present the results of fitting mock spectra from the AbacusSummit suite of N-body simulations provided in three redshift bins to mimic the types of dark time tracers targeted by the DESI survey. We show that the theories behave similarly and give consistent constraints in both the forward-modeling and ShapeFit compressed fitting approaches. We additionally generate (noiseless) synthetic data from each pipeline to be fit by the others, varying the scale cuts in order to show that the models agree within the range of scales for which we expect 1-loop perturbation theory to be applicable. Finally, this work lays the foundation of Full-Shape analysis with DESI Y1 galaxy samples where in the tests we performed, we found no systematic error associated with the modeling of the galaxy redshift space power spectrum for this volume.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Forward modeling fluctuations in the DESI LRGs target sample using image simulations

We use the forward modeling pipeline, Obiwan, to study the imaging systematics of the Luminous Red Galaxies (LRGs) targeted by the Dark Energy Spectroscopic Instrument (DESI). Imaging systematics refers to the false fluctuation of galaxy densities due to varying observing conditions and astrophysical foregrounds corresponding to the imaging surveys from which DESI LRG target galaxies are selected. We update the Obiwan pipeline, which we previously developed to simulate the optical images used to target DESI data, to further simulate WISE images in the infrared. This addition allows simulating the DESI LRGs sample, which utilizes WISE data in the target selection. Deep DESI imaging data combined with a method to account for biases in their shapes is used to define a truth sample of potential LRG targets. We inject these data evenly throughout the DESI Legacy Imaging Survey footprint at declinations between -30 and 32.375 degrees. We simulate a total of 15 million galaxies to obtain a simulated LRG sample (Obiwan LRGs) that predicts the variations in target density due to imaging properties. We find that the simulations predict the trends with depth observed in the data, including how they depend on the intrinsic brightness of the galaxies. We observe that faint LRGs are the main contributing source of the imaging systematics trend induced by depth. We also find significant trends in the data against Galactic extinction that are not predicted by Obiwan. These trends depend strongly on the particular map of Galactic extinction chosen to test against, implying systematic contamination in the Galactic extinction maps is a likely root cause (e.g., Cosmic-Infrared Background, dust temperature correction). We additionally observe a morphological change of the DESI LRGs population evidenced by a correlation between OII emission line average intensity and the size of the z-band PSF. This effect most likely results from uncertainties in background subtraction. The detailed findings we present should be used to guide any observational systematics mitigation treatment for the clustering of the DESI LRGs sample.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Window convolution of the galaxy clustering bispectrum

In galaxy survey analysis, the observed clustering statistics do not directly match theoretical predictions but rather have been processed by a window function that arises from the survey geometry including the sky footprint, redshift-dependent background number density and systematic weights. While window convolution of the power spectrum is well studied, for the bispectrum with a larger number of degrees of freedom, it poses a significant numerical and computational challenge. In this work, we consider the effect of the survey window in the tripolar spherical harmonic decomposition of the bispectrum and lay down a formal procedure for their convolution via a series expansion of configuration-space three-point correlation functions, which was first proposed by Sugiyama et al. (2019). We then provide a linear algebra formulation of the full window convolution, where an unwindowed bispectrum model vector can be directly premultiplied by a window matrix specific to each survey geometry. To validate the pipeline, we focus on the Dark Energy Spectroscopic Instrument (DESI) Data Release 1 (DR1) luminous red galaxy (LRG) sample in the South Galactic Cap (SGC) in the redshift bin 0.4 ≤ z ≤ 0.6. We first perform convergence checks on the measurement of the window function from discrete random catalogues, and then investigate the convergence of the window convolution series expansion truncated at a finite of number of terms as well as the performance of the window matrix. This work highlights the differences in window convolution between the power spectrum and bispectrum, and provides a streamlined pipeline for the latter for current surveys such as DESI and the Euclid mission.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

DESI DR1 Lyα 1D power spectrum: the optimal estimator measurement

The one-dimensional power spectrum P 1D of Lyα forest offers rich insights into cosmological and astrophysical parameters, including constraints on the sum of neutrino masses, warm dark matter models, and the thermal state of the intergalactic medium. We present the measurement of P 1D using the optimal quadratic maximum likelihood estimator applied to over 300,000 Lyα quasars from Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) survey. This sample represents the largest to date for P 1D measurements and is larger than the Extended Baryon Oscillation Spectroscopic Survey (eBOSS) by a factor of 1.7. We conduct a meticulous investigation of instrumental and analysis systematics and quantify their impact on P 1D . This includes the development of a cross-exposure estimator that eliminates the need to model the pipeline noise and has strong potential for future P 1D measurements. We also present new insights into metal contamination through the 1D correlation function. Using a fitting function we measure the evolution of the Lyα forest bias with high precision: b F (z) = (-0.218 ± 0.002) × ((1 + z)/4) 2.96±0.06 . In a companion validation paper, we substantially extend our previous suite of CCD image simulations to quantify the pipeline's exquisite performance accurately. In another companion paper, we present DR1 P 1D measurements using the Fast Fourier Transform (FFT) approach to power spectrum estimation. These two measurements produce a forest bias parameter that differs by 2.2 sigma. However, our model is simplistic, so this disagreement will be investigated in future work.

Lyman alpha forest↗
DOE OSTI2026

DESI DR1 Ly α 1D power spectrum: Validation of estimators

The Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) is the largest sample to date for small-scale Lyα forest cosmology, accessed through its one-dimensional power spectrum (P 1D ). The Lyα forest P 1D is extracted from quasar spectra that are highly inhomogeneous (both in wavelength and between quasars) in noise properties due to intrinsic properties of the quasar, atmospheric and astrophysical contamination, and also sensitive to low-level details of the spectral extraction pipeline. We employ two estimators in DR1 analysis to measure P 1D : the optimal estimator and the fast Fourier transform (FFT) estimator. To ensure robustness of our DR1 measurements, we validate these two power spectrum and covariance matrix estimation methodologies against the challenging aspects of the data. First, using a set of 20 synthetic 1D realizations of DR1, we derive the masking bias corrections needed for the FFT estimator and the continuum fitting bias needed for both estimators. We demonstrate that both estimators, including their covariances, are unbiased with these corrections using the Kolmogorov-Smirnov test. Second, we substantially extend our previous suite of CCD image simulations to include 675,000 quasars, allowing us to accurately quantify the pipeline's performance. This set of simulations reveals biases at the highest k values, corresponding to a resolution error of a few percent. We base the resolution systematics error budget of DR1 P 1D on these values, but do not derive corrections from them since the simulation fidelity is insufficient for precise corrections.

Lyman alpha forest↗
DOE OSTI2022

A Fast Radio Burst Backend for the Tianlai Dish Pathfinder Array

The Tianlai Dish Pathfinder Array is a radio interferometer array consisting of 16 six-meter dish antennas. The original digital backend integration time is at the seconds level, designed for H I intensity mapping experiment. A new digital backend with millisecond response is added to enable it to search for fast radio burst during its observations. The design and calibration of this backend, and the real time search pipeline for it are described in this paper. It is capable of forming 16 digital beams for each linear polarization, covering an area of 19.6 square degrees. The search pipeline is capable of searching for, recording and classifying FRBs automatically in real time. Finally, in commissioning, we succeeded in capturing the signal pulses from the pulsars PSR B0329+54 and B2021+51.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

DeepAdversaries: examining the robustness of deep learning models for galaxy morphology classification

With increased adoption of supervised deep learning methods for work with cosmological survey data, the assessment of data perturbation effects (that can naturally occur in the data processing and analysis pipelines) and the development of methods that increase model robustness are increasingly important. In the context of morphological classification of galaxies, we study the effects of perturbations in imaging data. In particular, we examine the consequences of using neural networks when training on baseline data and testing on perturbed data. We consider perturbations associated with two primary sources: (a) increased observational noise as represented by higher levels of Poisson noise and (b) data processing noise incurred by steps such as image compression or telescope errors as represented by one-pixel adversarial attacks. We also test the efficacy of domain adaptation techniques in mitigating the perturbation-driven errors. We use classification accuracy, latent space visualizations, and latent space distance to assess model robustness in the face of these perturbations. For deep learning models without domain adaptation, we find that processing pixel-level errors easily flip the classification into an incorrect class and that higher observational noise makes the model trained on low-noise data unable to classify galaxy morphologies. On the other hand, we show that training with domain adaptation improves model robustness and mitigates the effects of these perturbations, improving the classification accuracy up to 23% on data with higher observational noise. Domain adaptation also increases up to a factor of ${\approx}2.3$ the latent space distance between the baseline and the incorrectly classified one-pixel perturbed image, making the model more robust to inadvertent perturbations. Successful development and implementation of methods that increase model robustness in astronomical survey pipelines will help pave the way for many more uses of deep learning for astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Latent Twins

Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.

Latent Twins↗
DOE OSTI2023

Sensitivity Analysis of Genome-Scale Metabolic Flux Prediction

TRIMER, Transcription Regulation Integrated with MEtabolic Regulation, is a genome-scale modeling pipeline targeting at metabolic engineering applications. Using TRIMER, regulated metabolic reactions can be effectively predicted by integrative modeling of metabolic reactions with a Transcription Factor (TF)-gene regulatory network (TRN), which is modeled via a Bayesian network (BN). In this paper, we focus on sensitivity analysis of metabolic flux prediction for uncertainty quantification of BN structures for TRN modeling in TRIMER. We propose a computational strategy to construct the uncertainty class of TRN models based on the inferred regulatory order uncertainty given transcriptomic expression data. With that, we analyze the prediction sensitivity of the TRIMER pipeline for the metabolite yields of interest. The obtained sensitivity analyses can guide Optimal Experimental Design (OED) to help acquire new data that can enhance TRN modeling and achieve specific metabolic engineering objectives, including metabolite yield alterations. Here we have performed small- and large-scale simulated experiments, demonstrating the effectiveness of our developed sensitivity analysis strategy for BN structure learning to quantify the edge importance in terms of metabolic flux prediction uncertainty reduction and its potential to effectively guide OED.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Automated segmentation of soft X-ray tomography: Native cellular structure with submicron resolution at high-throughput for whole-cell quantitative imaging in yeast

Soft X-ray tomography (SXT) is an invaluable tool for quantitatively analyzing cellular structures at suboptical isotropic resolution. However, it has traditionally depended on manual segmentation, limiting its scalability for large datasets. Here, we leverage a deep learning-based autosegmentation pipeline to segment and label cellular structures in hundreds of cells across three Saccharomyces cerevisiae strains. This task-based pipeline uses manual iterative refinement to improve segmentation accuracy for key structures, including the cell body, nucleus, vacuole, and lipid droplets, enabling high-throughput and precise phenotypic analysis. Using this approach, we quantitatively compared the three-dimensional (3D) whole-cell morphometric characteristics of wild-type, VPH1-GFP, and vac14 strains, uncovering detailed strain-specific cell and organelle size and shape variations. We show the utility of SXT data for precise 3D curvature analysis of entire organelles and cells and detection of fine morphological features using surface meshes. Our approach facilitates comparative analyses with high spatial precision and statistical throughput, uncovering subtle morphological features at the single-cell and population level. This workflow significantly enhances our ability to characterize cell anatomy and supports scalable studies on the mesoscale, with applications in investigating cellular architecture, organelle biology, and genetic research across diverse biological contexts.

Chen, Jianhua [Lawrence Berkeley National Laborato↗
DOE OSTI2026

GenomeDepot: data management system for microbial comparative genomics

Summary GenomeDepot is an open-source web-based platform for annotation, management, and comparative analysis of microbial genomic sequences and associated data including ortholog families, protein domains, operons, regulatory interactions, strain taxonomy, and sample metadata. GenomeDepot supports rapid creation of websites for user-defined genome collections that include bioinformatic tools for interactive genome browsing, Basic Local Alignment Search Tool (BLAST) search, annotation search, comparative genomic neighborhood visualization, and sequence download. Gene function annotations are generated by a customizable annotation pipeline. The pipeline runs annotation tools in Conda environments and can be easily extended with additional user-specified tools. Availability and implementation GenomeDepot is open source and distributed under the GNU General Public License via GitHub (https://github.com/aekazakov/genome-depot). GenomeDepot is implemented in Python and was tested in Ubuntu Linux. Full installation instructions and documentation are available at https://aekazakov.github.io/genome-depot/. GenomeDepot demo server is freely accessible at https://iseq.lbl.gov/demogd/.

Kazakov, Alexey [Lawrence Berkeley National Labora↗
DOE OSTI2022

MIDAS2: Metagenomic Intra-species Diversity Analysis System

The Metagenomic Intra-Species Diversity Analysis System (MIDAS) is a scalable metagenomic pipeline that identifies single nucleotide variants (SNVs) and gene copy number variants in microbial populations. Here, we present MIDAS2, which addresses the computational challenges presented by increasingly large reference genome databases, while adding functionality for building custom databases and leveraging paired-end reads to improve SNV accuracy. This fast and scalable reengineering of the MIDAS pipeline enables thousands of metagenomic samples to be efficiently genotyped.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Meta2DB: Curated Shotgun Metagenomic Feature Sets and Metadata for Health State Prediction

Meta2DB is a curated metagenomic and metadata database that provides structurally consistent microbiome taxonomy feature count tables for 13 897 samples across 84 studies, 23 disease states, and 34 geographical locations. All samples were uniformly processed using a streamlined metagenomic classification pipeline that employs a unique and comprehensive reference database indexed to contain all sequences across all kingdoms of life that were present in the NCBI Nucleotide (nt) database retrieved on 4 January 2023. This pipeline leverages high-performance computing (HPC) resources at Lawrence Livermore National Laboratory and was used to process 50TB of publicly available raw metagenomic sequence data. Extensive metadata curation was carried out through a combination of manual curation and automated parsing, producing a consistent inter-study metadata table specifically structured to facilitate training of ML models for prediction of human health.

Kok, C [Lawrence Livermore National Laboratory (LL↗
DOE OSTI2022

A Chromosome-Scale Genome Assembly of a Helicoverpa zea Strain Resistant to Bacillus thuringiensis Cry1Ac Insecticidal Protein

Helicoverpa zea (Lepidoptera: Noctuidae) is an insect pest of major cultivated crops in North and South America. The species has adapted to different host plants and developed resistance to several insecticidal agents, including Bacillus thuringiensis (Bt) insecticidal proteins in transgenic cotton and maize. Helicoverpa zea populations persist year-round in tropical and subtropical regions, but seasonal migrations into temperate zones increase the geographic range of associated crop damage. To better understand the genetic basis of these physiological and ecological characteristics, we generated a high-quality chromosome-level assembly for a single H. zea male from Bt-resistant strain, HzStark_Cry1AcR. Hi-C data were used to scaffold an initial 375.2 Mb contig assembly into 30 autosomes and the Z sex chromosome (scaffold N50 = 12.8 Mb and L50 = 14). The scaffolded assembly was error-corrected with a novel pipeline, polishCLR. The mitochondrial genome was assembled through an improved pipeline and annotated. Assessment of this genome assembly indicated 98.8% of the Lepidopteran Benchmark Universal Single-Copy Ortholog set were complete (98.5% as complete single copy). Repetitive elements comprised approximately 29.5% of the assembly with the plurality (11.2%) classified as retroelements. This chromosome-scale reference assembly for H. zea, ilHelZeax1.1, will facilitate future research to evaluate and enhance sustainable crop production practices.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Plot-level rapid screening for photosynthetic parameters using proximal hyperspectral imaging

Abstract Photosynthesis is currently measured using time-laborious and/or destructive methods which slows research and breeding efforts to identify crop germplasm with higher photosynthetic capacities. We present a plot-level screening tool for quantification of photosynthetic parameters and pigment contents that utilizes hyperspectral reflectance from sunlit leaf pixels collected from a plot (~2 m×2 m) in <1 min. Using field-grown Nicotiana tabacum with genetically altered photosynthetic pathways over two growing seasons (2017 and 2018), we built predictive models for eight photosynthetic parameters and pigment traits. Using partial least squares regression (PLSR) analysis of plot-level sunlit vegetative reflectance pixels from a single visible near infra-red (VNIR) (400–900 nm) hyperspectral camera, we predict maximum carboxylation rate of Rubisco (Vc,max, R2=0.79) maximum electron transport rate in given conditions (J1800, R2=0.59), maximal light-saturated photosynthesis (Pmax, R2=0.54), chlorophyll content (R2=0.87), the Chl a/b ratio (R2=0.63), carbon content (R2=0.47), and nitrogen content (R2=0.49). Model predictions did not improve when using two cameras spanning 400–1800 nm, suggesting a robust, widely applicable and more ‘cost-effective’ pipeline requiring only a single VNIR camera. The analysis pipeline and methods can be used in any cropping system with modified species-specific PLSR analysis to offer a high-throughput field phenotyping screening for germplasm with improved photosynthetic performance in field trials.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Prospects for Galactic and stellar astrophysics with asteroseismology of giant stars in the TESS continuous viewing zones and beyond

ABSTRACT The NASA Transiting Exoplanet Survey Satellite (NASA-TESS) mission presents a treasure trove for understanding the stars it observes and the Milky Way, in which they reside. We present a first look at the prospects for Galactic and stellar astrophysics by performing initial asteroseismic analyses of bright (G < 11) red giant stars in the TESS southern continuous viewing zone (SCVZ). Using three independent pipelines, we detect νmax and Δν in 41 per cent of the 15 405 star parent sample (6388 stars), with consistency at a level of $\sim \! 2{{\ \rm per\ cent}}$ in νmax and $\sim \! 5{{\ \rm per\ cent}}$ in Δν. Based on this, we predict that seismology will be attainable for ∼3 × 105 giants across the whole sky and at least 104 giants with ≥1 yr of observations in the TESS-CVZs, subject to improvements in analysis and data reduction techniques. The best quality TESS-CVZ data, for 5574 stars where pipelines returned consistent results, provide high-quality power spectra across a number of stellar evolutionary states. This makes possible studies of, for example, the asymptotic giant branch bump. Furthermore, we demonstrate that mixed ℓ = 1 modes and rotational splitting are cleanly observed in the 1-yr data set. By combining TESS-CVZ data with TESS-HERMES, SkyMapper, APOGEE, and Gaia, we demonstrate its strong potential for Galactic archaeology studies, providing good age precision and accuracy that reproduces well the age of high [α/Fe] stars and relationships between mass and kinematics from previous studies based on e.g. Kepler. Better quality astrometry and simpler target selection than the Kepler sample makes this data ideal for studies of the local star formation history and evolution of the Galactic disc. These results provide a strong case for detailed spectroscopic follow-up in the CVZs to complement that which has been (or will be) collected by current surveys.

Mackereth, J. Ted↗
DOE OSTI2021

Dark energy survey year 3 results: weak lensing shape catalogue

ABSTRACT We present and characterize the galaxy shape catalogue from the first 3 yr of Dark Energy Survey (DES) observations, over an effective area of 4143 deg2 of the southern sky. We describe our data analysis process and our self-calibrating shear measurement pipeline metacalibration, which builds and improves upon the pipeline used in the DES Year 1 analysis in several aspects. The DES Year 3 weak-lensing shape catalogue consists of 100 204 026 galaxies, measured in the riz bands, resulting in a weighted source number density of neff = 5.59 gal arcmin−2 and corresponding shape noise σe = 0.261. We perform a battery of internal null tests on the catalogue, including tests on systematics related to the point spread function (PSF) modelling, spurious catalogue B-mode signals, catalogue contamination, and galaxy properties.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Nothing to see here: failed supernovae are faint or rare

ABSTRACT The absence of Type IIP core-collapse supernovae arising from progenitors above 17 solar masses suggests the existence of another evolutionary path by which massive stars end their lives. The direct collapse of a stellar core to a black hole without the production of a bright, explosive transient is expected to produce a long-lived, dim, red transient known as a failed supernova. Despite the detection of a number of candidates for disappearing massive stars in recent years, conclusive observational evidence for failed supernovae remains elusive. A custom-built pipeline designed for the detection of faint transients is used to re-analyse 10 yr of observations of 231 nearby galaxies from the PTF/ZTF surveys. This analysis recovers known supernovae, and yields a number of interesting transients. However, none of these are consistent with a failed supernova. Through Monte Carlo tests the recovery efficiency of our pipeline is quantified. By assuming failed supernovae occur as a Poissonian process with zero detections in the data set, 95 per cent upper limits to the rate of failed supernovae are calculated as a function of failed supernova absolute magnitude. We estimate failed supernovae to be less than 0.61, 0.33, 0.26, or 0.23 of the core-collapse SN rate for absolute magnitudes of −11, −12, −13, and −14, respectively. Finally, we show that if they exist, the Vera C. Rubin Observatory will find 1.7–3.7 failed SNe per year for an absolute bolometric luminosity of ∼6 × 1039 erg s−1 out to distances of 33–43 Mpc, depending on their assumed spectral energy distribution.

Byrne, R. A. (ORCID:0000000177661737)↗