Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A hybrid neural architecture: Online attosecond x-ray characterization

The emergence of high-repetition-rate x-ray free-electron lasers (XFELs), such as SLAC’s LCLS-II, serves as our canonical example for autonomous controls that necessitate high-throughput diagnostics paired with streaming computational pipelines capable of single-shot analysis with extremely low latency. We present the deterministic characterization with an integrated parallelizable hybrid resolver architecture, a hybrid machine learning framework designed for fast, accurate analysis of XFEL diagnostics using angular streaking-based sinogram images. This architecture integrates convolutional neural networks and bidirectional long short-term memory models to denoise input, identify x-ray sub-spike features, and extract sub-spike relative delays with sub-30 attosecond temporal resolution. Deployed on low-latency hardware, it achieves over 10 kHz throughput with 168.3 μs inference latency, indicating scalability to 14 kHz with field-programmable gate array integration. By transforming regression tasks into classification problems and leveraging optimized error encoding, we achieve high precision with low-latency performance that is critical for real-time streaming event selection and experimental control feedback signals. This represents a key development in real-time control pipelines for next-generation autonomous science, generally, and high repetition-rate x-ray experiments in particular.

Accelerator Physics (physics.acc-ph)↗
DOE OSTI2026

Selection of a Pair of Experiments to Optimally Reduce Uncertainty in Targeted Nuclear Data

We propose a novel process to select a pair of differential and integral experiments that best reduce uncertainties in targeted 239 ⁢Pu nuclear data while compressing the current nuclear data pipeline from 20 to 3 years. 239⁢ Pu nuclear data are poorly understood for neutrons in the intermediate energy range due to sparsity and uncertainty in historical experiments. New experiments targeting this range will enable better understanding of these nuclear data, but choosing the ideal experiments to conduct is challenging. Beginning with a prior distribution represented by samples of nuclear data generated from theory, generalized least squares adjustments are made to incorporate data from historical experiments. To quantify potential uncertainty reduction obtainable from a pair of candidate experiments, we compute the D-optimality criterion of the posterior covariance of intermediate energy range nuclear data compared to the equivalent covariance after additional adjustment to the pair of candidate experiments. Repeating the process for each of many candidate pairs facilitates the final selection. Results support 63⁢ Cu total cross section measurements for differential experiments and alumina and alumina/graphite configurations for integral experiments. This analysis enables choosing differential and integral experiments to be executed concurrently while shortening decision times relative to the current nuclear data pipeline.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

In situ tumor model for longitudinal in silico imaging trials

Abstract Objective.In this article, we introduce a computational model for simulating the growth of breast cancer lesions accounting for the stiffness of surrounding anatomical structures.Approach.In our model, ligaments are classified as the most rigid structures while the softer parts of the breast are occupied by fat and glandular tissues As a result of these variations in tissue elasticity, the rapidly proliferating tumor cells are met with differential resistance. It is found that these cells are likely to circumvent stiffer terrains such as ligaments, instead electing to proliferate preferentially within the more yielding confines of the breast’s soft topography. By manipulating the interstitial tumor pressure in direct proportion to the elastic constants of the tissues surrounding the tumor, this model thus creates the potential for realizing a database of unique lesion morphology sculpted by the distinctive topography of each local anatomical infrastructure. We modeled the growth of simulated lesions within volumes extracted from fatty breast models, developed by Graffet alwith a resolution of 50μm generated with the open-source and readily available Virtual Imaging Clinical Trials for Regulatory Evaluation (VICTRE) imaging pipeline. To visualize and validate the realism of the lesion models, we leveraged the imaging component of the VICTRE pipeline, which replicates the siemens mammomat inspiration mammography system in a digital format. This system was instrumental in generating digital mammogram (DM) images for each breast model containing the simulated lesions.Results.By utilizing the DM images, we were able to effectively illustrate the imaging characteristics of the lesions as they integrated with the anatomical backgrounds. Our research also involved a reader study that compared 25 simulated DM regions of interest (ROIs) with inserted lesions from our models with DM ROIs from the DDSM dataset containing real manifestations of breast cancer. In general the simulation time for the lesions was approximately 2.5 hours, but it varied depending on the lesion’s local environment.Significance.The lesion growth model will facilitate and enhance longitudinal in silico trials investigating the progression of breast cancer.

Engineering↗
DOE OSTI2021

The persistence of large scale structures. Part I. Primordial non-Gaussianity

Abstract We develop an analysis pipeline for characterizing the topology of large scale structure and extracting cosmological constraints based on persistent homology . Persistent homology is a technique from topological data analysis that quantifies the multiscale topology of a data set, in our context unifying the contributions of clusters, filament loops, and cosmic voids to cosmological constraints. We describe how this method captures the imprint of primordial local non-Gaussianity on the late-time distribution of dark matter halos, using a set of N-body simulations as a proxy for real data analysis. For our best single statistic, running the pipeline on several cubic volumes of size 40 (Gpc/h) 3 , we detect f NL loc =10 at 97.5% confidence on ~ 85% of the volumes. Additionally wetest our ability to resolve degeneracies betweenthe topological signature of f NL loc and variation of σ 8 and argue that correctly identifying nonzero f NL loc in this case is possible via an optimal template method. Our method relies on information living at $\mathcal{O}$(10) Mpc/h, a complementary scale with respect to commonly used methods such as the scale-dependent bias in the halo/galaxy power spectrum. Therefore, while still requiring a large volume, our method does not require sampling long-wavelength modes to constrain primordial non-Gaussianity. Moreover, our statistics are interpretable: we are able to reproduce previous results in certain limits and we make new predictions for unexplored observables, such as filament loops formed by dark matter halos in a simulation box.

Astronomy & Astrophysics↗
DOE OSTI2023

The Atacama Cosmology Telescope: map-based noise simulations for DR6

Abstract The increasing statistical power of cosmic microwave background (CMB) datasets requires a commensurate effort in understanding their noise properties. The noise in maps from ground-based instruments is dominated by large-scale correlations, which poses a modeling challenge. This paper develops novel models of the complex noise covariance structure in the Atacama Cosmology Telescope Data Release 6 (ACT DR6) maps. We first enumerate the noise properties that arise from the combination of the atmosphere and the ACT scan strategy. We then prescribe a class of Gaussian, map-based noise models, including a new wavelet-based approach that uses directional wavelet kernels for modeling correlated instrumental noise. The models are empirical, whose only inputs are a small number of independent realizations of the same region of sky. We evaluate the performance of these models against the ACT DR6 data by drawing ensembles of noise realizations. Applying these simulations to the ACT DR6 power spectrum pipeline reveals a ∼ 20% excess in the covariance matrix diagonal when compared to an analytic expression that assumes noise properties are uniquely described by their power spectrum. Along with our public code,mnms, this work establishes a necessary element in the science pipelines of both ACT DR6 and future ground-based CMB experiments such as the Simons Observatory (SO).

Astronomy & Astrophysics↗
DOE OSTI2024

Optimization of foreground moment deprojection for semi-blind CMB polarization reconstruction

Abstract Upcoming Cosmic Microwave Background (CMB) experiments, aimed at measuring primordial CMB polarization B-modes, require exquisite control of instrumental systematics and Galactic foreground contamination. Blind minimum-variance techniques, like the Needlet Internal Linear Combination (NILC), have proven effective in reconstructing the CMB polarization signal and mitigating foregrounds and systematics across diverse sky models without suffering from foreground mismodelling errors. Still, residual foreground contamination from NILC may bias the recovered CMB polarization at large angular scales when confronted with the most complex foreground scenarios.By adding constraints to NILC to deproject statistical moments of the Galactic emission, the Constrained Moment ILC (cMILC) method has been demonstrated to further enhance foreground subtraction, albeit with an associated increase in overall noise variance. Faced with this trade-off between foreground bias reduction and overall variance minimization, there is still no recipe on which moments to deproject and which are better suited for blind variance minimization. To address this, we introduce the optimized cMILC (ocMILC) pipeline, which performs full automated optimization of the required number and set of foreground moments to deproject, pivot parameter values, and deprojection coefficients across the sky and angular scales, depending on the actual sky complexity, available frequency coverage, and experiment sensitivity. The optimal number of moments for deprojection, before paying significant noise penalty, is determined through a data diagnosis inspired by the Generalized NILC (GNILC) method.Validated on B-mode simulations of thePICOspace mission concept with four challenging foreground models, ocMILC exhibits lower Galactic foreground contamination compared to NILC and cMILC at all angular scales, with limited noise penalty. This multi-layer optimization enables the ocMILC pipeline to achieve unbiased posteriors of the tensor-to-scalar ratio, regardless of foreground complexity.

Astronomy & Astrophysics↗
DOE OSTI2025

A comparison of effective field theory models of redshift space galaxy power spectra for DESI 2024 and future surveys

In preparation for the next generation of galaxy redshift surveys, and in particular the year-one data release from the Dark Energy Spectroscopic Instrument (DESI), we investigate the consistency of a variety of effective field theory models that describe the galaxy-galaxy power spectra in redshift space into the quasi-linear regime using 1-loop perturbation theory. These models are employed in the pipelines velocileptors, PyBird, and Folpsν. While these models have been validated independently, a detailed comparison with consistent choices has not been attempted. After briefly discussing the theoretical differences between the models we describe how to provide a more apples-to-apples comparison between them. We present the results of fitting mock spectra from the AbacusSummit suite of N-body simulations provided in three redshift bins to mimic the types of dark time tracers targeted by the DESI survey. We show that the theories behave similarly and give consistent constraints in both the forward-modeling and ShapeFit compressed fitting approaches. We additionally generate (noiseless) synthetic data from each pipeline to be fit by the others, varying the scale cuts in order to show that the models agree within the range of scales for which we expect 1-loop perturbation theory to be applicable. Finally, this work lays the foundation of Full-Shape analysis with DESI Y1 galaxy samples where in the tests we performed, we found no systematic error associated with the modeling of the galaxy redshift space power spectrum for this volume.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Forward modeling fluctuations in the DESI LRGs target sample using image simulations

We use the forward modeling pipeline, Obiwan, to study the imaging systematics of the Luminous Red Galaxies (LRGs) targeted by the Dark Energy Spectroscopic Instrument (DESI). Imaging systematics refers to the false fluctuation of galaxy densities due to varying observing conditions and astrophysical foregrounds corresponding to the imaging surveys from which DESI LRG target galaxies are selected. We update the Obiwan pipeline, which we previously developed to simulate the optical images used to target DESI data, to further simulate WISE images in the infrared. This addition allows simulating the DESI LRGs sample, which utilizes WISE data in the target selection. Deep DESI imaging data combined with a method to account for biases in their shapes is used to define a truth sample of potential LRG targets. We inject these data evenly throughout the DESI Legacy Imaging Survey footprint at declinations between -30 and 32.375 degrees. We simulate a total of 15 million galaxies to obtain a simulated LRG sample (Obiwan LRGs) that predicts the variations in target density due to imaging properties. We find that the simulations predict the trends with depth observed in the data, including how they depend on the intrinsic brightness of the galaxies. We observe that faint LRGs are the main contributing source of the imaging systematics trend induced by depth. We also find significant trends in the data against Galactic extinction that are not predicted by Obiwan. These trends depend strongly on the particular map of Galactic extinction chosen to test against, implying systematic contamination in the Galactic extinction maps is a likely root cause (e.g., Cosmic-Infrared Background, dust temperature correction). We additionally observe a morphological change of the DESI LRGs population evidenced by a correlation between OII emission line average intensity and the size of the z-band PSF. This effect most likely results from uncertainties in background subtraction. The detailed findings we present should be used to guide any observational systematics mitigation treatment for the clustering of the DESI LRGs sample.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Window convolution of the galaxy clustering bispectrum

In galaxy survey analysis, the observed clustering statistics do not directly match theoretical predictions but rather have been processed by a window function that arises from the survey geometry including the sky footprint, redshift-dependent background number density and systematic weights. While window convolution of the power spectrum is well studied, for the bispectrum with a larger number of degrees of freedom, it poses a significant numerical and computational challenge. In this work, we consider the effect of the survey window in the tripolar spherical harmonic decomposition of the bispectrum and lay down a formal procedure for their convolution via a series expansion of configuration-space three-point correlation functions, which was first proposed by Sugiyama et al. (2019). We then provide a linear algebra formulation of the full window convolution, where an unwindowed bispectrum model vector can be directly premultiplied by a window matrix specific to each survey geometry. To validate the pipeline, we focus on the Dark Energy Spectroscopic Instrument (DESI) Data Release 1 (DR1) luminous red galaxy (LRG) sample in the South Galactic Cap (SGC) in the redshift bin 0.4 ≤ z ≤ 0.6. We first perform convergence checks on the measurement of the window function from discrete random catalogues, and then investigate the convergence of the window convolution series expansion truncated at a finite of number of terms as well as the performance of the window matrix. This work highlights the differences in window convolution between the power spectrum and bispectrum, and provides a streamlined pipeline for the latter for current surveys such as DESI and the Euclid mission.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

DESI DR1 Lyα 1D power spectrum: the optimal estimator measurement

The one-dimensional power spectrum P 1D of Lyα forest offers rich insights into cosmological and astrophysical parameters, including constraints on the sum of neutrino masses, warm dark matter models, and the thermal state of the intergalactic medium. We present the measurement of P 1D using the optimal quadratic maximum likelihood estimator applied to over 300,000 Lyα quasars from Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) survey. This sample represents the largest to date for P 1D measurements and is larger than the Extended Baryon Oscillation Spectroscopic Survey (eBOSS) by a factor of 1.7. We conduct a meticulous investigation of instrumental and analysis systematics and quantify their impact on P 1D . This includes the development of a cross-exposure estimator that eliminates the need to model the pipeline noise and has strong potential for future P 1D measurements. We also present new insights into metal contamination through the 1D correlation function. Using a fitting function we measure the evolution of the Lyα forest bias with high precision: b F (z) = (-0.218 ± 0.002) × ((1 + z)/4) 2.96±0.06 . In a companion validation paper, we substantially extend our previous suite of CCD image simulations to quantify the pipeline's exquisite performance accurately. In another companion paper, we present DR1 P 1D measurements using the Fast Fourier Transform (FFT) approach to power spectrum estimation. These two measurements produce a forest bias parameter that differs by 2.2 sigma. However, our model is simplistic, so this disagreement will be investigated in future work.

Lyman alpha forest↗
DOE OSTI2026

DESI DR1 Ly α 1D power spectrum: Validation of estimators

The Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) is the largest sample to date for small-scale Lyα forest cosmology, accessed through its one-dimensional power spectrum (P 1D ). The Lyα forest P 1D is extracted from quasar spectra that are highly inhomogeneous (both in wavelength and between quasars) in noise properties due to intrinsic properties of the quasar, atmospheric and astrophysical contamination, and also sensitive to low-level details of the spectral extraction pipeline. We employ two estimators in DR1 analysis to measure P 1D : the optimal estimator and the fast Fourier transform (FFT) estimator. To ensure robustness of our DR1 measurements, we validate these two power spectrum and covariance matrix estimation methodologies against the challenging aspects of the data. First, using a set of 20 synthetic 1D realizations of DR1, we derive the masking bias corrections needed for the FFT estimator and the continuum fitting bias needed for both estimators. We demonstrate that both estimators, including their covariances, are unbiased with these corrections using the Kolmogorov-Smirnov test. Second, we substantially extend our previous suite of CCD image simulations to include 675,000 quasars, allowing us to accurately quantify the pipeline's performance. This set of simulations reveals biases at the highest k values, corresponding to a resolution error of a few percent. We base the resolution systematics error budget of DR1 P 1D on these values, but do not derive corrections from them since the simulation fidelity is insufficient for precise corrections.

Lyman alpha forest↗
DOE OSTI2022

A Fast Radio Burst Backend for the Tianlai Dish Pathfinder Array

The Tianlai Dish Pathfinder Array is a radio interferometer array consisting of 16 six-meter dish antennas. The original digital backend integration time is at the seconds level, designed for H I intensity mapping experiment. A new digital backend with millisecond response is added to enable it to search for fast radio burst during its observations. The design and calibration of this backend, and the real time search pipeline for it are described in this paper. It is capable of forming 16 digital beams for each linear polarization, covering an area of 19.6 square degrees. The search pipeline is capable of searching for, recording and classifying FRBs automatically in real time. Finally, in commissioning, we succeeded in capturing the signal pulses from the pulsars PSR B0329+54 and B2021+51.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

DeepAdversaries: examining the robustness of deep learning models for galaxy morphology classification

With increased adoption of supervised deep learning methods for work with cosmological survey data, the assessment of data perturbation effects (that can naturally occur in the data processing and analysis pipelines) and the development of methods that increase model robustness are increasingly important. In the context of morphological classification of galaxies, we study the effects of perturbations in imaging data. In particular, we examine the consequences of using neural networks when training on baseline data and testing on perturbed data. We consider perturbations associated with two primary sources: (a) increased observational noise as represented by higher levels of Poisson noise and (b) data processing noise incurred by steps such as image compression or telescope errors as represented by one-pixel adversarial attacks. We also test the efficacy of domain adaptation techniques in mitigating the perturbation-driven errors. We use classification accuracy, latent space visualizations, and latent space distance to assess model robustness in the face of these perturbations. For deep learning models without domain adaptation, we find that processing pixel-level errors easily flip the classification into an incorrect class and that higher observational noise makes the model trained on low-noise data unable to classify galaxy morphologies. On the other hand, we show that training with domain adaptation improves model robustness and mitigates the effects of these perturbations, improving the classification accuracy up to 23% on data with higher observational noise. Domain adaptation also increases up to a factor of ${\approx}2.3$ the latent space distance between the baseline and the incorrectly classified one-pixel perturbed image, making the model more robust to inadvertent perturbations. Successful development and implementation of methods that increase model robustness in astronomical survey pipelines will help pave the way for many more uses of deep learning for astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Latent Twins

Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.

Latent Twins↗
DOE OSTI2023

Sensitivity Analysis of Genome-Scale Metabolic Flux Prediction

TRIMER, Transcription Regulation Integrated with MEtabolic Regulation, is a genome-scale modeling pipeline targeting at metabolic engineering applications. Using TRIMER, regulated metabolic reactions can be effectively predicted by integrative modeling of metabolic reactions with a Transcription Factor (TF)-gene regulatory network (TRN), which is modeled via a Bayesian network (BN). In this paper, we focus on sensitivity analysis of metabolic flux prediction for uncertainty quantification of BN structures for TRN modeling in TRIMER. We propose a computational strategy to construct the uncertainty class of TRN models based on the inferred regulatory order uncertainty given transcriptomic expression data. With that, we analyze the prediction sensitivity of the TRIMER pipeline for the metabolite yields of interest. The obtained sensitivity analyses can guide Optimal Experimental Design (OED) to help acquire new data that can enhance TRN modeling and achieve specific metabolic engineering objectives, including metabolite yield alterations. Here we have performed small- and large-scale simulated experiments, demonstrating the effectiveness of our developed sensitivity analysis strategy for BN structure learning to quantify the edge importance in terms of metabolic flux prediction uncertainty reduction and its potential to effectively guide OED.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Automated segmentation of soft X-ray tomography: Native cellular structure with submicron resolution at high-throughput for whole-cell quantitative imaging in yeast

Soft X-ray tomography (SXT) is an invaluable tool for quantitatively analyzing cellular structures at suboptical isotropic resolution. However, it has traditionally depended on manual segmentation, limiting its scalability for large datasets. Here, we leverage a deep learning-based autosegmentation pipeline to segment and label cellular structures in hundreds of cells across three Saccharomyces cerevisiae strains. This task-based pipeline uses manual iterative refinement to improve segmentation accuracy for key structures, including the cell body, nucleus, vacuole, and lipid droplets, enabling high-throughput and precise phenotypic analysis. Using this approach, we quantitatively compared the three-dimensional (3D) whole-cell morphometric characteristics of wild-type, VPH1-GFP, and vac14 strains, uncovering detailed strain-specific cell and organelle size and shape variations. We show the utility of SXT data for precise 3D curvature analysis of entire organelles and cells and detection of fine morphological features using surface meshes. Our approach facilitates comparative analyses with high spatial precision and statistical throughput, uncovering subtle morphological features at the single-cell and population level. This workflow significantly enhances our ability to characterize cell anatomy and supports scalable studies on the mesoscale, with applications in investigating cellular architecture, organelle biology, and genetic research across diverse biological contexts.

Chen, Jianhua [Lawrence Berkeley National Laborato↗
DOE OSTI2026

GenomeDepot: data management system for microbial comparative genomics

Summary GenomeDepot is an open-source web-based platform for annotation, management, and comparative analysis of microbial genomic sequences and associated data including ortholog families, protein domains, operons, regulatory interactions, strain taxonomy, and sample metadata. GenomeDepot supports rapid creation of websites for user-defined genome collections that include bioinformatic tools for interactive genome browsing, Basic Local Alignment Search Tool (BLAST) search, annotation search, comparative genomic neighborhood visualization, and sequence download. Gene function annotations are generated by a customizable annotation pipeline. The pipeline runs annotation tools in Conda environments and can be easily extended with additional user-specified tools. Availability and implementation GenomeDepot is open source and distributed under the GNU General Public License via GitHub (https://github.com/aekazakov/genome-depot). GenomeDepot is implemented in Python and was tested in Ubuntu Linux. Full installation instructions and documentation are available at https://aekazakov.github.io/genome-depot/. GenomeDepot demo server is freely accessible at https://iseq.lbl.gov/demogd/.

Kazakov, Alexey [Lawrence Berkeley National Labora↗
DOE OSTI2022

MIDAS2: Metagenomic Intra-species Diversity Analysis System

The Metagenomic Intra-Species Diversity Analysis System (MIDAS) is a scalable metagenomic pipeline that identifies single nucleotide variants (SNVs) and gene copy number variants in microbial populations. Here, we present MIDAS2, which addresses the computational challenges presented by increasingly large reference genome databases, while adding functionality for building custom databases and leveraging paired-end reads to improve SNV accuracy. This fast and scalable reengineering of the MIDAS pipeline enables thousands of metagenomic samples to be efficiently genotyped.

59 BASIC BIOLOGICAL SCIENCES↗