Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “PIPELINES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

An Optimized Informatics Pipeline for Mass Spectrometry-Based Peptidomics

Comprehensive MS analysis of peptidome, the intracellular and intercellular products of protein degradation, has the potential to provide novel insights on endogenous proteolytic processing and their utility in disease diagnosis and prognosis. Along with the advances in MS instrumentation, a plethora of proteomics data analysis tools have been applied for direct use in peptidomics; however an evaluation of the currently available informatics pipelines for peptidomics data analysis has yet to be reported. In this study, we set off by evaluating the results of several popular MS/MS database search engines including MS-GF+, SEQUEST and MS-Align+ for peptidomics data analysis, followed by identification and label-free quantification using the well-established accurate mass and time (AMT) tag and newly developed informed quantification (IQ) approaches, both based on direct LC-MS analysis. Our result demonstrated that MS-GF+ outperformed both SEQUEST and MS-Align+ in identifying peptidome peptides. Using a database established from the MS-GF+ peptide identifications, both the AMT tag and IQ approaches provided significantly deeper peptidome coverage and less missing value for each individual data set than the MS/MS methods, while achieving robust label-free quantification. Besides having an excellent correlation with the AMT tag quantification results, IQ also provided slightly higher peptidome coverage than AMT. Taken together, we propose an optimal informatics pipeline combining MS-GF+ for initial database searching with IQ (or AMT) for identification and label-free quantification for high-throughput, comprehensive and quantitative peptidomics analysis.

Environmental Molecular Sciences Laboratory↗
DOE OSTI2021

Data processing pipeline for Tianlai experiment

The Tianlai project is a 21cm intensity mapping experiment for detecting dark energy by measuring the baryon acoustic oscillation (BAO) features in the large scale structure power spectrum. This experiment provides an opportunity to test the data processing methods for cosmological 21cm signal extraction, which is still a great challenge in current radio astronomy research. The 21cm signal is much weaker than the foregrounds and easily aected by the imperfections in the instrumental responses. Furthermore, processing the large volumes of interferometer data poses a practical challenge. We have developed a data processing pipeline called tlpipe to process the drift scan survey data from the Tianlai experiment. It performs oine data processing tasks such as radio frequency interference (RFI) agging, array calibration, binning, and map-making, etc. It also includes utility functions needed for the data analysis, such as data selection, transformation, visualization and others. A number of new algorithms are implemented, for example the eigenvector decomposition method for array calibration and the Tikhnov regularization for m-mode analysis. In this paper we describe the design and implementation of the pipeline and illustrate its functions with some analysis of real data. Finally, we outline directions for future development of this publicly code.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

A Finite-Volume method for compressible non-equilibrium two-phase flows in networks of elastic pipelines using the Baer–Nunziato model

A novel Finite-Volume scheme for the numerical computations of compressible two-phase flows in pipelines is proposed for the fully non-equilibrium Baer–Nunziato model. The present FV approach is the extension of the method proposed in Daude and Galon (2018) in the context of the Euler equations to the Baer–Nunziato model. In addition, proper approximations of the non-conservative terms are proposed to consider jumps of volume fraction as well as jumps of cross-section in order to respect uniform pressure and velocity profiles preservation. In particular, focus is given to the numerical treatment of abrupt changes in area and to networks wherein several pipelines are connected at junctions. The proposed method makes it possible to avoid the use of an iterative procedure for the solution of the junction problem. The present approach can also deal with general Equations Of State. In addition, the fluid–structure interaction of compressible fluid flowing in flexible pipes is also considered. Lastly, the proposed scheme is then assessed on a variety of shock-tubes and other transient flow problems and experiments demonstrating its capability to resolve such problems efficiently, accurately and robustly.

42 ENGINEERING↗
DOE OSTI2022

A machine learning pipeline for membrane segmentation of cryo-electron tomograms

We describe how to use several machine learning techniques organized in a learning pipeline to segment and identify cell membrane structures from cryo electron tomograms. These tomograms are difficult to analyze with traditional segmentation tools. The learning pipeline in our approach starts from supervised learning via a special convolutional neural network trained with simulated data. It continues with semi-supervised reinforcement learning and/or a region merging technique that tries to piece together disconnected components belonging to the same membrane structure. A parametric or non-parametric fitting procedure is then used to enhance the segmentation results and quantify uncertainties in the fitting. Domain knowledge is used in generating the training data for the neural network and in guiding the fitting procedure through the use of appropriately chosen priors and constraints. We demonstrate that the approach proposed here works well for extracting membrane surfaces in two real tomogram datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Machine learning pipeline for denoising low signal-to-noise ratio and out-of-distribution transmission electron microscopy datasets

High-resolution transmission electron microscopy (HRTEM) is crucial for observing material’s structural and morphological evolution at Angstrom scales, but the electron beam can alter these processes. Devices such as CMOS-based direct-electron detectors operating in electron-counting mode can be utilized to substantially reduce the electron dosage. However, the resulting images often lead to a low signal-to-noise ratio, which requires frame integration that sacrifices temporal resolution. Several machine learning (ML) models have been recently developed to successfully denoise HRTEM images. Yet, these models are often computationally expensive, and their inference speeds on GPUs are outpaced by the imaging speed of advanced detectors, precluding in situ analysis. Furthermore, the performance of these denoising models on datasets with imaging conditions that deviate from the training datasets has not been evaluated. To mitigate these gaps, we propose a new self-supervised ML denoising pipeline specifically designed for time-series HRTEM images. This pipeline integrates a blind-spot convolution neural network with pre-processing and post-processing steps, including drift correction and low-pass filtering. Results demonstrate that our model outperforms various other ML and non-ML denoising methods in noise reduction and contrast enhancement, leading to improved visual clarity of atomic features. Additionally, the model is drastically faster than U-Net-based ML models and demonstrates excellent out-of-distribution generalization. The model’s computational inference speed is in the order of milliseconds per image, rendering it suitable for application in in-situ HRTEM experiments.

36 MATERIALS SCIENCE↗
DOE OSTI2022

A reusable neural network pipeline for unidirectional fiber segmentation

Abstract Fiber-reinforced ceramic-matrix composites are advanced, temperature resistant materials with applications in aerospace engineering. Their analysis involves the detection and separation of fibers, embedded in a fiber bed, from an imaged sample. Currently, this is mostly done using semi-supervised techniques. Here, we present an open, automated computational pipeline to detect fibers from a tomographically reconstructed X-ray volume. We apply our pipeline to a non-trivial dataset by Larson et al . To separate the fibers in these samples, we tested four different architectures of convolutional neural networks. When comparing our neural network approach to a semi-supervised one, we obtained Dice and Matthews coefficients reaching up to 98%, showing that these automated approaches can match human-supervised methods, in some cases separating fibers that human-curated algorithms could not find. The software written for this project is open source, released under a permissive license, and can be freely adapted and re-used in other domains.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2014

Mathematical model of testing of pipeline integrity by thermal fields

Thermal fields testing at the ground surface above a pipeline are considered. One method to obtain and investigate an ideal thermal field in different environments is a direct numerical simulation of heat transfer processes taking into account the most important physical factors. In the paper a mathematical model of heat propagation from an underground source is described with accounting of physical factors such as filtration of water in soil and solar radiation. Thermal processes are considered in 3D origin where the heat source is a pipeline with constant temperature and non-uniform isolated shell (with 'damages'). This problem leads to solution of heat diffusivity equation with nonlinear boundary conditions. Approaches to analysis of thermal fields are considered to detect damages.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2015

Application of the SEGUE Stellar Parameter Pipeline to LAMOST Stellar Spectra

We describe an application of the SEGUE Stellar Parameter Pipeline (SSPP) to medium-resolution stellar spectra obtained by the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST), in order to determine estimates of the stellar atmospheric parameters (T{sub eff}, log g, and [Fe/H]) and the abundance ratios ([α/Fe] and [C/Fe]). By performing a coordinate match with the LAMOST stellar database, we selected stars with LAMOST spectra in common with stars having available spectroscopy from the Apache Point Observatory Galactic Evolution Experiment (APOGEE), the RAdial Velocity Experiment (RAVE), and the Sloan Extension for Galactic Understanding and Exploration (SEGUE). We ran the selected LAMOST stellar spectra from each survey through SSPP, and compared the stellar parameters down to signal-to-noise ratio (S/N) of 10 and chemical abundances down to S/N = 20 derived by SSPP with those determined by the APOGEE, RAVE, and SEGUE software pipelines. Our results show that the derived stellar parameters generally agree quite well, even though there exist some small systematic offsets with small scatter in T{sub eff}, log g, and [Fe/H], due to the use of different temperature scales, abundance scales, and calibrations adopted by each survey. Comparison of the [α/Fe] determinations for LAMOST spectra suggests no sign of significant systematic offsets (<–0.04 dex), with a small scatter (<0.08 dex) relative to stars in common with APOGEE and SEGUE. The [C/Fe] estimates determined for the LAMOST spectra also exhibit good agreement, with a very small offset (∼0.01 dex) and scatter (∼0.12 dex) relative to the SEGUE stars, while there exists about a –0.19 dex offset, with a small scatter of ∼0.13 dex, for the APOGEE sample. Due to the existence of small offsets in the stellar parameters and abundances among difference data sets, optimal results when combining the different data sets will be obtained by removing the offsets. Once accomplished, the stellar parameters and chemical abundances estimated by SSPP from the LAMOST stellar spectra should provide a reliable database for studies of the Galactic disk and halo systems.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2013

The Gemini NICI planet-finding campaign: The companion detection pipeline

We present high-contrast image processing techniques used by the Gemini NICI Planet-Finding Campaign to detect faint companions to bright stars. The Near-Infrared Coronographic Imager (NICI) is an adaptive optics instrument installed on the 8 m Gemini South telescope, capable of angular and spectral difference imaging and specifically designed to image exoplanets. The Campaign data pipeline achieves median contrasts of 12.6 mag at 0.''5 and 14.4 mag at 1'' separation, for a sample of 45 stars (V = 4.3-13.9 mag) from the early phase of the campaign. We also present a novel approach to calculating contrast curves for companion detection based on 95% completeness in the recovery of artificial companions injected into the raw data, while accounting for the false-positive rate. We use this technique to select the image processing algorithms that are more successful at recovering faint simulated point sources. We compare our pipeline to the performance of the Locally Optimized Combination of Images (LOCI) algorithm for NICI data and do not find significant improvement with LOCI.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Large-scale real-time signal processing in physics experiments: the ALICE TPC FPGA pipeline

For LHC Run 3, the ALICE Time Projection Chamber was upgraded to operate in continuous readout mode. Interaction rates of up to 50 kHz in Pb-Pb collisions require real-time processing of more than 3 TB s -1 of raw detector data. This requirement is met by a custom FPGA-based processing pipeline that performs the complete front-end data treatment fully in-stream, including common-mode correction, pedestal subtraction, ion-tail filtering, zero suppression, and dense data packing. A central element of the design is a highly parallel common-mode correction algorithm operating directly on the streaming data. It robustly identifies signal-free readout channels on a time-bin basis and applies pad-dependent scaling to compensate for local variations in capacitive coupling in the GEM readout. In combination with pedestal subtraction and ion-tail filtering, this enables accurate baseline restoration under extreme high-occupancy conditions, preventing signal loss while efficiently suppressing noise prior to zero suppression. The pipeline operates continuously at the full detector bandwidth and reduces the raw input rate of approximately 3 TB s -1 to about 900 GBps for Pb-Pb collisions at the target interaction rate. Overall, it represents a large-scale FPGA-based real-time signal-processing implementation for high-energy physics detector readout.

Digital signal processing (DSP)↗
DOE OSTI2021

New high-throughput endstation to accelerate the experimental optimization pipeline for synchrotron X-ray footprinting

Synchrotron X-ray footprinting (XF) is a growing structural biology technique that leverages radiation-induced chemical modifications via X-ray radiolysis of water to produce hydroxyl radicals that probe changes in macromolecular structure and dynamics in solution states of interest. The X-ray Footprinting of Biological Materials (XFP) beamline at the National Synchrotron Light Source II provides the structural biology community with access to instrumentation and expert support in the XF method, and is also a platform for development of new technological capabilities in this field. Hee, the design and implementation of a new high-throughput endstation device based around use of a 96-well PCR plate form factor and supporting diagnostic instrumentation for synchrotron XF is described. This development enables a pipeline for rapid comprehensive screening of the influence of sample chemistry on hydroxyl radical dose using a convenient fluorescent assay, illustrated here with a study of 26 organic compounds. The new high-throughput endstation device and sample evaluation pipeline now available at the XFP beamline provide the worldwide structural biology community with a robust resource for carrying out well optimized synchrotron XF studies of challenging biological systems with complex sample compositions.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2023

A Deep Learning Pipeline for Optimizing Large-scale Phase Field Simulations

Phase field (PF) simulations are computationally expensive but remain a key analysis tool to understand the complex mechanisms of additive manufacturing (AM) processes. Each PF simulation-aided analysis requires thousands of node hours on leadership-class supercomputers. One of the main goals of these analyses is the study of microstructure evolution during the build process which begins with the onset of nucleation. Nucleation occurs under certain thermomechanical conditions which are not known a priori and many PF simulations are required to identify ranges of input thermo-mechanical parameters that can result in the onset of nucleation. Since many of the simulations do not result in nucleation, an analysis campaign often ends up wasting tremendous amounts of precious computing resources executing nucleation-absent simulations. The goal of this work is to design and train deep learning models to inform a PF simulation about the likelihood of the occurrence of nucleation in a future simulation time-step based on the state summary over a finite number of past time-steps of a running simulation. If the prediction determines that the running simulation is unlikely to reach nucleation in the allotted time, then its execution is stopped immediately ultimately resulting in vast reduction in wasted computations when accrued over all the PF simulations typically performed in a single or multiple analysis campaign(s). The paper presents the performance of a machine learning pipeline that uses a convolutional neural network (CNN) model to learn an embedding which is then used with a self-attention network to build a multi-task deep learning model to predict the likelihood of nucleation. The model also predicts the input parameters used in a simulation. Performance is compared with a baseline pipeline that uses an off-the-shelf LeNet-5 model to learn the initial embedding. Despite their smaller size, performance results indicate significant improvement in accuracy of the proposed models compared to the larger baseline models.

Kannan, Ramakrishnan {ramki}↗
DOE OSTI2023

Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest

Data compression is becoming critical for data-intensive scientific applications. Scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Prior work has shown that a pipeline can be built to guarantee error on the primary data (PD) within user-defined bounds and achieve near-floating point QoI errors. In this paper, we present novel computational approaches for accelerating the pipeline and demonstrate results that enable concurrent execution of compression in parallel with the simulation nodes. This allows compression, including the writing of the required compression data, for the previous time step to be completed while the simulation proceeds with the current time step. Overall, the approach presented in this paper results in a 6–8 times improvement in computational overhead compared to previous work. These results were obtained using data generated by a large-scale fusion code called XGC, which produces hundreds of terabytes of data in a single day.

Banerjee, Tania↗
DOE OSTI2018

State and Parameter Estimation for Natural Gas Pipeline Networks Using Transient State Data

Here, we formulate two estimation problems for pipeline systems in which measurements of the compressible gas flowing through a network of pipes are affected by time-varying injections, withdrawals, and compression. We consider a state estimation problem that is then extended to a joint state and parameter estimation problem that can be used for data assimilation. In both formulations, the flow dynamics are described on each pipe by space- and time-dependent densities and mass flux which evolve according to a system of coupled partial differential equations, in which momentum dissipation is modeled using the Darcy-Wiesbach friction approximation. These dynamics are first spatially discretized to obtain a system of nonlinear ordinary differential equations on which state and parameter estimation formulations are given as nonlinear least squares problems. A rapid, scalable computational method for performing a nonlinear least squares estimation is developed. Extensive simulations and computational experiments on multiple pipeline test networks demonstrate the effectiveness of the formulations in obtaining state and parameter estimates in the presence of measurement and process noise.

03 NATURAL GAS↗
DOE OSTI2019

Atom Modeling PipeLine

This repository contains code for the computational framework to build ML models for drug discovery purposes. Our end-to-end pipeline extracts features from model-ready datasets and trains and saves the model to our model zoo or to disk. Our pipeline generates a variety of molecular features and both shallow and deep ML models. The HPC-specific module we have developed conducts efficient parallelized search of the model hyperparameter space and reports the best-performing hyperparameters for each of these feature/model combinations.

Minnich, AmandaJean↗
DOE OSTI2023

IMG Annotation Pipeline (IMGAP) v5.1.13

The IMG Annotation Pipeline is a collection of Bash and Python scripts to control a workflow for structural and functional annotation of prokaryotic genomes, metagenomes, and metatranscriptomes. The bash scripts in general control the overall workflow and are wrappers around 3rd party executables (not included in repo) that predict features or functions. Whereas the Python scripts do post-processing of raw output in terms of filtering or format transformation and in some cases contain some logic for picking the correct predictions or resolving overlaps. The pipeline is tailored to produce results required by IMG (https://img.jgi.doe.gov/) and is executed on every dataset submitted to IMG via https://img.jgi.doe.gov/submit. These consist of internal genomes, metagenomes and metatranscriptomes sequenced and assembled at the JGI, as well as datasets submitted by external users (non-lab/JGI affiliates).

Huntemann, Marcel↗
DOE OSTI2026

Biosynth Pipeline v1.0

BioPKS Pipeline is a computational pipeline for retrosynthetic design of small molecule biosynthesis pathways (e.g. retrobiosynthesis). It combines capabilities by interfacing with existing retrobiosynthesis tools- RetroTide (developed at LBNL) and DORAnet to create pathways that combine multiple biosynthesis approaches- both megasynthase assembly line enzymes and single step enzymes.

Backman, Tyler [Lawrence Berkeley National Laborat↗
DOE OSTI2020

A pipeline for targeted metagenomics of environmental bacteria

Background:Metagenomics and single cell genomics provide a window into the genetic repertoire of yet uncultivated microorganisms, but both methods are usually taxonomically untargeted. The combination of fluorescence in situ hybridization (FISH) and fluorescence activated cell sorting (FACS) has the potential to enrich taxonomically well-defined clades for genomic analyses. Methods:Cells hybridized with a taxon-specific FISH probe are enriched based on their fluorescence signal via flow cytometric cell sorting. A recently developed FISH procedure, the hybridization chain reaction (HCR)-FISH, provides the high signal intensities required for flow cytometric sorting while maintaining the integrity of the cellular DNA for subsequent genome sequencing. Sorted cells are subjected to shotgun sequencing, resulting in targeted metagenomes of low diversity. Results: Pure cultures of different taxonomic groups were used to (1) adapt and optimize the HCR-FISH protocol and (2) assess the effects of various cell fixation methods on both the signal intensity for cell sorting and the quality of subsequent genome amplification and sequencing. Best results were obtained for ethanol-fixed cells in terms of both HCR-FISH signal intensity and genome assembly quality. Our newly developed pipeline was successfully applied to a marine plankton sample from the North Sea yielding good quality metagenome assembled genomes from a yet uncultivated flavobacterial clade. Conclusions: With the developed pipeline, targeted metagenomes at various taxonomic levels can be efficiently retrieved from environmental samples. The resulting metagenome assembled genomes allow for the description of yet uncharacterized microbial clades.

59 BASIC BIOLOGICAL SCIENCES↗