Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Analysing Solar-like Oscillations with an Automatic Pipeline

The Kepler mission will provide a huge amount of asteroseismic data during the next few years, among which hundreds of solar-like stars will be targeted. The amount of stars and their observation length represent a step forward in the comprehension of the stellar evolution that has already been initiated by CoRoT and MOST missions. Up to now, the slow cadence of observed targets allowed an individual and personalized analysis of each star. During the survey phase of Kepler, this will be impossible. This is the reason why, within the AsteroFLAG team, we have been developing automatic pipelines for the Kepler solar-like oscillation stars. Our code starts by finding the frequency-range where p-mode power is present and, after fitting the background, it looks for the mode amplitudes as well as the central frequency of the p-mode hump. A good estimation of the large separation can thus be inferred in this region. If the signal to noise is high enough, the code obtains the characteristics of the p modes by doing a global fitting on the power spectrum. Here, we will first describe a few features of this pipeline and its application to AsteroFLAG synthetic data to check the validity of the code.

79 ASTRONOMY AND ASTROPHYSICS↗

Mathematical model of testing of pipeline integrity by thermal fields

Thermal fields testing at the ground surface above a pipeline are considered. One method to obtain and investigate an ideal thermal field in different environments is a direct numerical simulation of heat transfer processes taking into account the most important physical factors. In the paper a mathematical model of heat propagation from an underground source is described with accounting of physical factors such as filtration of water in soil and solar radiation. Thermal processes are considered in 3D origin where the heat source is a pipeline with constant temperature and non-uniform isolated shell (with 'damages'). This problem leads to solution of heat diffusivity equation with nonlinear boundary conditions. Approaches to analysis of thermal fields are considered to detect damages.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

THE SEGUE STELLAR PARAMETER PIPELINE. IV. VALIDATION WITH AN EXTENDED SAMPLE OF GALACTIC GLOBULAR AND OPEN CLUSTERS

Spectroscopic and photometric data for likely member stars of five Galactic globular clusters (M3, M53, M71, M92, and NGC 5053) and three open clusters (M35, NGC 2158, and NGC 6791) are processed by the current version of the SEGUE Stellar Parameter Pipeline (SSPP), in order to determine estimates of metallicities and radial velocities (RVs) for the clusters. These results are then compared to values from the literature. We find that the mean metallicity (([Fe/H])) and mean radial velocity ((RV)) estimates for each cluster are almost all within 2{sigma} of the adopted literature values; most are within 1{sigma}. We also demonstrate that the new version of the SSPP achieves small, but noteworthy, improvements in ([Fe/H]) estimates at the extrema of the cluster metallicity range, as compared to a previous version of the pipeline software. These results provide additional confidence in the application of the SSPP for studies of the abundances and kinematics of stellar populations in the Galaxy.

79 ASTRONOMY AND ASTROPHYSICS↗

Application of the SEGUE Stellar Parameter Pipeline to LAMOST Stellar Spectra

We describe an application of the SEGUE Stellar Parameter Pipeline (SSPP) to medium-resolution stellar spectra obtained by the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST), in order to determine estimates of the stellar atmospheric parameters (T{sub eff}, log g, and [Fe/H]) and the abundance ratios ([α/Fe] and [C/Fe]). By performing a coordinate match with the LAMOST stellar database, we selected stars with LAMOST spectra in common with stars having available spectroscopy from the Apache Point Observatory Galactic Evolution Experiment (APOGEE), the RAdial Velocity Experiment (RAVE), and the Sloan Extension for Galactic Understanding and Exploration (SEGUE). We ran the selected LAMOST stellar spectra from each survey through SSPP, and compared the stellar parameters down to signal-to-noise ratio (S/N) of 10 and chemical abundances down to S/N = 20 derived by SSPP with those determined by the APOGEE, RAVE, and SEGUE software pipelines. Our results show that the derived stellar parameters generally agree quite well, even though there exist some small systematic offsets with small scatter in T{sub eff}, log g, and [Fe/H], due to the use of different temperature scales, abundance scales, and calibrations adopted by each survey. Comparison of the [α/Fe] determinations for LAMOST spectra suggests no sign of significant systematic offsets (<–0.04 dex), with a small scatter (<0.08 dex) relative to stars in common with APOGEE and SEGUE. The [C/Fe] estimates determined for the LAMOST spectra also exhibit good agreement, with a very small offset (∼0.01 dex) and scatter (∼0.12 dex) relative to the SEGUE stars, while there exists about a –0.19 dex offset, with a small scatter of ∼0.13 dex, for the APOGEE sample. Due to the existence of small offsets in the stellar parameters and abundances among difference data sets, optimal results when combining the different data sets will be obtained by removing the offsets. Once accomplished, the stellar parameters and chemical abundances estimated by SSPP from the LAMOST stellar spectra should provide a reliable database for studies of the Galactic disk and halo systems.

79 ASTRONOMY AND ASTROPHYSICS↗

A STELLAR MODEL-FITTING PIPELINE FOR ASTEROSEISMIC DATA FROM THE KEPLER MISSION

Over the past two decades, helioseismology has revolutionized our understanding of the interior structure and dynamics of the Sun. Asteroseismology will soon place this knowledge into a broader context by providing structural data for hundreds of Sun-like stars. Solar-like oscillations have already been detected from the ground in several stars, and NASA's Kepler mission is poised to unleash a flood of stellar pulsation data. Deriving reliable asteroseismic information from these observations demands a significant improvement in our analysis methods. In this paper, we report the initial results of our efforts to develop an objective stellar model-fitting pipeline for asteroseismic data. The cornerstone of our automated approach is an optimization method using a parallel genetic algorithm. We describe the details of the pipeline and we present the initial application to Sun-as-a-star data, yielding an optimal model that accurately reproduces the known solar properties.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The Gemini NICI planet-finding campaign: The companion detection pipeline

We present high-contrast image processing techniques used by the Gemini NICI Planet-Finding Campaign to detect faint companions to bright stars. The Near-Infrared Coronographic Imager (NICI) is an adaptive optics instrument installed on the 8 m Gemini South telescope, capable of angular and spectral difference imaging and specifically designed to image exoplanets. The Campaign data pipeline achieves median contrasts of 12.6 mag at 0.''5 and 14.4 mag at 1'' separation, for a sample of 45 stars (V = 4.3-13.9 mag) from the early phase of the campaign. We also present a novel approach to calculating contrast curves for companion detection based on 95% completeness in the recovery of artificial companions injected into the raw data, while accounting for the false-positive rate. We use this technique to select the image processing algorithms that are more successful at recovering faint simulated point sources. We compare our pipeline to the performance of the Locally Optimized Combination of Images (LOCI) algorithm for NICI data and do not find significant improvement with LOCI.

79 ASTRONOMY AND ASTROPHYSICS↗

Large-scale real-time signal processing in physics experiments: the ALICE TPC FPGA pipeline

For LHC Run 3, the ALICE Time Projection Chamber was upgraded to operate in continuous readout mode. Interaction rates of up to 50 kHz in Pb-Pb collisions require real-time processing of more than 3 TB s -1 of raw detector data. This requirement is met by a custom FPGA-based processing pipeline that performs the complete front-end data treatment fully in-stream, including common-mode correction, pedestal subtraction, ion-tail filtering, zero suppression, and dense data packing. A central element of the design is a highly parallel common-mode correction algorithm operating directly on the streaming data. It robustly identifies signal-free readout channels on a time-bin basis and applies pad-dependent scaling to compensate for local variations in capacitive coupling in the GEM readout. In combination with pedestal subtraction and ion-tail filtering, this enables accurate baseline restoration under extreme high-occupancy conditions, preventing signal loss while efficiently suppressing noise prior to zero suppression. The pipeline operates continuously at the full detector bandwidth and reduces the raw input rate of approximately 3 TB s -1 to about 900 GBps for Pb-Pb collisions at the target interaction rate. Overall, it represents a large-scale FPGA-based real-time signal-processing implementation for high-energy physics detector readout.

Digital signal processing (DSP)↗

New high-throughput endstation to accelerate the experimental optimization pipeline for synchrotron X-ray footprinting

Synchrotron X-ray footprinting (XF) is a growing structural biology technique that leverages radiation-induced chemical modifications via X-ray radiolysis of water to produce hydroxyl radicals that probe changes in macromolecular structure and dynamics in solution states of interest. The X-ray Footprinting of Biological Materials (XFP) beamline at the National Synchrotron Light Source II provides the structural biology community with access to instrumentation and expert support in the XF method, and is also a platform for development of new technological capabilities in this field. Hee, the design and implementation of a new high-throughput endstation device based around use of a 96-well PCR plate form factor and supporting diagnostic instrumentation for synchrotron XF is described. This development enables a pipeline for rapid comprehensive screening of the influence of sample chemistry on hydroxyl radical dose using a convenient fluorescent assay, illustrated here with a study of 26 organic compounds. The new high-throughput endstation device and sample evaluation pipeline now available at the XFP beamline provide the worldwide structural biology community with a robust resource for carrying out well optimized synchrotron XF studies of challenging biological systems with complex sample compositions.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

A Deep Learning Pipeline for Optimizing Large-scale Phase Field Simulations

Phase field (PF) simulations are computationally expensive but remain a key analysis tool to understand the complex mechanisms of additive manufacturing (AM) processes. Each PF simulation-aided analysis requires thousands of node hours on leadership-class supercomputers. One of the main goals of these analyses is the study of microstructure evolution during the build process which begins with the onset of nucleation. Nucleation occurs under certain thermomechanical conditions which are not known a priori and many PF simulations are required to identify ranges of input thermo-mechanical parameters that can result in the onset of nucleation. Since many of the simulations do not result in nucleation, an analysis campaign often ends up wasting tremendous amounts of precious computing resources executing nucleation-absent simulations. The goal of this work is to design and train deep learning models to inform a PF simulation about the likelihood of the occurrence of nucleation in a future simulation time-step based on the state summary over a finite number of past time-steps of a running simulation. If the prediction determines that the running simulation is unlikely to reach nucleation in the allotted time, then its execution is stopped immediately ultimately resulting in vast reduction in wasted computations when accrued over all the PF simulations typically performed in a single or multiple analysis campaign(s). The paper presents the performance of a machine learning pipeline that uses a convolutional neural network (CNN) model to learn an embedding which is then used with a self-attention network to build a multi-task deep learning model to predict the likelihood of nucleation. The model also predicts the input parameters used in a simulation. Performance is compared with a baseline pipeline that uses an off-the-shelf LeNet-5 model to learn the initial embedding. Despite their smaller size, performance results indicate significant improvement in accuracy of the proposed models compared to the larger baseline models.

Kannan, Ramakrishnan {ramki}↗

Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest

Data compression is becoming critical for data-intensive scientific applications. Scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Prior work has shown that a pipeline can be built to guarantee error on the primary data (PD) within user-defined bounds and achieve near-floating point QoI errors. In this paper, we present novel computational approaches for accelerating the pipeline and demonstrate results that enable concurrent execution of compression in parallel with the simulation nodes. This allows compression, including the writing of the required compression data, for the previous time step to be completed while the simulation proceeds with the current time step. Overall, the approach presented in this paper results in a 6–8 times improvement in computational overhead compared to previous work. These results were obtained using data generated by a large-scale fusion code called XGC, which produces hundreds of terabytes of data in a single day.

Banerjee, Tania↗

State and Parameter Estimation for Natural Gas Pipeline Networks Using Transient State Data

Here, we formulate two estimation problems for pipeline systems in which measurements of the compressible gas flowing through a network of pipes are affected by time-varying injections, withdrawals, and compression. We consider a state estimation problem that is then extended to a joint state and parameter estimation problem that can be used for data assimilation. In both formulations, the flow dynamics are described on each pipe by space- and time-dependent densities and mass flux which evolve according to a system of coupled partial differential equations, in which momentum dissipation is modeled using the Darcy-Wiesbach friction approximation. These dynamics are first spatially discretized to obtain a system of nonlinear ordinary differential equations on which state and parameter estimation formulations are given as nonlinear least squares problems. A rapid, scalable computational method for performing a nonlinear least squares estimation is developed. Extensive simulations and computational experiments on multiple pipeline test networks demonstrate the effectiveness of the formulations in obtaining state and parameter estimates in the presence of measurement and process noise.

03 NATURAL GAS↗

Atom Modeling PipeLine

This repository contains code for the computational framework to build ML models for drug discovery purposes. Our end-to-end pipeline extracts features from model-ready datasets and trains and saves the model to our model zoo or to disk. Our pipeline generates a variety of molecular features and both shallow and deep ML models. The HPC-specific module we have developed conducts efficient parallelized search of the model hyperparameter space and reports the best-performing hyperparameters for each of these feature/model combinations.

Minnich, AmandaJean↗

IMG Annotation Pipeline (IMGAP) v5.1.13

The IMG Annotation Pipeline is a collection of Bash and Python scripts to control a workflow for structural and functional annotation of prokaryotic genomes, metagenomes, and metatranscriptomes. The bash scripts in general control the overall workflow and are wrappers around 3rd party executables (not included in repo) that predict features or functions. Whereas the Python scripts do post-processing of raw output in terms of filtering or format transformation and in some cases contain some logic for picking the correct predictions or resolving overlaps. The pipeline is tailored to produce results required by IMG (https://img.jgi.doe.gov/) and is executed on every dataset submitted to IMG via https://img.jgi.doe.gov/submit. These consist of internal genomes, metagenomes and metatranscriptomes sequenced and assembled at the JGI, as well as datasets submitted by external users (non-lab/JGI affiliates).

Huntemann, Marcel↗

Biosynth Pipeline v1.0

BioPKS Pipeline is a computational pipeline for retrosynthetic design of small molecule biosynthesis pathways (e.g. retrobiosynthesis). It combines capabilities by interfacing with existing retrobiosynthesis tools- RetroTide (developed at LBNL) and DORAnet to create pathways that combine multiple biosynthesis approaches- both megasynthase assembly line enzymes and single step enzymes.

Backman, Tyler [Lawrence Berkeley National Laborat↗

A pipeline for targeted metagenomics of environmental bacteria

Background:Metagenomics and single cell genomics provide a window into the genetic repertoire of yet uncultivated microorganisms, but both methods are usually taxonomically untargeted. The combination of fluorescence in situ hybridization (FISH) and fluorescence activated cell sorting (FACS) has the potential to enrich taxonomically well-defined clades for genomic analyses. Methods:Cells hybridized with a taxon-specific FISH probe are enriched based on their fluorescence signal via flow cytometric cell sorting. A recently developed FISH procedure, the hybridization chain reaction (HCR)-FISH, provides the high signal intensities required for flow cytometric sorting while maintaining the integrity of the cellular DNA for subsequent genome sequencing. Sorted cells are subjected to shotgun sequencing, resulting in targeted metagenomes of low diversity. Results: Pure cultures of different taxonomic groups were used to (1) adapt and optimize the HCR-FISH protocol and (2) assess the effects of various cell fixation methods on both the signal intensity for cell sorting and the quality of subsequent genome amplification and sequencing. Best results were obtained for ethanol-fixed cells in terms of both HCR-FISH signal intensity and genome assembly quality. Our newly developed pipeline was successfully applied to a marine plankton sample from the North Sea yielding good quality metagenome assembled genomes from a yet uncultivated flavobacterial clade. Conclusions: With the developed pipeline, targeted metagenomes at various taxonomic levels can be efficiently retrieved from environmental samples. The resulting metagenome assembled genomes allow for the description of yet uncharacterized microbial clades.

59 BASIC BIOLOGICAL SCIENCES↗

MetaboDirect: an analytical pipeline for the processing of FT-ICR MS-based metabolomic data

Background: Microbiomes are now recognized as the main drivers of ecosystem function ranging from the oceans and soils to humans and bioreactors. However, a grand challenge in microbiome science is to characterize and quantify the chemical currencies of organic matter (i.e., metabolites) that microbes respond to and alter. Critical to this has been the development of Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS), which has drastically increased molecular characterization of complex organic matter samples, but challenges users with hundreds of millions of data points where readily available, user-friendly, and customizable software tools are lacking. Results: Here, we build on years of analytical experience with diverse sample types to develop MetaboDirect, an open-source, command-line-based pipeline for the analysis (e.g., chemodiversity analysis, multivariate statistics), visualization (e.g., Van Krevelen diagrams, elemental and molecular class composition plots), and presentation of direct injection high-resolution FT-ICR MS data sets after molecular formula assignment has been performed. When compared to other available FT-ICR MS software, MetaboDirect is superior in that it requires a single line of code to launch a fully automated framework for the generation and visualization of a wide range of plots, with minimal coding experience required. Among the tools evaluated, MetaboDirect is also uniquely able to automatically generate biochemical transformation networks (ab initio) based on mass differences (mass difference network-based approach) that provide an experimental assessment of metabolite connections within a given sample or a complex metabolic system, thereby providing important information about the nature of the samples and the set of microbial reactions or pathways that gave rise to them. Finally, for more experienced users, MetaboDirect allows users to customize plots, outputs, and analyses. Conclusion: Application of MetaboDirect to FT-ICR MS-based metabolomic data sets from a marine phage-bacterial infection experiment and a Sphagnum leachate microbiome incubation experiment showcase the exploration capabilities of the pipeline that will enable the research community to evaluate and interpret their data in greater depth and in less time. It will further advance our knowledge of how microbial communities influence and are influenced by the chemical makeup of the surrounding system. The source code and User’s guide of MetaboDirect are freely available through (https://github.com/Coayala/MetaboDirect) and (https://metabodirect.readthedocs.io/en/latest/), respectively.

54 ENVIRONMENTAL SCIENCES↗

The standard operating procedure of the DOE-JGI Metagenome Annotation Pipeline (MAP v.4)

Abstract The DOE-JGI Metagenome Annotation Pipeline (MAP v.4) performs structural and functional annotation for metagenomic sequences that are submitted to the Integrated Microbial Genomes with Microbiomes (IMG/M) system for comparative analysis. The pipeline runs on nucleotide sequences provided via the IMG submission site. Users must first define their analysis projects in GOLD and then submit the associated sequence datasets consisting of scaffolds/contigs with optional coverage information and/or unassembled reads in fasta and fastq file formats. The MAP processing consists of feature prediction including identification of protein-coding genes, non-coding RNAs and regulatory RNAs, as well as CRISPR elements. Structural annotation is followed by functional annotation including assignment of protein product names and connection to various protein family databases.

59 BASIC BIOLOGICAL SCIENCES↗

Distributed fiber sensor and machine learning data analytics for pipeline protection against extrinsic intrusions and intrinsic corrosions

This paper presents an integrated technical framework to protect pipelines against both malicious intrusions and piping degradation using a distributed fiber sensing technology and artificial intelligence. A distributed acoustic sensing (DAS) system based on phase-sensitive optical time-domain reflectometry (φ-OTDR) was used to detect acoustic wave propagation and scattering along pipeline structures consisting of straight piping and sharp bend elbow. Signal to noise ratio of the DAS system was enhanced by femtosecond induced artificial Rayleigh scattering centers. Data harnessed by the DAS system were analyzed by neural network-based machine learning algorithms. The system identified with over 85% accuracy in various external impact events, and over 94% accuracy for defect identification through supervised learning and 71% accuracy through unsupervised learning.

Peng, Zhaoqiang↗