Results for “pipeline” — Page 6

DOE OSTI2023

Multi-choice Viromics Pipeline (MVP) v1

MVP stands for Multi-choice Viromics Pipeline. It is a pipeline that utilizes a suite of state-of-art tools: geNomad to identify viruses, proviruses, and plasmids in sequencing data, CheckV to assess the quality, and completeness of identified viral genomes, including identification of host contamination for integrated proviruses, A custom code for a rapid genome clustering based on pairwise ANI, Bowtie2, Samtools, and CoverM to calculate coverage of individual viral genomes by read mapping, A custom code to create a vOTU table of abundance, MMseqs2 to compare viral proteins to multiple databases. It provides a quick, and intuitive pipeline to get viral sequences and corresponding properties that can be used for downstream analyses.

Roux, Simon↗

DOE OSTI2025

Data Agnostic Feature-Target Analysis & Ranking Machine Learning Pipeline (DAFTAR-ML) v0.1.0

DAFTAR-ML is a specialized machine-learning pipeline that identifies relevant features based on their relationship to a target variable. Many ML pipelines focus solely on prediction, and feature ranking is often absent or lacks robust statistical methods. DAFTAR-ML performs its tasks with this outcome in mind. Model training is robust, using nested cross-validation and hyperparameter tuning. Instead of relying on native feature-importance scores, it employs SHAP (SHapley Additive exPlanations) to quantify feature importance. The pipeline also produces comprehensive results, including publication-quality visualizations.

Melie, Tina [Lawrence Berkeley National Laboratory↗

DOE OSTI2022

HT-SIP: a semi-automated stable isotope probing pipeline identifies cross-kingdom interactions in the hyphosphere of arbuscular mycorrhizal fungi

Abstract Background Linking the identity of wild microbes with their ecophysiological traits and environmental functions is a key ambition for microbial ecologists. Of many techniques that strive for this goal, Stable-isotope probing—SIP—remains among the most comprehensive for studying whole microbial communities in situ. In DNA-SIP, actively growing microorganisms that take up an isotopically heavy substrate build heavier DNA, which can be partitioned by density into multiple fractions and sequenced. However, SIP is relatively low throughput and requires significant hands-on labor. We designed and tested a semi-automated, high-throughput SIP (HT-SIP) pipeline to support well-replicated, temporally resolved amplicon and metagenomics experiments. We applied this pipeline to a soil microhabitat with significant ecological importance—the hyphosphere zone surrounding arbuscular mycorrhizal fungal (AMF) hyphae. AMF form symbiotic relationships with most plant species and play key roles in terrestrial nutrient and carbon cycling. Results Our HT-SIP pipeline for fractionation, cleanup, and nucleic acid quantification of density gradients requires one-sixth of the hands-on labor compared to manual SIP and allows 16 samples to be processed simultaneously. Automated density fractionation increased the reproducibility of SIP gradients compared to manual fractionation, and we show adding a non-ionic detergent to the gradient buffer improved SIP DNA recovery. We applied HT-SIP to 13 C-AMF hyphosphere DNA from a 13 CO 2 plant labeling study and created metagenome-assembled genomes (MAGs) using high-resolution SIP metagenomics (14 metagenomes per gradient). SIP confirmed the AMF Rhizophagus intraradices and associated MAGs were highly enriched (10–33 atom% 13 C), even though the soils’ overall enrichment was low (1.8 atom% 13 C). We assembled 212 13 C-hyphosphere MAGs; the hyphosphere taxa that assimilated the most AMF-derived 13 C were from the phyla Myxococcota, Fibrobacterota, Verrucomicrobiota, and the ammonia-oxidizing archaeon genus Nitrososphaera . Conclusions Our semi-automated HT-SIP approach decreases operator time and improves reproducibility by targeting the most labor-intensive steps of SIP—fraction collection and cleanup. We illustrate this approach in a unique and understudied soil microhabitat—generating MAGs of actively growing microbes living in the AMF hyphosphere (without plant roots). The MAGs’ phylogenetic composition and gene content suggest predation, decomposition, and ammonia oxidation may be key processes in hyphosphere nutrient cycling.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

A miniaturized feedstocks-to-fuels pipeline for screening the efficiency of deconstruction and microbial conversion of lignocellulosic biomass

Sustainably grown biomass is a promising alternative to produce fuels and chemicals and reduce the dependency on fossil energy sources. However, the efficient conversion of lignocellulosic biomass into biofuels and bioproducts often requires extensive testing of components and reaction conditions used in the pretreatment, saccharification, and bioconversion steps. This restriction can result in a significant and unwieldy number of combinations of biomass types, solvents, microbial strains, and operational parameters that need to be characterized, turning these efforts into a daunting and time-consuming task. Here we developed a high-throughput feedstocks-to-fuels screening platform to address these challenges. The result is a miniaturized semi-automated platform that leverages the capabilities of a solid handling robot, a liquid handling robot, analytical instruments, and a centralized data repository, adapted to operate as an ionic-liquid-based biomass conversion pipeline. The pipeline was tested by using sorghum as feedstock, the biocompatible ionic liquid cholinium phosphate as pretreatment solvent, a “one-pot” process configuration that does not require ionic liquid removal after pretreatment, and an engineered strain of the yeast Rhodosporidium toruloides that produces the jet-fuel precursor bisabolene as a conversion microbe. By the simultaneous processing of 48 samples, we show that this configuration and reaction conditions result in sugar yields (~70%) and bisabolene titers (~1500 mg/L) that are comparable to the efficiencies observed at larger scales but require only a fraction of the time. We expect that this Feedstocks-to-Fuels pipeline will become an effective tool to screen thousands of bioenergy crop and feedstock samples and assist process optimization efforts and the development of predictive deconstruction approaches.

09 BIOMASS FUELS↗

DOE OSTI2022

Simultaneous flow of zero-carbon and conventional fuel liquids through Trans-Alaska Pipeline System (Abstract)

The purpose of this CRADA is to address key technical challenges specific to the transport of ammonia, a promising carbon-free fuel and hydrogen-carrier, through crude oil pipelines. Specifically, the project seeks to develop novel technologies for preparing liquid ammonia/ hydrocarbon mixtures for the dual purpose of (i) pipeline transport, and (ii) developing advanced marine fuel blends. By demonstrating compatibility with the Trans-Alaskan Pipeline System (TAPS) ammonia/oil blends may improve access to stranded natural gas and help overcome low-flow issues associated with declining oil productivity. In Alaska, this technology enhances the capabilities of TAPS allowing it to function as a statewide “Hydrogen Highway” for exporting green or blue ammonia. This is strategically important for Alaska which lacks statewide electrical transmission infrastructure but contains vast renewable and fossil resources located in remote regions with few local markets. Nationally, Alaska-sourced green (hydropower) or blue ammonia has the potential to improve reliability of a Pacific Northwest hydrogen storage hub as a hydrogen-carrier by helping to overcome seasonality of green hydrogen produced from solar or wind energy. Globally, this technology project has significant potential to improve the safety and efficacy of ammonia-rich fuel compositions for use in maritime propulsion and other mid-sized engines.

10 SYNTHETIC FUELS↗

DOE OSTI2023

Field Testing of Self-Healing Metallic Coatings for Internal Corrosion Protection of Natural Gas Pipelines

Internal corrosion occurs in natural gas pipelines primarily due to the presence of water, carbon dioxide, and hydrogen sulfide. Internal corrosion can eventually result in leakage, cracks, and rupture of the pipeline. The objective of this work is to mitigate internal corrosion in steel pipelines transporting natural gas using cold spray coatings. The corrosion behavior of carbon steel coated with self-healing zinc chromium (ZnCr) and zinc niobium (ZnNb) cold spray coatings was investigated in a natural gas environment. For comparison purposes, hot-dip galvanized steel (HDGS) was tested under the same conditions as cold spray coatings. The field test was performed at the NW Natural gas storage facility in Mist, Oregon. The field test was conducted in a 6-inch diameter pipe transporting wet natural gas out of an underground storage well at 500 psi and 4.4 °C. The coupons were tested for 15 and 32 days under stagnant and flow conditions, respectively. Weight loss method was used to measure the corrosion rate of metallic coatings. Post-corrosion surface characterization was performed on the specimen using a scanning electron microscopy (SEM) equipped with energy dispersive X-ray spectroscopy (EDS). Crystalline phases were determined by X-ray diffraction (XRD). The field-test results confirmed that the metallic coatings provided corrosion protection of carbon steel exposed to wet natural gas under stagnant and flow conditions. The formation of the ZnCO 3 layer on top of ZnNb and ZnCr coatings led to passivation of the coatings which helps to reduce the self-corrosion. These layers form a barrier for diffusion of corrosive species to the surface.

03 NATURAL GAS↗

DOE OSTI2023

A Data Processing Pipeline for Socio-Technical Network Analysis [Slides]

With the rapid adoption of emerging technologies, there is a need to catalog and model sociotechnical interdependencies that have been historically used to influence the operation of Critical Infrastructure networks including the impacts of mergers and acquisitions, hostile takeovers, and foreign investment. Our research intends to address this need with two primary contributions. First, we have developed a data curation and processing pipeline to generate sociotechnical networks extracted from a variety of data sources including SEC filings and infrastructure asset databases. The pipeline, implemented in Apache Airflow, extracts and normalizes the representation of entities and relations, specified within ontologies. Second, networks produced by our pipeline enable the development of graph-theoretic metrics that consider the properties of network components in addition to its topology. Measures of network complexity, such as degree distribution, reachability analyses, temporal analysis, and community detection may be adapted to indicate adversarial organizational influence. Our intent is to provide an extensible, machine-actionable approach to quickly communicate such models, reproduce previous results, and adapt them to new, unanticipated situations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Simultaneous flow of zero-carbon and conventional fuel liquids through Trans-Alaska Pipeline System [Abstract]

The purpose of this CRADA is to address key technical challenges specific to the transport of ammonia, a promising carbon-free fuel and hydrogen-carrier, through crude oil pipelines. Specifically, the project seeks to develop novel technologies for preparing liquid ammonia/ hydrocarbon mixtures for the dual purpose of (i) pipeline transport, and (ii) developing advanced marine fuel blends. By demonstrating compatibility with the Trans-Alaskan Pipeline System (TAPS) ammonia/oil blends may improve access to stranded natural gas and help overcome low-flow issues associated with declining oil productivity. In Alaska, this technology enhances the capabilities of TAPS allowing it to function as a statewide “Hydrogen Highway” for exporting green or blue ammonia. This is strategically important for Alaska which lacks statewide electrical transmission infrastructure but contains vast renewable and fossil resources located in remote regions with few local markets. Nationally, Alaska-sourced green (hydropower) or blue ammonia has the potential to improve reliability of a Pacific Northwest hydrogen storage hub as a hydrogen-carrier by helping to overcome seasonality of green hydrogen produced from solar or wind energy. Globally, this technology project has significant potential to improve the safety and efficacy of ammonia-rich fuel compositions for use in maritime propulsion and other mid-sized engines.

08 HYDROGEN↗

DOE OSTI2024

FECM/NETL Hydrogen Pipeline Cost Model (2024): Description and User’s Manual

The FECM/NETL Hydrogen Pipeline Cost Model (H2_P_COM) estimates costs for transporting gaseous hydrogen in a pipeline from a source, such as a hydrogen production facility, to a final destination which may be a user of the hydrogen or a distribution center where hydrogen in the pipeline is diverted to multiple end users. This document provides two main functions. First, the document describes the equations and algorithms that are used by the model to calculate technical quantities (such as the minimum inner pipe diameter needed to transport a user-specified H 2 mass flow rate a specified distance) and engineering-economic quantities (such as capital costs, operating costs, and cash flows). Second, the document is a user’s manual for the model that describes the procedures the user must follow to run the model. The model can be accessed at this URL: https://www.netl.doe.gov/energy-analysis/details?id=db897190-8e26-40b1-9535-ee78ac934193

08 HYDROGEN↗

DOE OSTI2024

Evaluating the Nation's Pipeline Infrastructure with NETL's Advanced Infrastructure Integrity Model (AIIM)

This poster is a part of BIL-EDX4CCS Task 36: Advanced Infrastructure Integrity Modeling to Evaluate Existing Energy Infrastructure Reusability and Risk, the goal of which is to produce a smart tool that will assess existing energy infrastructure reusability and risk using the Advanced Infrastructure Integrity Model (AIIM). This model forecasts lifespan and potential risk using a multitude of factors such as incidents reports, structural characteristics, and the surrounding environment. The project aims to provide scientific insights for a better understanding of carbon storage (CS), potential to support CS stakeholder needs, national decarbonization, and mitigating climate change. AIIM will utilize an energy infrastructure database as its input, developed by acquiring publicly available data as well as NETL derived products. These resources include incidents, geohazards, and infrastructure variables. Soil data in the form of rasters and pipeline incident reports were processed and a script was developed to count the number of times features such as roads, railroads, and rivers intersected with pipeline segments which were then converted to points. Distance to oil and natural gas wells, petroleum ports, intermodal freight facilities, and geologic structures were also calculated. After data preparation and quality control was completed, the data was integrated into the pipeline points. Once models are complete, a smart tool will be created in the form of an online dashboard.

Malay, Caleb↗

DOE OSTI2024

Techno-economic Model and Analysis for Hydrogen (H2) Pipeline Transportation

Presentation at the 9th ELAEE (Latin American Energy Economics Meeting) July 28th – 30th, 2024 in PUC-Rio, Rio de Janeiro, Brazil. The presentation highlights the FECM/NETL Hydrogen Pipeline Cost Model (H2_P_COM). The model estimates costs for transporting gaseous hydrogen in a pipeline from a source, such as a hydrogen production facility, to a final destination which may be a user of the hydrogen or a distribution center where hydrogen in the pipeline is diverted to multiple end users.

Cunha, Luciane↗

DOE OSTI2025

Investigation of the Mechanical Degradation of Zinc-Based Cold Spray Coatings for Steel Pipelines

Internal corrosion in wet natural gas is a big challenge in the oil and gas industry due to corrosive constituents such as carbon dioxide (CO2), hydrogen sulfide (H2S), other forms of sulfur, and water in the gas stream. To mitigate internal corrosion, zinc-based cold spray coatings were designed for use in natural gas pipelines to increase the lifespan of the pipeline network. However, one of the requirements in designing internal coatings is the resistance of the coatings to mechanical forces applied on the pipeline's internal wall during pigging operations. These forces are primarily compressive and shear/friction forces. This study examines material properties that must be considered when evaluating mechanical considerations.

adhesion↗

DOE OSTI2025

Summary of Carbon Dioxide Pipeline Systems and Incident Data in North America

Pipelines are historically seen as the primary transportation mode for carbon dioxide (CO 2 ) streams in the context of carbon capture and storage (CCS) and oil and gas industries. Pipeline transmission of CO 2 over longer distances is regarded as most efficient and economical when the CO 2 is in the dense phase, i.e., in liquid or supercritical regime, due to transporting CO 2 in dense phase that allows for a smaller-diameter pipeline to move a given flow, which optimizes project cost.

42 ENGINEERING↗

DOE OSTI2025

The DECADE cosmic shear project III: validation of analysis pipeline using spatially inhomogeneous data

We present the pipeline for the cosmic shear analysis of the Dark Energy Camera All Data Everywhere (DECADE) weak lensing dataset: a catalog consisting of 107 million galaxies observed by the Dark Energy Camera (DECam) in the northern Galactic cap. The catalog derives from a large number of disparate observing programs and is therefore more inhomogeneous across the sky compared to existing lensing surveys. First, we use simulated data-vectors to show the sensitivity of our constraints to different analysis choices in our inference pipeline, including sensitivity to residual systematics. Next we use simulations to validate our covariance modeling for inhomogeneous datasets. Finally, we show that our choices in the end-to-end cosmic shear pipeline are robust against inhomogeneities in the survey, by extracting relative shifts in the cosmology constraints across different subsets of the footprint/catalog and showing they are all consistent within 1σ to 2σ. This is done for forty-six subsets of the data and is carried out in a fully consistent manner: for each subset of the data, we re-derive the photometric redshift estimates, shear calibrations, survey transfer functions, the data vector, measurement covariance, and finally, the cosmological constraints. Our results show that existing analysis methods for weak lensing cosmology can be fairly resilient towards inhomogeneous datasets. This also motivates exploring a wider range of image data for pursuing such cosmological constraints.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

LoVoCCS. I. Survey Introduction, Data Processing Pipeline, and Early Science Results

We present the Local Volume Complete Cluster Survey (LoVoCCS; we pronounce it as "low-vox" or "law-vox," with stress on the second syllable), an NSF's National Optical-Infrared Astronomy Research Laboratory survey program that uses the Dark Energy Camera to map the dark matter distribution and galaxy population in 107 nearby (0.03 < z < 0.12) X-ray luminous ([0.1–2.4 keV] L X500 > 10 44 erg s –1 ) galaxy clusters that are not obscured by the Milky Way. The survey will reach Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) Year 1–2 depth (for galaxies r = 24.5, i = 24.0, signal-to-noise ratio (S/N) > 20; u = 24.7, g = 25.3, z = 23.8, S/N > 10) and conclude in ~2023 (coincident with the beginning of LSST science operations), and will serve as a zeroth-year template for LSST transient studies. We process the data using the LSST Science Pipelines that include state-of-the-art algorithms and analyze the results using our own pipelines, and therefore the catalogs and analysis tools will be compatible with the LSST. We demonstrate the use and performance of our pipeline using three X-ray luminous and observation-time complete LoVoCCS clusters: A3911, A3921, and A85. A3911 and A3921 have not been well studied previously by weak lensing, and we obtain similar lensing analysis results for A85 to previous studies.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Testing the LSST Difference Image Analysis Pipeline Using Synthetic Source Injection Analysis

Abstract We evaluate the performance of the Legacy Survey of Space and Time Science Pipelines Difference Image Analysis (DIA) on simulated images. By adding synthetic sources to galaxies on images, we trace the recovery of injected synthetic sources to evaluate the pipeline on images from the Dark Energy Science Collaboration Data Challenge 2. The pipeline performs well, with efficiency and flux accuracy consistent with the signal-to-noise ratio of the input images. We explore different spatial degrees of freedom for the Alard–Lupton polynomial-Gaussian image subtraction kernel and analyze for trade-offs in efficiency versus artifact rate. Increasing the kernel spatial degrees of freedom reduces the artifact rate without loss of efficiency. The flux measurements with different kernel spatial degrees of freedom are consistent. We also here provide a set of DIA flags that substantially filter out artifacts from the DIA source table. We explore the morphology and possible origins of the observed remaining subtraction artifacts and suggest that given the complexity of these artifact origins, a convolution kernel with a set of flexible bases with spatial variation may be needed to yield further improvements.

Liu, S. (ORCID:0000000244612143)↗

DOE OSTI2023

A Data Processing Pipeline for Adversarial Socio-Technical Network Analysis

With the rapid adoption of emerging technologies, there is a need to catalog and model sociotechnical interdependencies that have been historically used to influence the operation of Critical Infrastructure networks including the impacts of mergers and acquisitions, hostile takeovers, and foreign investment. Our research intends to address this need with two primary contributions. First, we have developed a data curation and processing pipeline to generate sociotechnical networks extracted from a variety of data sources including SEC filings and infrastructure asset databases. The pipeline, implemented in Apache Airflow, extracts and normalizes the representation of entities and relations, specified within ontologies. Our intent is to provide an extensible, machine-actionable approach to quickly communicate such models, reproduce previous results, and adapt them to new, unanticipated situations. Second, networks produced by our pipeline enable the development of graph-theoretic metrics that consider the properties of network components in addition to its topology. Metadata associated with network components---whether semantic, temporal, or geospatial---affects the alignment of generated networks with assumptions underlying complexity metrics. Validation of generated networks relative to component types defined by an ontology, may allow the research community to adapt metrics to the semantics of the domains being studied. Generated networks may be processed as knowledge, dynamic, or spatial graphs and enables a variety of analyses including automated reasoning and measures of network complexity. Automated reasoning views extracted entities and relations as a knowledge graph; this enables application of inference rules that represent historically-attested adversarial business methods and applies that behavior to a specific geographic context. Measures of network complexity, including degree distribution, reachability analyses, temporal analysis, and community detection can be adapted to indicate adversarial organizational influence.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Evaluation of NETL’s Self-Healing Metallic Coating for Internal Corrosion Protection of Natural Gas Pipelines: Field Test

Steel pipelines are a safe, reliable, and affordable way to transport natural gas. However, the presence of impurities (e.g., water, carbon dioxide, and hydrogen sulfide) in the natural gas can cause internal corrosion, which can lead to pipeline leaks and failure. This paper reports on field tests of an innovative self-healing, corrosion-resistant, metallic coating developed at the National Energy Technology Laboratory (NETL) for protecting the interior surfaces of pipelines.

carbon dioxide (CO2)↗