SEARCH · Engineering Papers

Results for “pipelining”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

Control over colloidal nanocrystal morphology (size, size distribution, and shape) is important for tailoring the functionality of individual nanocrystals and their ensemble behavior. Despite this, traditional methods to quantify nanocrystal morphology are laborious. New developments in automated morphology classification will accelerate these analyses but the assessment of machine learning models is limited by human accuracy for ground truth, causing even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic image rendering to solve the ground truth problem of nanocrystal morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space, we trained a convolutional neural network to link unique morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology from experimental transmission electron microscopy (TEM) images. Using improved computer vision techniques, 42,650 nanocrystals were identified, assessed, and labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-practice manual measurements. Further, a classification algorithm was trained with a prediction accuracy of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr 3 nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on morphology. This pipeline provides a simple, efficient, and unbiased method to quantify nanocrystal morphology and represents a practical route to construct large datasets with an absolute ground truth for training unbiased morphology-based machine learning algorithms.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

DOE OSTI2023

Assessing Compatibility of Natural Gas Pipeline Materials with Hydrogen, CO 2 , and Ammonia

Here, in this study, we examine the efficacy of repurposing natural gas (NG) pipelines for transporting hydrogen blends (with NG), ammonia, and CO 2 (gaseous and supercritical) from the standpoint of materials compatibility. Some information pertaining to component performance is also included, especially those components critical for pressurization and monitoring of flow. A listing of critical pipeline components and materials was developed, and their compatibilities was assessed for each fluid or gas type based on known compatibilities. Results indicate that pipeline materials should be suitable for gaseous CO 2 and anhydrous ammonia, but hydrogen blends greater than 12% may be problematic. Current compressor/regulator stations will not be suitable for use with either supercritical CO 2 or ammonia. Important knowledge gaps were identified, including (1) polymer performance with hydrogen/NG blends at low pressures, (2) compressor/regulator station polymers and epoxy coating materials with supercritical CO 2 , and (3) metal performances of hydrogen/NG blends at low pressures.

03 NATURAL GAS↗

DOE OSTI2025

Using active learning to improve quasar identification for the DESI spectra processing pipeline

The Dark Energy Spectroscopic Instrument (DESI) survey uses an automatic spectral classification pipeline to classify spectra. QuasarNET is a convolutional neural network used as part of this pipeline originally trained using data from the Baryon Oscillation Spectroscopic Survey (BOSS). In this paper we implement an active learning algorithm to optimally select spectra to use for training a new version of the QuasarNET weights file using only DESI data, with the goal of improving classification accuracy. This active learning algorithm includes a novel outlier rejection step using a Self-Organizing Map to ensure we label spectra representative of the larger quasar sample observed in DESI. We perform two iterations of the active learning pipeline, assembling a final dataset of 5600 labeled spectra, a small subset of the approximately 1.3 million quasar targets in DESI's Data Release 1. When splitting the spectra into training and validation subsets we achieve similar performance to the previously trained weights file in completeness and purity calculated on the validation dataset but do so with less than one tenth of the amount of training data. The new weights also more consistently classify objects in the same way when used on unlabeled data compared to the old weights file. In the process of improving QuasarNET's classification accuracy we discovered a systemic error in QuasarNET's redshift estimation and used our findings to improve our understanding of QuasarNET's redshifts.

Machine learning↗

DOE OSTI2024

The FRB-searching Pipeline of the Tianlai Cylinder Pathfinder Array

This paper presents the design, calibration, and survey strategy of the Fast Radio Burst (FRB) digital backend and its real-time data processing pipeline employed in the Tianlai Cylinder Pathfinder Array. The array, consisting of three parallel cylindrical reflectors and equipped with 96 dual-polarization feeds, is a radio interferometer array designed for conducting drift scans of the northern celestial semi-sphere. The FRB digital backend enables the formation of 96 digital beams, effectively covering an area of approximately 40 square degrees with the 3 dB beam. Our pipeline demonstrates the capability to conduct an automatic search of FRBs, detecting at quasi-real-time and classifying FRB candidates automatically. The current FRB searching pipeline has an overall recall rate of 88%. During the commissioning phase, we successfully detected signals emitted by four well-known pulsars: PSR B0329+54, B2021+51, B0823+26, and B2020+28. We report the first discovery of an FRB by our array, designated as FRB 20220414A. We also investigate the optimal arrangement for the digitally formed beams to achieve maximum detection rate by numerical simulation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2025

Poplar: a phylogenomics pipeline

Motivation Generating phylogenomic trees from the genomic data is essential in understanding biological systems. Each step of this complex process has received extensive attention and has been significantly streamlined over the years. Given the public availability of data, obtaining genomes for a wide selection of species is straightforward. However, analyzing that data to generate a phylogenomic tree is a multistep process with legitimate scientific and technical challenges, often requiring a significant input from a domain-area scientist. Results We present Poplar, a new, streamlined computational pipeline, to address the computational logistical issues that arise when constructing the phylogenomic trees. It provides a framework that runs state-of-the-art software for essential steps in the phylogenomic pipeline, beginning from a genome with or without an annotation, and resulting in a species tree. Running Poplar requires no external databases. In the execution, it enables parallelism for execution for clusters and cloud computing. The trees generated by Poplar match closely with state-of-the-art published trees. The usage and performance of Poplar is far simpler and quicker than manually running a phylogenomic pipeline. Availability and implementation Freely available on GitHub at https://github.com/sandialabs/poplar. Implemented using Python and supported on Linux.

Koning, Elizabeth [Sandia National Laboratories (S↗

DOE OSTI2026

Feature Engineering and Ensemble Methods for Imbalanced ICS Intrusion Detection: Pipeline Audit and Constrained Evaluation

Industries are becoming increasingly connected and are more vulnerable to cyberattacks due to the widened attack surface. Industrial Control Systems (ICS) are among the most critical sectors that malicious actors can target, as such attacks can cause significant operational disruption and physical damage. It is imperative to detect such attacks as early as possible. This paper evaluates constraint-conditioned optimistic performance estimates for traditional ML models in ICS intrusion detection (i.e., estimates obtained under contiguous, non-shuffled temporal evaluation without test-set alteration, but with pre-split feature engineering that may introduce temporal leakage, due to dataset constraints). Our findings are threefold. First, we quantify how iterative feature engineering affects tree-based ensemble performance and examine how pipeline decisions (split strategy, sampling scope, and cleaning policy) can inflate or reduce reported IDS results under constraint-bound evaluation. Second, we compare intrinsic class-imbalance handling across ensemble models. Third, under our current pipeline constraints (including pre-split feature engineering), CatBoost achieves the best performance on Water Storage Tank (accuracy: 0.9831, class-1 F1: 0.9682), while Light- GBM achieves the best performance on Gas Pipeline (accuracy: 0.9618, class-1 F1: 0.9086).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

In-Situ and Ex-Situ Studies on the Morphology Changes of Polymer Pipeline Materials for Use in Hydrogen Gas Environments

The US natural gas infrastructure is a national asset that could be used to deliver hydrogen and hydrogen blends of natural gas as a pathway to reduce carbon emissions. The distribution system comprises nearly 50% plastic pipe composed of medium- and high-density polyethylene materials (MDPE and HDPE). While these materials perform adequately for natural gas, research on their hydrogen compatibility is essential to understand if any immediate and long-term risks are associated with hydrogen addition. The Blended Gas CRADA, a HyBlend project, has established a comprehensive test method for evaluating MDPE and HDPE of various plastic resin compositions of pipeline material in pure hydrogen and 20% hydrogen/80% methane blends. Both in-situ and ex-situ measurements were performed to capture hydrogen-induced changes in the polyethylene material's crystalline, amorphous and their interphase regions. We investigated MDPE and HDPE pipeline materials made from different polymer resin systems to evaluate the effects of hydrogen gas. The materials were characterized by their density, diffusion coefficient, free volume ratio, and degree of crystallinity. Various advanced characterization methods, including in situ NMR, ex situ XRD, ex situ DSC, and ex situ TDA, were used to analyze the effects of changes in crystalline, amorphous, and interphase regions due to gas exposure. Time-dependent post-decompression quasi-static tensile tests were conducted to explore the effects of gas exposure time on the mechanical behavior of the pipe materials. This work will highlight the time sensitivities during and after gas exposure. The correlation between gas-induced polyethylene morphology changes and the associated material performance will be addressed for the intended applications. These studies will show that polyethylene resin composition and material exposure are important factors when considering whether hydrogen gas affects pipeline materials positively or negatively.

Simmons, Kevin L.↗

DOE OSTI2023

Multi-choice Viromics Pipeline (MVP) v1

MVP stands for Multi-choice Viromics Pipeline. It is a pipeline that utilizes a suite of state-of-art tools: geNomad to identify viruses, proviruses, and plasmids in sequencing data, CheckV to assess the quality, and completeness of identified viral genomes, including identification of host contamination for integrated proviruses, A custom code for a rapid genome clustering based on pairwise ANI, Bowtie2, Samtools, and CoverM to calculate coverage of individual viral genomes by read mapping, A custom code to create a vOTU table of abundance, MMseqs2 to compare viral proteins to multiple databases. It provides a quick, and intuitive pipeline to get viral sequences and corresponding properties that can be used for downstream analyses.

Roux, Simon↗

DOE OSTI2025

Data Agnostic Feature-Target Analysis & Ranking Machine Learning Pipeline (DAFTAR-ML) v0.1.0

DAFTAR-ML is a specialized machine-learning pipeline that identifies relevant features based on their relationship to a target variable. Many ML pipelines focus solely on prediction, and feature ranking is often absent or lacks robust statistical methods. DAFTAR-ML performs its tasks with this outcome in mind. Model training is robust, using nested cross-validation and hyperparameter tuning. Instead of relying on native feature-importance scores, it employs SHAP (SHapley Additive exPlanations) to quantify feature importance. The pipeline also produces comprehensive results, including publication-quality visualizations.

Melie, Tina [Lawrence Berkeley National Laboratory↗

DOE OSTI2022

HT-SIP: a semi-automated stable isotope probing pipeline identifies cross-kingdom interactions in the hyphosphere of arbuscular mycorrhizal fungi

Abstract Background Linking the identity of wild microbes with their ecophysiological traits and environmental functions is a key ambition for microbial ecologists. Of many techniques that strive for this goal, Stable-isotope probing—SIP—remains among the most comprehensive for studying whole microbial communities in situ. In DNA-SIP, actively growing microorganisms that take up an isotopically heavy substrate build heavier DNA, which can be partitioned by density into multiple fractions and sequenced. However, SIP is relatively low throughput and requires significant hands-on labor. We designed and tested a semi-automated, high-throughput SIP (HT-SIP) pipeline to support well-replicated, temporally resolved amplicon and metagenomics experiments. We applied this pipeline to a soil microhabitat with significant ecological importance—the hyphosphere zone surrounding arbuscular mycorrhizal fungal (AMF) hyphae. AMF form symbiotic relationships with most plant species and play key roles in terrestrial nutrient and carbon cycling. Results Our HT-SIP pipeline for fractionation, cleanup, and nucleic acid quantification of density gradients requires one-sixth of the hands-on labor compared to manual SIP and allows 16 samples to be processed simultaneously. Automated density fractionation increased the reproducibility of SIP gradients compared to manual fractionation, and we show adding a non-ionic detergent to the gradient buffer improved SIP DNA recovery. We applied HT-SIP to 13 C-AMF hyphosphere DNA from a 13 CO 2 plant labeling study and created metagenome-assembled genomes (MAGs) using high-resolution SIP metagenomics (14 metagenomes per gradient). SIP confirmed the AMF Rhizophagus intraradices and associated MAGs were highly enriched (10–33 atom% 13 C), even though the soils’ overall enrichment was low (1.8 atom% 13 C). We assembled 212 13 C-hyphosphere MAGs; the hyphosphere taxa that assimilated the most AMF-derived 13 C were from the phyla Myxococcota, Fibrobacterota, Verrucomicrobiota, and the ammonia-oxidizing archaeon genus Nitrososphaera . Conclusions Our semi-automated HT-SIP approach decreases operator time and improves reproducibility by targeting the most labor-intensive steps of SIP—fraction collection and cleanup. We illustrate this approach in a unique and understudied soil microhabitat—generating MAGs of actively growing microbes living in the AMF hyphosphere (without plant roots). The MAGs’ phylogenetic composition and gene content suggest predation, decomposition, and ammonia oxidation may be key processes in hyphosphere nutrient cycling.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

A miniaturized feedstocks-to-fuels pipeline for screening the efficiency of deconstruction and microbial conversion of lignocellulosic biomass

Sustainably grown biomass is a promising alternative to produce fuels and chemicals and reduce the dependency on fossil energy sources. However, the efficient conversion of lignocellulosic biomass into biofuels and bioproducts often requires extensive testing of components and reaction conditions used in the pretreatment, saccharification, and bioconversion steps. This restriction can result in a significant and unwieldy number of combinations of biomass types, solvents, microbial strains, and operational parameters that need to be characterized, turning these efforts into a daunting and time-consuming task. Here we developed a high-throughput feedstocks-to-fuels screening platform to address these challenges. The result is a miniaturized semi-automated platform that leverages the capabilities of a solid handling robot, a liquid handling robot, analytical instruments, and a centralized data repository, adapted to operate as an ionic-liquid-based biomass conversion pipeline. The pipeline was tested by using sorghum as feedstock, the biocompatible ionic liquid cholinium phosphate as pretreatment solvent, a “one-pot” process configuration that does not require ionic liquid removal after pretreatment, and an engineered strain of the yeast Rhodosporidium toruloides that produces the jet-fuel precursor bisabolene as a conversion microbe. By the simultaneous processing of 48 samples, we show that this configuration and reaction conditions result in sugar yields (~70%) and bisabolene titers (~1500 mg/L) that are comparable to the efficiencies observed at larger scales but require only a fraction of the time. We expect that this Feedstocks-to-Fuels pipeline will become an effective tool to screen thousands of bioenergy crop and feedstock samples and assist process optimization efforts and the development of predictive deconstruction approaches.

09 BIOMASS FUELS↗

DOE OSTI2021

Literature Review of Electromagnetic Pulse (EMP) and Geomagnetic Disturbance (GMD) Effects on Oil and Gas Pipeline Systems

This document summarizes the findings of a review of published literature regarding the potential impacts of electromagnetic pulse (EMP) and geomagnetic disturbance (GMD) phenomena on oil and gas pipeline systems. The impacts of telluric currents on pipelines and their associated cathodic protection systems has been well studied. The existing literature describes implications for corrosion protection system design and monitoring to mitigate these impacts. Effects of an EMP on pipelines is not a thoroughly explored subject. Most directly related articles only present theoretical models and approaches rather than specific analyses and in-field testing. Literature on SCADA components and EMP is similarly sparse and the existing articles show a variety of impacts to control system components that range from upset and damage to no effect. The limited research and the range of observed impacts for the research that has been published suggests the need for additional work on GMD and EMP and natural gas SCADA components.

02 PETROLEUM↗

DOE OSTI2022

Machine Learning Modeling Pipeline for Extracting Nuclear Proliferation Events of Interest from Open Data Sources (U)

In FY2020, the Savannah River National Laboratory (SRNL) and the Sanghani Center for Artificial Intelligence and Data Analytics at Virginia Polytechnic Institute and State University entered a collaboration funded by Department of Energy’s (DOE) Office of Defense Nuclear Nonproliferation Research and Development. The project’s mission was to take the first steps toward developing a demonstration prototype system that uses multiple machine learning and data analytics methods on largescale open data sources to identify new, developing, and/or undeclared nuclear programs. Given the SRNL team’s on-site perspective of events culminating in the DOE’s decision to pursue the Savannah River Plutonium Processing Facility (SRPPF), the team targeted the identification of events and indicators in retrospective datasets that pointed to the activity of “fissile core fabrication at the Savannah River Site” prior to the official announcement in May of 2018. A preliminary modeling pipeline was developed in FY20 that showed the datasets contained adequate signal for continuation of efforts. In FY21, a modular demonstration prototype modeling pipeline has continued in development for two text-based data sources: a broad internet archive (Webhose Ltd.) and a decahose Twitter database (i.e., a global sampling of one in every ten Tweets). The techniques that have been developed rely on graph theory and anomaly detection to identify contextual shifts in key words and phrases at various points in time such that indicators of events of interest could be identified and subsequently, events could be extracted from the corpuses. The foundational concept behind the approaches is that contextual shifts in key words and phrases can act as indicators of events of interest. Both datasets have proven successful in extracting events of interest related to pit production at the Savannah River Site prior to the official announcement. In addition, the pipelines have generated a wide range of events broadly summarized as: the awarding of DOE contracts at major sites, DOE investments in various programs, accidents at DOE national laboratories, speculations about the fate of pit production in the DOE complex, domestic and international shipments and receipts of nuclear materials at DOE sites, termination of non-proliferation agreements with Russia, termination of MOX, new weapons development approvals/testing, nuclear posture reviews, major DOE cleanup/production milestones, political opinions, and nuclear watch groups’ opinions, among many others.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

University Pipeline (with UNM) for Criticality Safety Professionals [Slides]

The scope of the University Pipeline is to provide students with knowledge of NCS and to inspire students to pursue a career in NCS after graduation. There are several benefits of the University Pipeline: reduced training time and costs; interested students that naturally self-sort and pursue the discipline at the university level; and a pipeline of criticality safety candidates readily available within the DOE Complex so that unexpected organizational or mission changes can be reacted to with increased agility.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Simultaneous flow of zero-carbon and conventional fuel liquids through Trans-Alaska Pipeline System (Abstract)

The purpose of this CRADA is to address key technical challenges specific to the transport of ammonia, a promising carbon-free fuel and hydrogen-carrier, through crude oil pipelines. Specifically, the project seeks to develop novel technologies for preparing liquid ammonia/ hydrocarbon mixtures for the dual purpose of (i) pipeline transport, and (ii) developing advanced marine fuel blends. By demonstrating compatibility with the Trans-Alaskan Pipeline System (TAPS) ammonia/oil blends may improve access to stranded natural gas and help overcome low-flow issues associated with declining oil productivity. In Alaska, this technology enhances the capabilities of TAPS allowing it to function as a statewide “Hydrogen Highway” for exporting green or blue ammonia. This is strategically important for Alaska which lacks statewide electrical transmission infrastructure but contains vast renewable and fossil resources located in remote regions with few local markets. Nationally, Alaska-sourced green (hydropower) or blue ammonia has the potential to improve reliability of a Pacific Northwest hydrogen storage hub as a hydrogen-carrier by helping to overcome seasonality of green hydrogen produced from solar or wind energy. Globally, this technology project has significant potential to improve the safety and efficacy of ammonia-rich fuel compositions for use in maritime propulsion and other mid-sized engines.

10 SYNTHETIC FUELS↗

DOE OSTI2023

Field Testing of Self-Healing Metallic Coatings for Internal Corrosion Protection of Natural Gas Pipelines

Internal corrosion occurs in natural gas pipelines primarily due to the presence of water, carbon dioxide, and hydrogen sulfide. Internal corrosion can eventually result in leakage, cracks, and rupture of the pipeline. The objective of this work is to mitigate internal corrosion in steel pipelines transporting natural gas using cold spray coatings. The corrosion behavior of carbon steel coated with self-healing zinc chromium (ZnCr) and zinc niobium (ZnNb) cold spray coatings was investigated in a natural gas environment. For comparison purposes, hot-dip galvanized steel (HDGS) was tested under the same conditions as cold spray coatings. The field test was performed at the NW Natural gas storage facility in Mist, Oregon. The field test was conducted in a 6-inch diameter pipe transporting wet natural gas out of an underground storage well at 500 psi and 4.4 °C. The coupons were tested for 15 and 32 days under stagnant and flow conditions, respectively. Weight loss method was used to measure the corrosion rate of metallic coatings. Post-corrosion surface characterization was performed on the specimen using a scanning electron microscopy (SEM) equipped with energy dispersive X-ray spectroscopy (EDS). Crystalline phases were determined by X-ray diffraction (XRD). The field-test results confirmed that the metallic coatings provided corrosion protection of carbon steel exposed to wet natural gas under stagnant and flow conditions. The formation of the ZnCO 3 layer on top of ZnNb and ZnCr coatings led to passivation of the coatings which helps to reduce the self-corrosion. These layers form a barrier for diffusion of corrosive species to the surface.

03 NATURAL GAS↗

DOE OSTI2023

A Data Processing Pipeline for Socio-Technical Network Analysis [Slides]

With the rapid adoption of emerging technologies, there is a need to catalog and model sociotechnical interdependencies that have been historically used to influence the operation of Critical Infrastructure networks including the impacts of mergers and acquisitions, hostile takeovers, and foreign investment. Our research intends to address this need with two primary contributions. First, we have developed a data curation and processing pipeline to generate sociotechnical networks extracted from a variety of data sources including SEC filings and infrastructure asset databases. The pipeline, implemented in Apache Airflow, extracts and normalizes the representation of entities and relations, specified within ontologies. Second, networks produced by our pipeline enable the development of graph-theoretic metrics that consider the properties of network components in addition to its topology. Measures of network complexity, such as degree distribution, reachability analyses, temporal analysis, and community detection may be adapted to indicate adversarial organizational influence. Our intent is to provide an extensible, machine-actionable approach to quickly communicate such models, reproduce previous results, and adapt them to new, unanticipated situations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Simultaneous flow of zero-carbon and conventional fuel liquids through Trans-Alaska Pipeline System [Abstract]

08 HYDROGEN↗