Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipelining”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

MetaboDirect: an analytical pipeline for the processing of FT-ICR MS-based metabolomic data

Background: Microbiomes are now recognized as the main drivers of ecosystem function ranging from the oceans and soils to humans and bioreactors. However, a grand challenge in microbiome science is to characterize and quantify the chemical currencies of organic matter (i.e., metabolites) that microbes respond to and alter. Critical to this has been the development of Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS), which has drastically increased molecular characterization of complex organic matter samples, but challenges users with hundreds of millions of data points where readily available, user-friendly, and customizable software tools are lacking. Results: Here, we build on years of analytical experience with diverse sample types to develop MetaboDirect, an open-source, command-line-based pipeline for the analysis (e.g., chemodiversity analysis, multivariate statistics), visualization (e.g., Van Krevelen diagrams, elemental and molecular class composition plots), and presentation of direct injection high-resolution FT-ICR MS data sets after molecular formula assignment has been performed. When compared to other available FT-ICR MS software, MetaboDirect is superior in that it requires a single line of code to launch a fully automated framework for the generation and visualization of a wide range of plots, with minimal coding experience required. Among the tools evaluated, MetaboDirect is also uniquely able to automatically generate biochemical transformation networks (ab initio) based on mass differences (mass difference network-based approach) that provide an experimental assessment of metabolite connections within a given sample or a complex metabolic system, thereby providing important information about the nature of the samples and the set of microbial reactions or pathways that gave rise to them. Finally, for more experienced users, MetaboDirect allows users to customize plots, outputs, and analyses. Conclusion: Application of MetaboDirect to FT-ICR MS-based metabolomic data sets from a marine phage-bacterial infection experiment and a Sphagnum leachate microbiome incubation experiment showcase the exploration capabilities of the pipeline that will enable the research community to evaluate and interpret their data in greater depth and in less time. It will further advance our knowledge of how microbial communities influence and are influenced by the chemical makeup of the surrounding system. The source code and User’s guide of MetaboDirect are freely available through (https://github.com/Coayala/MetaboDirect) and (https://metabodirect.readthedocs.io/en/latest/), respectively.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Distributed fiber sensor and machine learning data analytics for pipeline protection against extrinsic intrusions and intrinsic corrosions

This paper presents an integrated technical framework to protect pipelines against both malicious intrusions and piping degradation using a distributed fiber sensing technology and artificial intelligence. A distributed acoustic sensing (DAS) system based on phase-sensitive optical time-domain reflectometry (φ-OTDR) was used to detect acoustic wave propagation and scattering along pipeline structures consisting of straight piping and sharp bend elbow. Signal to noise ratio of the DAS system was enhanced by femtosecond induced artificial Rayleigh scattering centers. Data harnessed by the DAS system were analyzed by neural network-based machine learning algorithms. The system identified with over 85% accuracy in various external impact events, and over 94% accuracy for defect identification through supervised learning and 71% accuracy through unsupervised learning.

Peng, Zhaoqiang↗
DOE OSTI2023

End-to-End Pipeline for Trigger Detection on Hit and Track Graphs

There has been a surge of interest in applying deep learning in particle and nuclear physics to replace labor-intensive offline data analysis with automated online machine learning tasks. This paper details a novel AI-enabled triggering solution for physics experiments in Relativistic Heavy Ion Collider and future Electron-Ion Collider. The triggering system consists of a comprehensive end-to-end pipeline based on Graph Neural Networks that classifies trigger events versus background events, makes online decisions to retain signal data, and enables efficient data acquisition. Here, the triggering system first starts with the coordinates of pixel hits lit up by passing particles in the detector, applies three stages of event processing (hits clustering, track reconstruction, and trigger detection), and labels all processed events with the binary tag of trigger versus background events. By switching among different objective functions, we train the Graph Neural Networks in the pipeline to solve multiple tasks: the edge-level track reconstruction problem, the edge-level track adjacency matrix prediction, and the graph-level trigger detection problem. We propose a novel method to treat the events as track-graphs instead of hit-graphs. This method focuses on intertrack relations and is driven by underlying physics processing. As a result, it attains a solid performance (around 72% accuracy) for trigger detection and outperforms the baseline method using hit-graphs by 2% higher accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Fatigue Performance of High-Strength Pipeline Steels and Their Welds in Hydrogen Gas Service

Objectives of the project include: Enable the use of high strength steel hydrogen pipelines, as significant cost savings can result by implementing high strength steels as compared to lower strength pipes. Demonstrate that girth welds in high-strength steel pipe exhibit fatigue performance similar to lower-strength steels in high-pressure hydrogen gas. Identify pathways for developing high-strength pipeline steels by establishing the relationship between microstructure constituents and hydrogen-accelerated fatigue crack growth (HA-FCG)

08 HYDROGEN↗
DOE OSTI2021

Remote methane sensor for emissions from pipelines and compressor stations using chirped-laser dispersion spectroscopy

Leak rates of methane (CH 4 ) from the natural gas supply chain result in lost profit from unsold product, public safety and property concerns due to potential explosion hazards, and a potentially large source of economic damages from legal liabilities. Yet large measurement challenges exist in identifying and quantifying CH 4 leak rates along the vast number and type of components in the natural gas supply chain. This is particularly true of the “midstream” components involved in the gathering, processing, compression, transmission, and storage of natural gas. This project developed and deployed new advances in chirped laser dispersion spectroscopy (CLaDS) to detect methane leaks from pipelines, compressor stations, and other midstream infrastructure from a remote position (standoff detection). The system was deployed from a van to measure fugitive methane leaks from a local compressor station as a proof-of-concept. The technique was validated through mobile laboratory measurements with in-situ sensors as well as controlled releases of methane. The system also mapped a plume from a controlled release of methane by tracking a small unmanned aerial system (sUAS) that carried a corner cube retroreflector which reflected the beam back to the instrument. The drone-based system quantified a leak rate to within 30% of the actual rate and localized the emission location within 5 m of the actual release location at standoff distances of 25-45 m. This sUAS-reflector tracking approach has benefits for mapping leak locations remotely using small drones flying around a facility. Benefits of a commercial sensor with these capabilities include reductions of leaks for pipeline operators (more profit), earlier detection of leaks to avoid catastrophic explosion hazards for public health and to mitigate property damage, and reduced methane emissions to the atmosphere (improving air quality).

03 NATURAL GAS↗
DOE OSTI2023

Building a Computational and Experimental Rapid Response Pipeline to Counter the Coronavirus Disease 2019 Outbreak and Emerging Biothreats

The COVID-19 pandemic underscored the promise of monoclonal antibody-based prophylactic and therapeutic drugs, especially where protective candidates can be rapidly identified or developed for emerging biothreats and escape variants. Current cutting-edge technologies for this purpose still rely on pathogen-exposed convalescent volunteers and a large screening effort to find a proverbial needle in a haystack. Computational design of protective antibodies based on pre-existing templates skips those requirements and allows for greater control over the breadth and target epitope, while also co-optimizing for potency and developability or other biophysical characteristics. We approached this problem by building and expanding an in vitro experimental rapid antibody production and characterization pipeline to support development of an autonomous, closed loop, active learning software system based on structural simulation and ground truth experimental data to design and evaluate antibody antigen interactions. Starting from early in the pandemic, we targeted SARS-CoV-2, the causative agent of COVID-19, by re-purposing neutralizing antibodies against SARS-CoV-1 that had been identified in the wake of that outbreak in the early 2000’s. We successfully re-targeted three different anti-SARS-CoV-1 antibodies to neutralize SARS-CoV-2 in vitro, where the antibodies were generated externally and tested through conventional binding and neutralization assays internally or with collaborators. As antibodies were identified from the blood of humans infected with SARS-CoV-2, we shifted to collaborate with academic partners to develop improved versions of their human-derived antibodies. This work reached its most important stage in rapid response to the emergence of the Omicron variant of concern (VOC) in late 2021. In a matter of weeks, enabled by on demand innovation to our screening pipeline, we computationally designed and experimentally characterized derivative antibodies of COV2-2130, one of two antibodies from Vanderbilt that form the basis of the AstraZeneca Evusheld prophylactic drug product. This drug product suffers a serious loss of efficacy against Omicron BA.1 and BA.1.1, the first Omicron strains. Due to tight integration of computational design and experimental evaluation, we were able to identify a pair of designs with potent neutralization of the main targets Omicron BA.1 and BA.1.1; but also the earlier Delta variant, and subsequent Omicron strains including BA.2, BA.4, BA.5, and BA.2.75, demonstrating that our multi-target design process can, by its nature, produce robust antibody designs that strictly improve over the parental antibody. These results, recognized by a 2022 Director’s Science and Technology award, have enabled the follow-on GUIDE program, to commence in FY23. While earlier design campaigns were substantially outsourced, we have engineered better and faster processes internally to better compliment, calibrate, and speed computational designs. As part of the follow-on GUIDE program, we will stand up a rapid and high-throughput antibody production and characterization facility staffed with the expertise and capabilities to foster our current collaboration across PLS and ENG as well as other partnerships toward computational design of biologics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A Framework for Non-Pipeline Alternatives Analysis and Review of Existing Approaches

States are increasingly interested in gas distribution system planning and the role that demand-side resources can play in maintaining an affordable, reliable, equitable, and low-carbon energy system. In particular, public utility commissions and utilities are exploring the role of non-pipeline alternatives (NPA) – an investment or activity that defers, reduces, or avoids the need to construct or replace a pipeline – in gas distribution system planning. NPAs are an emerging cost and risk mitigation tool that can provide gas utilities with an opportunity to reduce emissions, gas system costs, and customer risk by avoiding unnecessary infrastructure spending. Rather than address system issues with more costly and long-lived traditional capital projects, utilities can leverage demand-side NPA resources such as energy efficiency and electrification as well as supply-side NPA resources to meet system needs. However, there is limited guidance on performing an NPA and even fewer lessons learned. To fill this gap, the U.S. Department of Energy funded two reports on NPAs. The first report is a literature review that focuses on four states' NPA policies. The second report offers a framework for evaluating NPAs that states can adapt to meet their policy needs.

03 NATURAL GAS↗
DOE OSTI2024

CO 2 Pipeline Analysis for Existing Coal-Fired Power Plants

Previous studies have examined the needs for CO 2 transport and storage to achieve a net zero emissions economy by 2050 (Abramson and Christensen, 2021; Greig and Pascale, 2021; Chen and Pawar, 2023). This analysis examines the minimum CO 2 pipeline infrastructure necessary to transport and store CO 2 captured from coal-fired power plants. This analysis also examines an additional network configuration considering minimizing the number of state border crossings, reflecting non-economic considerations that may impact pipeline buildout.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2024

X-ray Spectral-Timing Pipeline to Investigate an Electron-Scattering Time Delay in Black Hole Accretion Disks [Slides]

The soft lag in black hole X-ray binaries (BHXRBs) refers to the time delay for the soft, thermal disk to respond to hard, variable coronal irradiation. This time lag increases from less than 1 ms to ~10 ms during the hard-to-soft state transition. Interpretations of soft lag trends appeal to changing the light-travel path via an evolving coronal height and/or inner accretion disk radius. Both interpretations neglect a time delay contribution from the reprocessing of irradiation inside the disk, where electron-scattering opacity dominates. A new theory considering a thermalization (electron scattering) time delay in the disk can plausibly produce ~10 ms time delays in the intermediate state. To further investigate the impacts of adding a time delay component from electron scattering in the disk, we are creating a spectral-timing pipeline that can analyze NICER (Neutron Star Interior Composition Explorer) X-ray observations of BHXRBs in outburst. In the near future, we will apply this spectral timing pipeline to develop a reverberation lag model from simulations that include the thermalization time delay from electron scattering in the disk.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

BatAnalysis - A Comprehensive Python Pipeline for Swift BAT Survey Analysis

The Swift Burst Alert Telescope (BAT) is a coded-aperture gamma-ray instrument with a large field of view that primarily operates in survey mode when it is not triggering on transient events. The survey data consist of 80- channel detector plane histograms that accumulate photon counts over periods of at least 5 minutes. These histograms are processed on the ground and are used to produce the survey data set between 14 and 195 keV. Survey data comprise >90% of all BAT data by volume and allow for the tracking of long-term light curves and spectral properties of cataloged and uncataloged hard X-ray sources. Until now, the survey data set has not been used to its full potential due to the complexity associated with its analysis and the lack of easily usable pipelines. Here, we introduce the BatAnalysis Python package, a wrapper for HEASoftpy, which provides a modern, opensource pipeline to process and analyze BAT survey data. BatAnalysis allows members of the community to use BAT survey data in more advanced analyses of astrophysical sources, including pulsars, pulsar wind nebula, active galactic nuclei, and other known/unknown transient events that may be detected in the hard X-ray band. We outline the steps taken by the Python code and exemplify its usefulness and accuracy by analyzing survey data of the Crab Nebula, NGC 2992, and a previously uncataloged MAXI transient. The BatAnalysis package allows for ~18 yr of BAT survey data to be used in a systematic way to study a large variety of astrophysical sources.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Red Hills Storage Hub Feasibility Study - Summary of Milestone 2 - Conceptual Basis of Plants and Pipeline Infrastructure

This document provides the first conceptual-level design of plants and pipeline infrastructure, including maps showing the project location, the potential footprint of the carbon capture and storage plant, the potential site of the capture and compression unit, and the conceptual pipeline layout for transporting the CO2 product from the capture facility to the future potential injection sites.

Carbon Capture and Storage,CarbonSAFE↗
DOE OSTI2022

Cold Spray Additive Manufacturing For New Pipeline Fabrication In Live, Natural Gas Distribution Mains

ULC Technologies (ULC) and the Penn State University Applied Research Lab (PSU ARL) developed a process and approach for rehabilitating live, natural gas distribution pipelines using Cold Spray Additive Manufacturing (CSAM). Robotic inline fabrication of a new pipe will be performed inside a natural gas main without disrupting customers. The material studied was blended with stainless steel which offers corrosion resistance, high strength and compatibility with hydrogen to enable a robust pipeline system that is future proofed. Stainless steel coatings sprayed and tested in the lab showed excellent strength, ductility, and porosity values. ULC initially sought to use methane as the process gas for compatibility with the host pipe, but after evaluating the field and lab challenges, nitrogen was deemed to be more suitable. Early robotic concepts have been developed that indicate technical feasibility. While the technology development originally targeted natural gas distribution mains, market research showed a broader need. Spot repairs performed in natural gas transmission mains may be a more economical approach for reinforcing distribution and transmission mains compared with full pipe-in-pipe rehabilitation. Additive manufacturing offers custom programming, as well as high accuracy and precision. This means custom contours can be manufactured in situ, offering flexibility in the repair solution, such as fabricating internal full circumferential sleeves. The value of the repair can be high at difficult-to-reach locations and in pipes that need reinforcement before the injection of methane-hydrogen blends.

03 NATURAL GAS↗
DOE OSTI2025

VA EDH Advanced Software Pipeline Framework Report: Enhancing Automation and Scalability

The VA Environmental Determinants of Health (EDH) Advanced Software Pipeline Framework is designed to enhance the efficiency, scalability, and security of geospatial data processing workflows. This framework integrates modern data orchestration and containerization technologies, including Prefect for workflow automation, Docker for containerization, and PostgreSQL/PostGIS for geospatial data storage and analysis. It ensures standardized, reproducible, and automated data processing, supporting VA objectives related to substance use risk assessment and recovery research. The pipeline addresses key scalability and performance challenges through horizontal and vertical scaling, high-performance computing (HPC) integration, parallel processing, task caching, and dynamic resource allocation. These optimizations improve throughput and reduce latency, allowing the system to efficiently manage large and complex datasets. Additionally, security and compliance measures—such as data encryption (SSL), Role-Based Access Control (RBAC), and adherence to GDPR and HIPAA standards—safeguard sensitive information throughout data transmission and storage. A key implementation of this framework includes the automation of shelter list geolocation workflows, ensuring that up-to-date data is readily available for VA decision-making. Lessons learned from this project include the transition from in-memory processing to incremental storage writes, improving resource management and reliability. Future enhancements aim to expand automation, integrate AI-driven anomaly detection, and incorporate high-performance computing resources. This framework provides a scalable, secure, and adaptable solution for managing geospatial datasets, reinforcing the VA’s ability to support clinical and strategic initiatives through data-driven decision-making.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Leveraging AI and Spatial Data to Unlock Pipeline Integrity Insights: NETL’s Advanced Infrastructure Integrity Model (AIIM)

Maintaining the integrity of natural gas infrastructure plays a critical role in ensuring energy security. Robust, data-driven foundational AI models for pipeline integrity can help address risk management and mitigation issues. Trusted foundational models can help with industry adoption and accelerate innovation by enhancing integrity predictions, reduce costs, and informing infrastructure build-out. The AIIM dashboard was released in 2022 and utilizes multi-ML models for ensemble-type insights. It was expanded to include analytics on reported incidents. It was developed as an ESRI Dashboard to support data visualization & interrogation and contains pipeline data and model results.

Advanced Infrastructure Integrity Model (AIIM)↗
DOE OSTI2026

SynthEsizing Novel H2 Sensors for Operational Resilience in Pipeline Infrastructure (SENSOR) (CRADA Final Report)

Hydrogen (H₂) is gaining attention as a versatile energy carrier with potential applications across industrial processes, power generation, and transportation. However, its practical deployment, particularly in large-scale distribution systems, faces significant infrastructure challenges. Transporting hydrogen through dedicated pipelines or blending it into existing natural gas networks can lead to serious issues, such as leakage due to the small size of hydrogen molecules and material degradation in pipelines through embrittlement. These technical risks raise safety concerns and could limit the integration of hydrogen into current energy infrastructures. Additionally, using hydrogen-enriched gas mixtures in combustion systems like turbines and engines introduces new performance and compatibility challenges that must be resolved before widespread use becomes feasible.

08 HYDROGEN↗
DOE OSTI2025

CO2 Streams Captured From Industrial Processes and Corrosion of Steel Pipelines

Based on the knowledge accumulated in the last decade related to Carbon Capture and Storage (CCS) processes and the identified gaps in knowledge, our study identifies and quantifies the key issues that impact the corrosion of materials, specifically relating to the integrity of structures for the CO2 transport pipelines and provides a better understanding of the materials’ compatibility with CO2 steams with impurities. This will help to understand the corrosion risks associated with CO2 transport and injection and leverage existing infrastructure or rebuild new pipeline networks to transport CO2 with impurities for EOR or geological storage.

CO2 corrosion↗
DOE OSTI2025

End-to-end deep learning pipeline for real-time Bragg peak segmentation: from training to large-scale deployment

X-ray crystallography reconstruction, which transforms discrete X-ray diffraction patterns into three-dimensional molecular structures, relies critically on accurate Bragg peak finding for structure determination. As X-ray free electron laser (XFEL) facilities advance toward MHz data rates (1 million images per second), traditional peak finding algorithms that require manual parameter tuning or exhaustive grid searches across multiple experiments become increasingly impractical. While deep learning approaches offer promising solutions, their deployment in high-throughput environments presents significant challenges in automated dataset labeling, model scalability, edge deployment efficiency, and distributed inference capabilities. We present an end-to-end deep learning pipeline with three key components: (1) a data engine that combines traditional algorithms with our peak matching algorithm to generate high-quality training data at scale, (2) a modular architecture that scales from a few million to hundreds of million parameters, enabling us to train large expert-level models offline while deploying smaller, distilled models at the edge, and (3) a decoupled producer-consumer architecture that separates specialized data source layer from model inference, enabling flexible deployment across diverse computing environments. Using this integrated approach, our pipeline achieves accuracy comparable to traditional methods tuned by human experts while eliminating the need for experiment-specific parameter tuning. Although current throughput requires optimization for MHz facilities, our system's scalable architecture and demonstrated model compression capabilities provide a foundation for future high-throughput XFEL deployments.

Wang, Cong↗
DOE OSTI2022

A computational pipeline to generate a synthetic dataset of metal ion sorption to oxides for AI/ML exploration

The charged mineral/electrolyte interfaces are ubiquitous in the surface and subsurface–including the surroundings of the geological disposal sites for radioactive waste. Therefore, understanding how ions interact with charged surfaces is critically important for predicting radionuclide mobility in the case of waste leakage. At present, the Surface Complexation Models (SCMs) are the most successful thermodynamic frameworks to describe ion retention by mineral surfaces. SCMs are interfacial speciation models that account for the effect of the electric field generated by charged surfaces on sorption equilibria. These models have been successfully used to analyze and interpret a broad range of experimental observations including potentiometric and electrokinetic titrations or spectroscopy. Unfortunately, many of the current procedures to solve and fit SCM to experimental data are not optimal, which leads to a non-transferable or non-unique description of interfacial electrostatics and consequently of the strength and extent of ion retention by mineral surfaces. Recent developments in Artificial Intelligence (AI) offer a new avenue to replace SCM solvers and fitting algorithms with trained AI surrogates. Unfortunately, there is a lack of a standardized dataset covering a wide range of SCM parameter values available for AI exploration and training–a gap filled by this study. Here, we described the computational pipeline to generate synthetic SCM data and discussed approaches to transform this dataset into AI-learnable input. First, we used this pipeline to generate a synthetic dataset of electrostatic properties for a broad range of the prototypical oxide/electrolyte interfaces. The next step is to extend this dataset to include complex radionuclide sorption and complexation, and finally, to provide trained AI architectures able to infer SCMs parameter values rapidly from experimental data. Here, we illustrated the AI-surrogate development using the ensemble learning algorithms, such as Random Forest and Gradient Boosting. These surrogate models allow a rapid prediction of the SCM model parameters, do not rely on an initial guess, and guarantee convergence in all cases.

Li, Chunhui↗