SEARCH · Engineering Papers

Results for “pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2025

RectifHydPlus Data Pipeline v1.1.0

The RectifHydPlus Data Pipeline is an open source and fully reproducible data processing pipeline for creating RectifHydPlus—a dataset of historical monthly net electricity generation for all US hydropower plants (>10MW). The pipeline is coded in R, applying tidyverse libraries and code principles, and using the targets data pipeline framework. All data inputs to the RectifHydPlus Data Pipeline are available from public sources. References to all data inputs, as well as instructions for running the RectifHydPlus Data Pipeline, are available on the GitLab code repository: https://code.ornl.gov/turnersw/rectifhydplus

Turner, SeanWilliam Donald [Oak Ridge National Lab↗

DOE OSTI2025

CMB-S4: Foreground-cleaning Pipeline Comparison for Measuring Primordial Gravitational Waves

We compare multiple foreground-cleaning pipelines for estimating the tensor-to-scalar ratio, r, using simulated maps of the planned CMB-S4 experiment within the context of the South Pole Deep Patch. To evaluate robustness, we analyze bias and uncertainty on r across various foreground suites using map-based simulations. The foreground-cleaning methods include: a parametric maximum likelihood approach applied to auto- and cross-power spectra between frequency maps; a map-based parametric maximum-likelihood method; and a harmonic-space internal linear combination using frequency maps. We summarize the conceptual basis of each method to highlight their similarities and differences. To better probe the impact of foreground residuals, we implement an iterative internal delensing step, leveraging a map-based pipeline to generate a lensing B-mode template from the large aperture telescope frequency maps. Our results show that the performance of the three approaches is comparable for simple and intermediate-complexity foregrounds, with σ(r) ranging from 3–5 ×10 −4 . However, biases at the 1σ–2σ level appear when analyzing more complex forms of foreground emission. By extending the baseline pipelines to marginalize over foreground residuals, we demonstrate that contamination can be reduced to within statistical uncertainties, albeit with a pipeline-dependent impact on σ(r), which translates to a detection significance between 2σ and 4σ for an input value of r = 0.003. These findings suggest varying levels of maturity among the tested pipelines, with the auto- and cross-spectra-based approach demonstrating the best stability and overall performance. Moreover, given the extremely low noise levels, mutual validation of independent foreground-cleaning pipelines is essential to ensure the robustness of any potential detection.

astronomy data analysis↗

NASA NTRS1956

Friction factors for flow of non-newtonian materials in pipelines

In many industries, non-Newtonian material, such as oils, lubricants, foods, cosmetics, and solid-liquid suspensions, are passed through pipelines during manufacture, application, or transportation. For the proper design of the pipelines, and evaluation of the flow resistance of the materials when flowing in these pipelines is essential. The pressure loss Δ p , due to a material flowing through a straight pipeline can be expressed as a function of a friction factor, φ, so that Δ p = ( pv 2 )/2 L/D φ where φ is a function of the flow properties of the material v is the mean velocity of the material and can be determined from the mas-flow rate, L is the length and D is the diameters of the pipeline, and p is the density of the material. The purpose of this paper is to present a generalized friction diagram, which can be used to determine φ for any material and flow condition, provided the flow properties of the material and the mean flow velocity in the pipeline are known. In composing this diagram generous use is made of the literature and only parts of the diagram and the presentation can claim originality.

Ruth N Weltmann↗

NASA NTRS2003

Creation and Implementation of a Workforce Development Pipeline Program at MSFC

Within the context of NASA's Education Programs, this Workforce Development Pipeline guide describes the goals and objectives of MSFC's Workforce Development Pipeline Program as well as the principles and strategies for guiding implementation. It is designed to support the initiatives described in the NASA Implementation Plan for Education, 1999-2003 (EP-1998-12-383-HQ) and represents the vision of the members of the Education Programs office at MSFC. This document: 1) Outlines NASA s Contribution to National Priorities; 2) Sets the context for the Workforce Development Pipeline Program; 3) Describes Workforce Development Pipeline Program Strategies; 4) Articulates the Workforce Development Pipeline Program Goals and Aims; 5) List the actions to build a unified approach; 6) Outlines the Workforce Development Pipeline Programs guiding Principles; and 7) The results of implementation.

Hix, Billy↗

NASA NTRS1989

Multinode reconfigurable pipeline computer

A multinode parallel-processing computer is made up of a plurality of innerconnected, large capacity nodes each including a reconfigurable pipeline of functional units such as Integer Arithmetic Logic Processors, Floating Point Arithmetic Processors, Special Purpose Processors, etc. The reconfigurable pipeline of each node is connected to a multiplane memory by a Memory-ALU switch NETwork (MASNET). The reconfigurable pipeline includes three (3) basic substructures formed from functional units which have been found to be sufficient to perform the bulk of all calculations. The MASNET controls the flow of signals from the memory planes to the reconfigurable pipeline and vice versa. the nodes are connectable together by an internode data router (hyperspace router) so as to form a hypercube configuration. The capability of the nodes to conditionally configure the pipeline at each tick of the clock, without requiring a pipeline flush, permits many powerful algorithms to be implemented directly.

Nosenchuck, Daniel M.↗

NASA NTRS2004

Caltrans Keeps the Spitzer Pipelines Moving

The computer pipelines used to process digital infrared astronomical images from NASA's Spitzer Space Telescope require various input calibration-data files for characterizing the attributes and behaviors of the onboard focal-plane-arrays and their detector pixels, such as operability, dark-current offset, linearity, non- uniformity, muxbleed, droop, and point-response functions. The telescope has three very different science instruments, each with three or four spectral-band-pass channels, depending on the instrument. Moreover, each instrument has various operating modes (e-g., full array or sub-array in one case) and parameters (e.g., integration time). Calibration data that depend on these considerations are needed by pipelines for generating both science products (production pipelines) and higher-level calibration products (calibration pipelines). The calibration files are created in various formats either 'off-line' or by the aforementioned calibration pipelines, depending on the above configuration details. Also, the calibration files are generally applicable to a certain time period and therefore must be selected accordingly for a given raw input image to be correctly processed. All of this complexity in selecting and retrieving calibration files for pipeline processing is handled by a procedural software-program called 'caltrans' . This software, which is implemented in C and interacts with an Informix database, was developed at the Spitzer Science Center (SSC) and is now deployed in SSC daily operations. The software is rule-based, very flexible, and, for efficiency, capable of retrieving multiple calibration files with a single software-execution command.

Spitzer↗

NASA NTRS2011

Improved, Low-Stress Economical Submerged Pipeline

A preliminary study has shown that the use of a high-strength composite fiber cloth material may greatly reduce fabrication and deployment costs of a subsea offshore pipeline. The problem is to develop an inexpensive submerged pipeline that can safely and economically transport large quantities of fresh water, oil, and natural gas underwater for long distances. Above-water pipelines are often not feasible due to safety, cost, and environmental problems, and present, fixed-wall, submerged pipelines are often very expensive. The solution is to have a submerged, compliant-walled tube that when filled, is lighter than the surrounding medium. Some examples include compliant tubes for transporting fresh water under the ocean, for transporting crude oil underneath salt or fresh water, and for transporting high-pressure natural gas from offshore to onshore. In each case, the fluid transported is lighter than its surrounding fluid, and thus the flexible tube will tend to float. The tube should be ballasted to the ocean floor so as to limit the motion of the tube in the horizontal and vertical directions. The tube should be placed below 100-m depth to minimize biofouling and turbulence from surface storms. The tube may also have periodic pumps to maintain flow without over-pressurizing, or it can have a single pump at the beginning. The tube may have periodic valves that allow sections of the tube to be repaired or maintained. Some examples of tube materials that may be particularly suited for these applications are non-porous composite tubes made of high-performance fibers such as Kevlar, Spectra, PBO, Aramid, carbon fibers, or high-strength glass. Above-ground pipes for transporting water, oil, and natural gas have typically been fabricated from fiber-reinforced plastic or from more costly high-strength steel. Also, previous suggested subsea pipeline designs have only included heavy fixed-wall pipes that can be very expensive initially, and can be difficult and expensive to deploy for long distances. A much less expensive Kevlar pipeline can be coiled up on a ship s deck and deployed in the water as the ship moves. Support ships can be used to drop sand into conduits below the uninflated tube, so that the tube remains in place when more buoyant fresh water later fills the tubes.

Jones, Jack A.↗

DOE OSTI2024

Distributed Optical Fiber Sensor Systems: Application to Natural Gas Pipeline Monitoring

Distributed fiber optic sensing is a cutting-edge technology that has found extensive applications in the monitoring of pipelines. Considering the low backscattering level of standard single-mode fiber as fiber under test, a Rayleigh enhanced optical fiber embedded within the tight-buffered cable is demonstrated in field testing. We analyzed the increased backscattering fiber cable's vibration performance to the conventional single-mode telecom fiber using a custom-built Φ-OTDR interrogator system. Thereafter, using a 4-inch steel pipeline with a flow rate of 5, 10, 15, and 20 ft/s and a fixed pressure level of 1000 psi, we field-tested the sensor system for monitoring natural gas pipeline acoustic vibrations. We also field tested Brillouin optical time domain analysis (BOTDA) system for pipeline hoop strain monitoring under various pressure conditions. The pilot-scale testing results presented in this study suggested that pipeline operators can accurately perform flow monitoring, leak detection, and pressure monitoring for pipeline integrity monitoring.

Lalam, Nageswara↗

DOE OSTI2025

Pilot-Scale Validation of Distributed Optical Fiber Sensors for Underground Pipeline Monitoring

Monitoring parameters such as hoop strain, pressure, and acoustic vibrations is key to detecting potential leaks, intrusions, or structural issues. Distributed optical fiber sensor (DOFS) systems provide a compelling solution for continuous, real-time monitoring over long distances. This paper details the development and pilot-scale implementation of DOFS systems for underground pipeline monitoring, evolving from a proof-of-concept stage. Multiple custom-designed DOFS interrogator units—such as optical frequency-domain reflectometry (OFDR), Brillouin optical time-domain analysis (BOTDA), and multimodal interferometer-based fiber acoustic sensor systems were tested to measure the key parameters, such as hoop strain, pipe pressure, surrounding soil temperature, and acoustic vibrations. The underground product pipeline’s outer diameter is 30 inches, the wall thickness is 1.28 inches, and 3 feet deep from the surface. The fiber deployment strategies and sensing data acquisition methods for these systems are discussed. The results demonstrate the effectiveness of DOFS in detecting hoop strain, temperature changes, and acoustic vibrations, showcasing their potential for real-time monitoring and enhancing pipeline safety. These findings from pilot-scale testing offer valuable insights into advancing pipeline monitoring technologies and improving the reliability of underground pipeline systems.

fiber optic sensors↗

DOE OSTI2025

Monitoring pipeline integrity of underground gas storage facilities using membrane-based electrochemical sensors

Effective monitoring of internal corrosion risk is crucial to ensuring the safety and longevity of natural gas pipeline infrastructure. While electrochemical sensors are commonly used to assess corrosion rates and corrosion indicators in aqueous fluids, they are rarely used in gas pipelines as these fluids lack the ionic conductivity needed for electrochemical measurements. The inclusion of ion-conductive membranes into electrochemical sensors can extend their functionality into humidified gas streams, providing critical information about emerging corrosion events that are common during withdrawal season in pipeline systems downstream from underground storage facilities. In parallel, new protective films, like those obtained through cold spray coating, are being developed to protect oil and gas pipelines and recover losses in structural integrity due to corrosion damage. Herein, we demonstrate how membrane-based electrochemical sensors (MBES) can be used to monitor fluid corrosivity by examining their response to changes in water content for a wide range of fluid compositions. It was found that MBES readings were highly sensitive to water content changes with membrane conductivity measurements varying from 10 –6 to 10 –1 S cm -1 , and corrosion rate measurements which varied from 10 –7 to 1 mm y -1 . Electron microscopy confirmed that the self-healing characteristics of metal coating films were still active despite their inclusion into an MBES probe. In conclusion, these findings indicate that membrane-based corrosion monitoring can be expanded to monitor coated-pipeline materials and provide early detection of emerging corrosion upsets relevant to underground gas storage facilities.

Electrochemical sensor↗

DOE OSTI2024

MVP: a modular viromics pipeline to identify, filter, cluster, annotate, and bin viruses from metagenomes

While numerous computational frameworks and workflows are available for recovering prokaryote and eukaryote genomes from metagenome data, only a limited number of pipelines are designed specifically for viromics analysis. With many viromics tools developed in the last few years alone, it can be challenging for scientists with limited bioinformatics experience to easily recover, evaluate quality, annotate genes, dereplicate, assign taxonomy, and calculate relative abundance and coverage of viral genomes using state-of-the-art methods and standards. Here, we describe Modular Viromics Pipeline (MVP) v.1.0, a user-friendly pipeline written in Python and providing a simple framework to perform standard viromics analyses. MVP combines multiple tools to enable viral genome identification, characterization of genome quality, filtering, clustering, taxonomic and functional annotation, genome binning, and comprehensive summaries of results that can be used for downstream ecological analyses. Overall, MVP provides a standardized and reproducible pipeline for both extensive and robust characterization of viruses from large-scale sequencing data including metagenomes, metatranscriptomes, viromes, and isolate genomes. As a typical use case, we show how the entire MVP pipeline can be applied to a set of 20 metagenomes from wetland sediments using only 10 modules executed via command lines, leading to the identification of 11,656 viral contigs and 8,145 viral operational taxonomic units (vOTUs) displaying a clear beta-diversity pattern. Further, acting as a dynamic wrapper, MVP is designed to continuously incorporate updates and integrate new tools, ensuring its ongoing relevance in the rapidly evolving field of viromics. MVP is available at https://gitlab.com/ccoclet/mvp and as versioned packages in PyPi and Conda.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Improving the Quality of Geothermal Data Through Data Standards and Pipelines Within the Geothermal Data Repository: Preprint

For machine learning outputs to be applicable to real world problems, high quality data are needed to ensure high quality results. With the more recent emphasis on machine learning in geothermal, there is an increasing need for greater focus on the quality of the data available for use in these projects. For example, Geothermal Operational Optimization Using Machine Learning (GOOML) utilized large quantities of geothermal power plant operational data to inform power plant operational configurations to maximize power generation. High quality datasets result from dependable sensors or devices collecting data, high frequency of measurements, sufficient data points, adequate metadata, reliable storage of data, and sufficient data curation. Another component that contributes to high quality data is reusability, which can be enhanced through data standardization. Data Standardization creates consistency in formatting and contents of like datasets, lessening preprocessing requirements and ensuring adequate information provided by a given dataset. The Geothermal Data Repository (GDR) aims to help improve data quality through automated data standardization for high-value datasets through the implementation of data pipelines alongside reliable and accessible long-term storage for datasets. As such, the GDR has decided to shift away from recommending the use of Excel-based content models and towards the implementation of automated data pipelines. This takes the burden of data standardization off the user and project team and will increase the availability of standardized geothermal data available through the GDR. A set of recommendations, or a data standard for each data type will exist with each data pipeline in order to advise data collection for maximum usability for future research. This paper serves to describe the GDR's proposed transition towards data standardization through automated data pipelines, to discuss the need for and value of such a shift, and to call for suggestions from the community regarding the most useful data standards and pipelines.

data↗

DOE OSTI2025

A machine-learning-driven data labeling pipeline for scientific analysis in MLExchange

This study introduces a novel labeling pipeline to accelerate the labeling process of scientific data sets by using artificial intelligence (AI)-guided tagging techniques. This pipeline includes a set of interconnected web-based graphical user interfaces (GUIs), where Data Clinic and MLCoach enable the preparation of machine learning (ML) models for data reduction and classification, respectively, while Label Maker is used for label assignment. Throughout this pipeline, data can be accessed through a direct connection to a file system or through Tiled for access through Hypertext Transfer Protocol (HTTP). Our experimental results present three use cases where this labeling pipeline has been instrumental for the study of large X-ray scattering data sets in the area of pattern recognition, the remote analysis of resonant soft X-ray scattering data and the fine-tuning process of foundation models. These use cases highlight the labeling capabilities of this pipeline, including the ability to label large data sets in a short period of time, to perform remote data analysis while minimizing data movement and to enhance the fine-tuning process of complex ML models with human involvement.

Chavez, Tanny (ORCID:0000000193172896)↗

DOE OSTI2024

Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs)

The Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs) are a set of prototype computational tools for simulating and optimizing midstream natural gas pipeline system operations subject to location and time-dependent hydrogen blending. The models can accurately resolve dynamic gas flows through large-scale pipeline networks using non-ideal gas equations of state. The codes can be used as decision support for planning and design decisions involving intra-day energy flow schedules as well as spatiotemporal economic values of natural gas, hydrogen, and net energy delivered to consumers while ensuring that pipeline hydraulic limitations, gas compressor station constraints, operational factors, and pre-existing shipping contracts are satisfied. The inputs to the codes are a model of the pipeline system as well as time-series data that specify boundary conditions on the network. For optimization, the code module requires price and quantity offers for natural gas and hydrogen and price and quantity bids for energy, which are used as time-dependent constraints in an optimal control problem. The outputs are time-series data that provide a predictive simulation of gas flows, mass fractions, and pressures, or with additional degrees of freedom give an approximately optimal solution for gas injections/withdrawals, compressor settings, and sensitivities to the objective function that provide locational values of energy.

Zlotnik, Anatoly↗

DOE OSTI2024

A Data Processing Pipeline To Extract A Knowledge Graph From Sec Documents For Socio-technical Analysis Of Critical Infrastructure Influence

Weaver, GabrielA.↗

DOE OSTI2025

A Data Processing Pipeline To Extract A Knowledge Graph From Heterogeneous Data For Socio-technical Analysis Of Critical Infrastructure Influence

The code is written in Python and consists of the following pipeline that is implemented in Apache Airflow. This pipeline intends to understand the companies that are directly or indirectly involved with a type of critical infrastructure system at some point in that system's lifecycle. The pipeline takes a configuration file that specifies a list of initial companies to consider, a geographic region of interest, and a set of SEC form types as well as other data sources (e.g. CrunchBase) from which to extract entities and relations. There are four main components to this pipeline as currently implemented: Entity Extraction, Network Construction, Analysis, and Visualization. First, Entity Extraction, is implemented as the `topear-extract_organizations` Apache Airflow workflow. Given an initial query that specifies a geographic region of interest and a time interval, the software will extract CI facilities of interest and organizations that have a direct influence relationship to those facilities (e.g. ownership). During the course of the LDRD, we focused on Electric Vehicle charging stations and this information is available via the Department of Energy (DOE) database on fueling stations maintained by NREL. Within the context of the DOE CESER project, we have focused on Battery Energy Storage Systems (BESS). Second, the Network Extraction component will iteratively construct a social network graph given the set of organizations and people extracted in the previous step. Organizations (and eventually People if desired) are then fed as a query to the `topgear-construct_social_network` Apache Airflow workflow which given a set of initial companies and data sets (e.g. SEC EDGAR form types, OpenCorporates, Crunchbase). This Airflow workflow will iteratively query such data sources to discover relationships with new organizations and people. For example, this module can iteratively query SEC EDGAR for metadata that documents the number of each type of form for the given set of companies and their location. This forms metadata represents a catalog of data sources from SEC EDGAR for the extracted social network knowledge graph. The pipeline then downloads these forms from the website and saves them in a build directory for further processing. These documents are then parsed for entities and relations. Again, we note that in additional to SEC data sources, this step can also pull in information on organizations via API services such as CrunchBase and OpenCorporates or bulk data sources. At the end of this step, the resultant social network, the Critical Infrastructure network, and the edges that encode relationships between organizations and CI facilities, form the Adversarial Socio-Technical Network (ASTN) that informs the analysis. Third, the Analysis component processes these generated ASTN. Previously, that has included the ability to compare prevalence of different vendors for a given infrastructure component type across different regions as well as identify common public and private investors across those vendors. This was demonstrated for EV Charging Stations across several different metropolitan areas within an IEEE PES GridEdge publication. More recently, we have looked at ways to identify infrastructure owners and operators of BESS with the most nameplate capacity across different states as well as other indictors of risk resulting from changes in ownership over time. Finally, the Visualization component consists of an HTML/CSS/JS framework by which users can interact geospatial, operational, and organizational relationships across a given portfolio of Critical Infrastructure facilities. The objective is to provide a library of UI/UX modules that can be repurposed for stakeholder-specific dashboards. All of the modules are related via a common event model that enables UI actions in one view to percolate across the other views.

Weaver, Gabriel [Idaho National Laboratory (INL), ↗

DOE OSTI2023

Abstract for CRADA between National Energy Technology Laboratory and Colonial Pipeline Company

The National Energy Technology Laboratory (NETL) and Colonial Pipeline Company (Participant) will collaborate in the field demonstration of optical fiber sensor systems developed at NETL on Participant’s fuel pipeline. The optical fiber sensor technologies are capable of distributed temperature and strain sensing, distributed acoustic sensing, and ultrasensitive acoustic sensing. Real-time monitoring of these parameters enables pipeline integrity monitoring, security monitoring, flow rate monitoring, etc. Successful demonstration on a real fuel pipeline at Participant’s facilities will validate the sensor technologies and installation methods at a real scale. This effort aligns with NETL’s mission in reliable and sustainable energy and reducing environmental effects due to pipeline failures.

42 ENGINEERING↗

DOE OSTI2023

Onshore U.S. Carbon Pipeline Deployment: Siting, Safety, and Regulation

Carbon capture, utilization, and storage technology has significant potential to reduce greenhouse gas emissions and mitigate the impact of climate change, particularly in hard-to-decarbonize industrial and commercial sectors. Unless utilization or storage occurs at the same location as carbon capture, which is rare, carbon must be transported from a point source to a utilization or storage site. Pipelines offer significant advantages for large-scale transportation of carbon dioxide over other methods. While studies show that reaching net-zero carbon emissions in the United States by 2050 will require between 29,000 and 66,000 miles of carbon pipelines, the U.S. had deployed fewer than 6,000 miles of carbon pipelines by 2022. Although closing this gap is important to achieving low-carbon goals, carbon pipelines operate in a complex and uncertain local, state and federal regulatory landscape and face public concerns about safety and siting. This report covers numerous regulatory issues surrounding carbon pipeline development, including the current narrow federal definition of carbon dioxide and the considerable variation in state and local governments’ laws and regulations. The report serves as a primer for regulators and stakeholders who seek to better understand the regulatory challenges and opportunities facing this critical infrastructure

01 COAL, LIGNITE, AND PEAT↗