Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

Automated Monitoring of Pipeline Rights-of-Way

NASA Ames Research Center and the Pipeline Research Council International, Inc. have partnered in the formation of a research program to identify and develop the key technologies required to enable automated detection of threats to gas and oil transmission and distribution pipelines. This presentation describes the Right-of-way Automated Monitoring (RAM) program and highlights research successes to date, continuing challenges to implementing the RAM objectives, and the program's ongoing work and plans.

Frost, Chard Ritchie↗
NASA NTRS2010

A Framework for Propagation of Uncertainties in the Kepler Data Analysis Pipeline

The Kepler space telescope is designed to detect Earth-like planets around Sun-like stars using transit photometry by simultaneously observing 100,000 stellar targets nearly continuously over a three and a half year period. The 96-megapixel focal plane consists of 42 charge-coupled devices (CCD) each containing two 1024 x 1100 pixel arrays. Cross-correlations between calibrated pixels are introduced by common calibrations performed on each CCD requiring downstream data products access to the calibrated pixel covariance matrix in order to properly estimate uncertainties. The prohibitively large covariance matrices corresponding to the ~75,000 calibrated pixels per CCD preclude calculating and storing the covariance in standard lock-step fashion. We present a novel framework used to implement standard propagation of uncertainties (POU) in the Kepler Science Operations Center (SOC) data processing pipeline. The POU framework captures the variance of the raw pixel data and the kernel of each subsequent calibration transformation allowing the full covariance matrix of any subset of calibrated pixels to be recalled on-the-fly at any step in the calibration process. Singular value decomposition (SVD) is used to compress and low-pass filter the raw uncertainty data as well as any data dependent kernels. The combination of POU framework and SVD compression provide downstream consumers of the calibrated pixel data access to the full covariance matrix of any subset of the calibrated pixels traceable to pixel level measurement uncertainties without having to store, retrieve and operate on prohibitively large covariance matrices. We describe the POU Framework and SVD compression scheme and its implementation in the Kepler SOC pipeline.

Clarke, Bruce D.↗
NASA NTRS2010

Presearch Data Conditioning in the Kepler Science Operations Center Pipeline

We describe the Presearch Data Conditioning (PDC) software component and its context in the Kepler Science Operations Center (SOC) pipeline. The primary tasks of this component are to correct systematic and other errors, remove excess flux due to aperture crowding, and condition the raw flux light curves for over 160,000 long cadence (~thirty minute) and 512 short cadence (~one minute) targets across the focal plane array. Long cadence corrected flux light curves are subjected to a transiting planet search in a subsequent pipeline module. We discuss the science algorithms for long and short cadence PDC: identification and correction of unexplained (i.e., unrelated to known anomalies) discontinuities; systematic error correction; and excess flux removal. We discuss the propagation of uncertainties from raw to corrected flux. Finally, we present examples of raw and corrected flux time series for flight data to illustrate PDC performance. Corrected flux light curves produced by PDC are exported to the Multi-mission Archive at Space Telescope [Science Institute] (MAST) and will be made available to the general public in accordance with the NASA/Kepler data release policy.

Twicken, Joseph D.↗
NASA NTRS2010

Data Validation in the Kepler Science Operations Center Pipeline

We present an overview of the Data Validation (DV) software component and its context within the Kepler Science Operations Center (SOC) pipeline and overall Kepler Science mission. The SOC pipeline performs a transiting planet search on the corrected light curves for over 150,000 targets across the focal plane array. We discuss the DV strategy for automated validation of Threshold Crossing Events (TCEs) generated in the transiting planet search. For each TCE, a transiting planet model is fitted to the target light curve. A multiple planet search is conducted by repeating the transiting planet search on the residual light curve after the model flux has been removed; if an additional detection occurs, a planet model is fitted to the new TCE. A suite of automated tests are performed after all planet candidates have been identified. We describe a centroid motion test to determine the significance of the motion of the target photocenter during transit and to estimate the coordinates of the transit source within the photometric aperture; a series of eclipsing binary discrimination tests on the parameters of the planet model fits to all transits and the sequences of odd and even transits; and a statistical bootstrap to assess the likelihood that the TCE would have been generated purely by chance given the target light curve with all transits removed. Keywords: photometry, data validation, Kepler, Earth-size planets

Wu, Hayley↗
NASA NTRS2012

Detecting Methane From Leaking Pipelines and as Greenhouse Gas in the Atmosphere

Laser remote sensing measurements of trace gases from orbit can provide unprecedented information about important planetary science and answer critical questions about planetary atmospheres. Methane (CH4) is the second most important anthropogenically produced greenhouse gas. Though its atmospheric abundance is much less than that of CO2 (1.78 ppm vs. 380 ppm), it has much larger greenhouse heating potential. CH4 also contributes to pollution in the lower atmosphere through chemical reactions, leading to ozone production. Atmospheric CH4 concentrations have been increasing as a result of increased fossil fuel production, rice farming, livestock, and landfills. Natural sources of CH4 include wetlands, wild fires, and termites, and perhaps other unknown sources. Important sinks for CH4 include non-saturated soils and oxidation by hydroxyl radicals in the atmosphere. Remotely measuring CH4 and other biogenic molecules (such as ethane and formaldehyde) on Mars also has important implications on the existence of life on Mars. Measuring CH4 at very low (ppb) concentrations from orbit will dramatically improve the sensitivity and spatial resolution in the search for CH4 vents and sub-surface life on other planets. A capability has been developed using lasers and spectroscopic detection techniques for the remote measurements of trace gases in open paths. Detection of CH4, CO2, H2O, and CO in absorption cells and in open paths, both in the mid- IR and near-IR region, has been demonstrated using an Optical Parametric Amplifier laser transmitter developed at GSFC. With this transmitter, it would be possible to develop a remote sensing methane instrument. CH4 detection also has very important commercial applications. Pipeline leak detection from an aircraft or a helicopter can significantly reduce cost, response time, and pinpoint the location. The main advantage is the ability to rapidly detect CH4 leaks remotely. This is extremely important for the petrochemical industry. This capability can be used in manned or unmanned airborne platforms for the detection of leaks in pipelines and other areas of interest where a CH4 leak is suspected.

Riris, Haris↗
NASA NTRS2017

Kepler: A Search for Terrestrial Planets - SOC 9.3 DR25 Pipeline Parameter Configuration Reports

This document describes the manner in which the pipeline and algorithm parameters for the Kepler Science Operations Center (SOC) science data processing pipeline were managed. This document is intended for scientists and software developers who wish to better understand the software design for the final Kepler codebase (SOC 9.3) and the effect of the software parameters on the Data Release (DR) 25 archival products.

exoplanet↗
NASA NTRS2017

Data Reduction Pipeline for the CHARIS Integral-Field Spectrograph I: Detector Readout Calibration and Data Cube Extraction

We present the data reduction pipeline for CHARIS, a high-contrast integral-field spectrograph for the Subaru Telescope. The pipeline constructs a ramp from the raw reads using the measured nonlinear pixel response and reconstructs the data cube using one of three extraction algorithms: aperture photometry, optimal extraction, or chi-squared fitting. We measure and apply both a detector flatfield and a lenslet flatfield and reconstruct the wavelength- and position-dependent lenslet point-spread function (PSF) from images taken with a tunable laser. We use these measured PSFs to implement a chi-squared-based extraction of the data cube, with typical residuals of approximately 5 percent due to imperfect models of the under-sampled lenslet PSFs. The full two-dimensional residual of the chi-squared extraction allows us to model and remove correlated read noise, dramatically improving CHARIS's performance. The chi-squared extraction produces a data cube that has been deconvolved with the line-spread function and never performs any interpolations of either the data or the individual lenslet spectra. The extracted data cube also includes uncertainties for each spatial and spectral measurement. CHARIS's software is parallelized, written in Python and Cython, and freely available on github with a separate documentation page. Astrometric and spectrophotometric calibrations of the data cubes and PSF subtraction will be treated in a forthcoming paper.

Brandt, Timothy D.↗
NASA NTRS2010

Pixel-Level Calibration in the Kepler Science Operations Center Pipeline

We present an overview of the pixel-level calibration of flight data from the Kepler Mission performed within the Kepler Science Operations Center Science Processing Pipeline. This article describes the calibration (CAL) module, which operates on original spacecraft data to remove instrument effects and other artifacts that pollute the data. Traditional CCD data reduction is performed (removal of instrument/detector effects such as bias and dark current), in addition to pixel-level calibration (correcting for cosmic rays and variations in pixel sensitivity), Kepler-specific corrections (removing smear signals which result from the lack of a shutter on the photometer and correcting for distortions induced by the readout electronics), and additional operations that are needed due to the complexity and large volume of flight data. CAL operates on long (~30 min) and short (~1 min) sampled data, as well as full-frame images, and produces calibrated pixel flux time series, uncertainties, and other metrics that are used in subsequent Pipeline modules. The raw and calibrated data are also archived in the Multi-mission Archive at Space Telescope at the Space Telescope Science Institute for use by the astronomical community.

Calibration↗
NASA NTRS2010

Data Validation in the Kepler Science Operations Center Pipeline

We present an overview of the Data Validation (DV) software component and its context within the Kepler ScienceOperations Center (SOC) pipeline and overall Kepler Science mission. The SOC pipeline performs a transiting planetsearch on the corrected light curves for over 150,000 targets across the focal plane array. We discuss the DV strategy forautomated validation of Threshold Crossing Events (TCEs) generated in the transiting planet search. For each TCE, atransiting planet model is fitted to the target light curve. A multiple planet search is conducted by repeating the transitingplanet search on the residual light curve after the model flux has been removed; if an additional detection occurs, aplanet model is fitted to the new TCE. A suite of automated tests are performed after all planet candidates have beenidentified. We describe a centroid motion test to determine the significance of the motion of the target photocenterduring transit and to estimate the coordinates of the transit source within the photometric aperture; a series of eclipsingbinary discrimination tests on the parameters of the planet model fits to all transits and the sequences of odd and eventransits; and a statistical bootstrap to assess the likelihood that the TCE would have been generated purely by chancegiven the target light curve with all transits removed.

photometry↗
NASA NTRSDate not supplied

Pipeline for Applications-Based Data Discovery

From disaster response and mitigation to monitoring water quality or protecting wildlife habitat, satellite Earth observation data can be applied in countless ways to meet pressing needs and benefit society. The crucial first step toward successful data application is data discovery. Potential users often know exactly what data they need--what Earth feature or phenomenon they need to observe, how frequently, and at what resolution or level of accuracy--but may still struggle to discover the existing observations that meet their needs. We have developed a pipeline to connect applications-based users to specific satellites and data collections within NASA's Earth observation program of record that are highly relevant to their data needs. This pipeline combines available information on satellite and instrument measurement characteristics with an innovative machine learning-based approach that identifies instruments that are most relevant to the feature or phenomenon of interest.

Katrina S Virts↗
NASA NTRS2023

BatAnalysis - A Comprehensive Python Pipeline for Swift BAT Survey Analysis

The Swift Burst Alert Telescope (BAT) is a coded aperture gamma-ray instrument with a large field of view that primarily operates in survey mode when it is not triggering on transient events. The survey data consists of eighty-channel detector plane histograms that accumulate photon counts over time periods of at least 5 minutes. These histograms are processed on the ground and are used to produce the survey dataset between 14 and 195 keV. Survey data comprises >90% of all BAT data by volume and allows for the tracking of long term light curves and spectral properties of cataloged and uncataloged hard X-ray sources. Until now, the survey dataset has not been used to its full potential due to the complexity associated with its analysis and the lack of easily usable pipelines. Here, we introduce the BatAnalysis python package , a wrapper for HEASoftpy, which provides a modern, open-source pipeline to process and analyze BAT survey data. BatAnalysis allows members of the community to use BAT survey data in more advanced analyses of astrophysical sources including pulsars, pulsar wind nebula, active galactic nuclei, and other known/unknown transient events that may be detected in the hard X-ray band. We outline the steps taken by the python code and exemplify its usefulness and accuracy by analyzing survey data from the Crab Pulsar, NGC 2992, and a previously uncataloged MAXI Transient. The BatAnalysis package allows for ∼ 18 years of BAT survey to be used in a systematic way to study a large variety of astrophysical sources.

Tyler Parsotan↗
NASA NTRSDate not supplied

Combinatorial Reasoning: Selecting Reasons in Generative AI Pipelines via Combinatorial Optimization

Recent Large Language Models (LLMs) have demonstrated impressive capabilities at tasks that require human intelligence and are a significant step towards human-like artificial intelligence (AI). Yet the performance of LLMs at reasoning tasks have been subpar and the reasoning capability of LLMs is a matter of significant debate. While it has been shown that the choice of the prompting technique to the LLM can alter its performance on a multitude of tasks, including reasoning, the best performing techniques require human-made prompts with the knowledge of the tasks at hand. We introduce a framework for what we call Combinatorial Reasoning (CR), a fully-automated prompting method, where reasons are sampled from an LLM pipeline and mapped into a Quadratic Unconstrained Binary Optimization (QUBO) problem. The framework investigates whether QUBO solutions can be profitably used to select a useful subset of the reasons to construct a Chain-of-Thought style prompt. We explore the acceleration of CR with specialized solvers. We also investigate the performance of simpler zero-shot strategies such as linear majority rule or random selection of reasons. Our preliminary study indicates that coupling a combinatorial solver to generative AI pipelines is an interesting avenue for AI reasoning and elucidates design principles for future CR methods.

combinatorial reasoning↗
NASA NTRSDate not supplied

Comparison of Ring Diagrams Based on the Doppler Shifts of Synthetic Data Obtained with Bisector Method and SDO/HMI Pipeline

Ring diagrams are cross-sections of three-dimensional spatiotemporal power spectra of solar oscillations. The rings reveal information about sub-surface flows and represent an important tool for helioseismology. Ring diagrams can be constructed using Doppler velocity or intensity maps of spectral lines. How the velocities are computed is an important factor for accuracy of information we can retrieve on subsurface flows. In our work, the ring diagrams are generated from Doppler shift data of synthesized Fe I 6173 Å line. We compare ring diagrams computed by two methods–HMI line-of-sight pipeline and the bisector of Fe line. Fe I line is synthesized for StellarBox 3D Radiative hydrodynamic simulations under LTE assumption. We aim to answer the following questions: 1.How do power spectra obtained from velocities computed with the HMI pipeline and bisector of the Fe I 6173Å compare? 2.How do the power spectral density retrieved with each method vary with heliocentric angle? 3.What is the effect of changing resolution on the power spectral density in ring diagrams obtained with the two methods?

SMD↗
NASA NTRS2025

An Efficient GPU-Accelerated Multi-Source Global Fit Pipeline for LISA Data Analysis

The large-scale analysis task of deciphering gravitational wave signals in the LISA data stream will be difficult, requiring a large amount of computational resources and extensive development of computational methods. Its high dimensionality, multiple model types, and complicated noise profile require a global fit to all parameters and input models simultaneously. In this work, we detail our global fit algorithm, called “Erebor,” designed to accomplish this challenging task. It is capable of analysing current state-of-the-art datasets and then growing into the future as more pieces of the pipeline are completed and added. We describe our pipeline strategy, the algorithmic setup, and the results from our analysis of the LDC2A Sangria dataset, which contains Massive Black Hole Binaries, compact Galactic Binaries, and a parameterized noise spectrum whose parameters are unknown to the user. The Erebor algorithm includes three unique and very useful contributions: GPU acceleration for enhanced computational efficiency; ensemble MCMC sampling with multiple MCMC walkers per temperature for better mixing and parallelized sample creation; and special online updates to reversible-jump (or trans-dimensional) sampling distributions to ensure sampler mixing and accurate initial estimates for detectable sources in the data. We recover posterior distributions for all 15 (6) of the injected MBHBs in the LDC2A training (hidden) dataset. We catalog ∼12000 Galactic Binaries (∼8000 as high confidence detections) for both the training and hidden datasets. All of the sources and their posterior distributions are provided in publicly available catalogs.

LISA global fit↗
NASA NTRS2025

Efficient GPU-Accelerated MultiSource Global Fit Pipeline for LISA Data Analysis

The large-scale analysis task of deciphering gravitational-wave signals in the LISA data stream will be difficult, requiring a large amount of computational resources and extensive development of computational methods. Its high dimensionality, multiple model types, and complicated noise profile require a global fit to all parameters and input models simultaneously. In this work, we detail our global fit algorithm, called “Erebor,” designed to accomplish this challenging task. It is capable of analyzing current state-of-the-art datasets and then growing into the future as more pieces of the pipeline are completed and added. We describe our pipeline strategy, the algorithmic setup, and the results from our analysis of the LDC2A Sangria dataset, which contains massive black hole binaries, compact galactic binaries, and a parametrized noise spectrum whose parameters are unknown to the user. The Erebor algorithm includes three unique and very useful contributions: GPU acceleration for enhanced computational efficiency; ensemble Markov Chain Monte Carlo (MCMC) sampling with multiple MCMC walkers per temperature for better mixing and parallelized sample creation; and special online updates to reversible-jump (or transdimensional) sampling distributions to ensure sampler mixing and accurate initial estimates for detectable sources in the data.We recover posterior distributions for all 15 (6) of the injected massive black hole binaries (MBHB) in the LDC2A training (hidden) dataset. We catalog ∼12000 galactic binaries (∼8000 as high confidence detections) for both the training and hidden datasets. All of the sources and their posterior distributions are provided in publicly available catalogs.

LISA↗
DOE OSTI2020

The updated DESGW processing pipeline for the third LIGO/VIRGO observing run

The DESGW group seeks to identify electromagnetic counterpartsof gravitational wave events seen by the LIGO-VIRGO network, such as thoseexpected from binary neutron star mergers or neutron star- black hole mergers.DESGW was active throughout the first two LIGO observing seasons, followingup several binary black hole mergers and the first binary neutron star merger,GW170817. We describe the modifications to the observing strategy generationand image processing pipeline between the second (ending in August 2017)and third (beginning in April 2019) LIGO observing seasons. The modifica-tions include a more robust observing strategy generator, further parallelizationof the image reduction software and dierence imaging processing pipeline,data transfer streamlining, and a web page listing identified counterpart candi-dates that updates in real time. Taken together, the additional parallelizationsteps enable us to identify potential electromagnetic counterparts within fullycalibrated search images in less than one hour, compared to the 3-5 hours itwould typically take during the first two seasons. These performance improve-ments are critical to the entire EM followup community, as rapid identification(or rejection) of candidates enables detailed spectroscopic followup by multipleinstruments as soon as possible, leading to more information about the environ-ment immediately following such gravitational wave events.

Herner, Kenneth R.↗
DOE OSTI2021

BETO 2021 Peer Review - Biomethanation to Upgrade Biogas to Pipeline Grade Methane WBS 5.1.3.102

We are developing, innovating and de-risking a biomethanation process capable of megawatt-scale deployment that upgrades biogas waste streams to produce pipeline quality renewable natural gas (RNG). Biomethanation is a two-step process using a methanogenic microorganism to convert renewable hydrogen (H2) and waste carbon dioxide (CO2) to renewable methane (CH4) - the primary component in natural gas. Using biogenic CO2 from biogas sources like dairies, wastewater treatment plants, and landfills allows production of this drop-in direct replacement fuel to participate in the growing number of carbon markets; like California's Low Carbon Fuel Standard and the Federal Renewable Fuel Standard. The end-of-project goal is to demonstrate pipeline quality RNG production (> 95% CH4, < 4% H2, <1% CO2, < 0.2% O2 and < 4 parts per million hydrogen sulfide) using real biogas feedstocks. We will accomplish this goal by designing and building a pressurized (18 bar) mobile lab-scale (20L) bioreactor research platform, including integrated electrolyzer system, based on lessons learned from operating the 700L pilot system from Southern California Gas Company. In collaboration with Electrochaea, natural gas utilities and Argonne National lab, will provide the data to establish a preliminary range of carbon intensity to help accelerate the deployment of utility-scale H2 production and qualify the biomethanation pathway process for RNG production.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Novel Membrane-Based Corrosion Sensors For Operating In Natural Gas Pipelines

Membrane-based electrochemical sensors (MBES) provide an attractive option for real-time continuous humidity and corrosion rate measurements inside of natural gas pipelines. The MBES, can use electrochemical techniques such as linear polarization resistance and electrochemical impedance spectroscopy even when a conductive aqueous phase is intermittent or unavailable. We now present field-test performance data of our MBES operated inside a multiphase water and natural gas flow facility. The MBES measured internal corrosion and conductivity at the top, bottom, and side of the pipe spool. Tests were carried out with flow rates of 4.6, 9.1, and 18.3 m/s, temperatures of 21, 29, and 35 °C, and a pressure of 1000 psia. All sensors followed similar trends in open circuit potential, corrosion rate, and conductivity over time. Neither flow rate, temperature, nor build-up of metal-based solids on the sensor surfaces impacted the quality of the electrochemical measurements. The field-test results confirm that the membrane-based electrochemical sensors are suitable for monitoring internal corrosion of operational natural-gas pipelines.

Duffy, Timothy↗