Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Analysis and Optimization of Parallel Software Pipeline Performance

Pipelining is a common strategy for extracting parallelism from a collection of independent computational tasks, each of which is spread among a number of processors and has an implied data dependence. When implemented on MIMD parallel computers with finite process interrupt times, pipeline algorithms suffer from slowdown--in addition to the expected pipeline fill time--due to a wave-like propagation of delays. This phenomenon, which has been observed experimentally using the performance monitoring system AIMS, is investigated analytically, and an optimal correction is derived to eliminate the wave. Efficiency increase through the correction is verified experimentally.

VanderWijngaart, Rob F.↗
NASA NTRS1995

The Effect of Interrupts on Software Pipeline Execution on Message-Passing Architectures

Pipelining is a common strategy for extracting parallelism from a collection of independent computational tasks, each of which is spread among a number of processors and has an implied data dependence. When implemented on MIMD parallel computers with finite process interrupt times, pipeline algorithms suffer from slowdown--in addition to the expected pipeline fill time--due to a wave-like propagation of delays. This phenomenon, which has been observed experimentally using the performance monitoring system AIMS, is investigated analytically, and an optimal correction is derived to eliminate the wave. Efficiency increase through the correction is verified experimentally.

VanderWijngaart, Rob F.↗
NASA NTRS2010

The Kepler Science Operations Center Pipeline Framework Extensions

The Kepler Science Operations Center (SOC) is responsible for several aspects of the Kepler Mission, including managing targets, generating on-board data compression tables, monitoring photometer health and status, processing the science data, and exporting the pipeline products to the mission archive. We describe how the generic pipeline framework software developed for Kepler is extended to achieve these goals, including pipeline configurations for processing science data and other support roles, and custom unit of work generators that control how the Kepler data are partitioned and distributed across the computing cluster. We describe the interface between the Java software that manages the retrieval and storage of the data for a given unit of work and the MATLAB algorithms that process these data. The data for each unit of work are packaged into a single file that contains everything needed by the science algorithms, allowing these files to be used to debug and evolve the algorithms offline.

Klaus, Todd C.↗
NASA NTRS2013

Dynamic Black-Level Correction and Artifact Flagging in the Kepler Data Pipeline

Instrument-induced artifacts in the raw Kepler pixel data include time-varying crosstalk from the fine guidance sensor (FGS) clock signals, manifestations of drifting moiré pattern as locally correlated nonstationary noise and rolling bands in the images which find their way into the calibrated pixel time series and ultimately into the calibrated target flux time series. Using a combination of raw science pixel data, full frame images, reverse-clocked pixel data and ancillary temperature data the Keplerpipeline models and removes the FGS crosstalk artifacts by dynamically adjusting the black level correction. By examining the residuals to the model fits, the pipeline detects and flags spatial regions and time intervals of strong time-varying blacklevel (rolling bands ) on a per row per cadence basis. These flags are made available to downstream users of the data since the uncorrected rolling band artifacts could complicate processing or lead to misinterpretation of instrument behavior as stellar. This model fitting and artifact flagging is performed within the new stand-alone pipeline model called Dynablack. We discuss the implementation of Dynablack in the Kepler data pipeline and present results regarding the improvement in calibrated pixels and the expected improvement in cotrending performances as a result of including FGS corrections in the calibration. We also discuss the effectiveness of the rolling band flagging for downstream users and illustrate with some affected light curves.

Clarke, B. D.↗
NASA NTRS2016

IN13B-1660: Analytics and Visualization Pipelines for Big Data on the NASA Earth Exchange (NEX) and OpenNEX

We are developing capabilities for an integrated petabyte-scale Earth science collaborative analysis and visualization environment. The ultimate goal is to deploy this environment within the NASA Earth Exchange (NEX) and OpenNEX in order to enhance existing science data production pipelines in both high-performance computing (HPC) and cloud environments. Bridging of HPC and cloud is a fairly new concept under active research and this system significantly enhances the ability of the scientific community to accelerate analysis and visualization of Earth science data from NASA missions, model outputs and other sources. We have developed a web-based system that seamlessly interfaces with both high-performance computing (HPC) and cloud environments, providing tools that enable science teams to develop and deploy large-scale analysis, visualization and QA pipelines of both the production process and the data products, and enable sharing results with the community. Our project is developed in several stages each addressing separate challenge - workflow integration, parallel execution in either cloud or HPC environments and big-data analytics or visualization. This work benefits a number of existing and upcoming projects supported by NEX, such as the Web Enabled Landsat Data (WELD), where we are developing a new QA pipeline for the 25PB system.

visualization↗
NASA NTRS2019

TESS Science Processing Operations Center Pipeline and Data Products

TESS launched 18 April 2018 to conduct a two-year, near all-sky survey for at least 50 small, nearby exoplanets for which masses can be ascertained and whose atmospheres can be characterized by ground- and space-based follow-on observations. TESS just completed its survey of the southern hemisphere, identifying >600 candidate exoplanets and unveiling a plethora of exciting non-exoplanet astrophysics results, such as asteroseismology, asteroids, and supernova. The TESS Science Processing Operations Center (SPOC) processes the data downlinked every two weeks to generate a range of data products hosted at the Mikulski Archive for Space Telescopes (MAST). For each sector (~1 month) of observations, the SPOC calibrates the image data for both 30-min Full Frame Images (FFIs) and up to 20,000 pre-selected 2-min target star postage stamps. Data products for the 2-min targets include simple aperture photometry and systematic error-corrected flux time series. The SPOC also conducts searches for transiting exoplanets in the 2-min data for each sector and generates Data Validation time series and associated reports for each transit-like feature identified in the search. Multi-sector searches for exoplanets are conducted periodically to discover longer period planets, including those in the James Webb Continuous Viewing Zone (CVZ), which are observed for up to one year. Data products also include co-trending basis vectors (CBVs) and calibration files, such as the Pixel Response Functions across the field of view of each of TESS's four cameras. To maximize the usability, the TESS science data products are modeled after those for Kepler, including Target Pixel Files and Light Curve files.In this talk, I describe the SPOC pipeline and the chief differences between the TESS and the Kepler pipelines, and the major updates to the SPOC pipeline (4.0) available now to the community at MAST. I also discuss the documentation available to the community to help them in properly interpreting and analyzing the TESS data products.The TESS Mission is funded by NASA's Science Mission Directorate as an Astrophysics Explorer Mission.

Jenkins, Jon M.↗
NASA NTRS2019

A Planning Pipeline for Large Multi-Agent Missions

In complex multi-agent applications, human operators are often tasked with planning and managing large heterogeneous teams of humans and autonomous vehicles. Although the use of these autonomous vehicles broadens the scope of meaningful applications, many of their systems remain unintuitive and difficult to master for human operators whose expertise lies in the application domain and not at the platform level. Current research focuses on the development of individual capabilities necessary to plan multi-agent missions of this scope, placing little emphasis on the integration of these components in to a full pipeline. The work presented in this paper presents a complete and user-agnostic planning pipeline for large multiagent missions known as the HOLII GRAILLE. The system takes a holistic approach to mission planning by integrating capabilities in human machine interaction, flight path generation, and validation and verification. Components – modules – of the pipeline are explored on an individual level, as well as their integration into a whole system. Lastly, implications for future mission planning are discussed.

Chandarana, Meghan↗
NASA NTRSDate not supplied

Development and Demonstration of a Digital NDE Pipeline for Streamlined Analysis of Ultrasonic Data

Currently when nondestructive evaluation (NDE) is performed on composite structures, the results, although recorded digitally are often manually interpreted and indicated (drawn) on the part being inspected by hand. Following this, a determination must be made on how to disposition that part. This decision could be based on engineering guidelines and best practices, rule-of-thumb, expert opinion or finite-element analysis of the part with some approximate representation of the damage. In order to streamline this process for ultrasonic inspection an effort was undertaken as part of NASA Advanced Composites Project to develop the Digital NDE Pipeline. The Digital NDE Pipeline is an integrated tool suite and associated framework that streamlines the inspection and defect disposition process through model-assisted inspection optimization, automated defect analysis of NDE data, and mapping of NDE data into finite element analysis software. This paper will provide an overview of the Digital NDE Pipeline and provide details of the individual tools developed along with the results of applying these tools to a demonstration test case.

Composites↗
NASA NTRSDate not supplied

TESS Science Processing Operations Center Pipeline Status and Updates

The past eighteen months have seen a number of important changes for the TESS Science Processing Operations Center (SPOC) and our archival data products as TESS embarked upon its first extended mission. First, the SPOC developed and deployed a new 20-sec cadence pipeline, promising to unveil exciting new astrophysics at these short timescales for up to 1000 targets per observing sector. We also developed an FFI light curve pipeline that creates light curves and associated data products for up to 160,000 targets in each sector and archive these as High-Level Science Products (HLSP) at the Mikulski Archive for Space Telescopes (MAST). Soon we plan to perform transiting planet searches on these light curves and to release Data Validation reports and associated data products to the MAST. We also present results from the first multi-year transiting planet search of sectors 1 through 36. Finally, we discuss major changes to the SPOC pipeline that motivated the reprocessing of the first year of data, including the application of target-and cadence-specific scattered light flags, and an update to the sky background correction algorithm to mitigate bias in the original algorithm for dim and/or severely crowded stars.

TESS↗
NASA NTRSDate not supplied

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan Dejesus Oribello↗
NASA NTRSDate not supplied

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan D Oribello↗
NASA NTRSDate not supplied

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗
NASA NTRS1974

A note on parallel and pipeline computation of fast unitary transforms

The parallel and pipeline organization of fast unitary transform algorithms such as the Fast Fourier Transform are discussed. The efficiency is pointed out of a combined parallel-pipeline processor of a transform such as the Haar transform in which 2 to the n minus 1 power hardware butterflies generate a transform of order 2 to the n power every computation cycle.

Fino, B. J.↗
NASA NTRS1975

Parallel and pipeline computation of fast unitary transforms

The letter discusses the parallel and pipeline organization of fast-unitary-transform algorithms such as the fast Fourier transform, and points out the efficiency of a combined parallel-pipeline processor of a transform such as the Haar transform, in which (2 to the n-th power) -1 hardware 'butterflies' generate a transform of order 2 to the n-th power every computation cycle.

Fino, B. J.↗
NASA NTRS1979

Improved coal-slurry pipeline

High strength steel pipeline carries hot mixture of powdered coal and coal derived oil to electric-power-generating station. Slurry is processed along way to remove sulfur, ash, and nitrogen and to recycle part of oil. System eliminates hazards and limitations associated with anticipated coal/water-slurry pipelines.

Dowler, W. L.↗
NASA NTRS1981

Study of Near-Threshold Fatigue Crack Propagation in Pipeline Steels in High Pressure Environments

Near threshold fatigue crack propagation in pipeline steels in high pressure environments was studied. The objective was to determine the level of threshold stress intensity for fatigue crack growth rate behavior in a high strength low alloy X60 pipeline-type steel. Complete results have been generated for gaseous hydrogen at ambient pressure, laboratory air at ambient pressure and approximately 60% relative humidity as well as vacuum of 0.000067 Pa ( 0.0000005 torr) at R-ratios = K(min)/K(max) of 0.1, 0.5, and 0.8. Fatigue crack growth rate behavior in gaseous hydrogen, methane, and methane plus 10 percent hydrogen at 6.89 MPa (100 psi) was determined.

Mitchell, M.↗
NASA NTRS1987

On the VLSI design of a pipeline Reed-Solomon decoder using systolic arrays

A new very large scale integration (VLSI) design of a pipeline Reed-Solomon decoder is presented. The transform decoding technique used in a previous article is replaced by a time domain algorithm through a detailed comparison of their VLSI implementations. A new architecture that implements the time domain algorithm permits efficient pipeline processing with reduced circuitry. Erasure correction capability is also incorporated with little additional complexity. By using a multiplexing technique, a new implementation of Euclid's algorithm maintains the throughput rate with less circuitry. Such improvements result in both enhanced capability and significant reduction in silicon area.

Shao, H. M.↗