Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Dynamic Black-Level Correction and Artifact Flagging in the Kepler Data Pipeline

Instrument-induced artifacts in the raw Kepler pixel data include time-varying crosstalk from the fine guidance sensor (FGS) clock signals, manifestations of drifting moiré pattern as locally correlated nonstationary noise and rolling bands in the images which find their way into the calibrated pixel time series and ultimately into the calibrated target flux time series. Using a combination of raw science pixel data, full frame images, reverse-clocked pixel data and ancillary temperature data the Keplerpipeline models and removes the FGS crosstalk artifacts by dynamically adjusting the black level correction. By examining the residuals to the model fits, the pipeline detects and flags spatial regions and time intervals of strong time-varying blacklevel (rolling bands ) on a per row per cadence basis. These flags are made available to downstream users of the data since the uncorrected rolling band artifacts could complicate processing or lead to misinterpretation of instrument behavior as stellar. This model fitting and artifact flagging is performed within the new stand-alone pipeline model called Dynablack. We discuss the implementation of Dynablack in the Kepler data pipeline and present results regarding the improvement in calibrated pixels and the expected improvement in cotrending performances as a result of including FGS corrections in the calibration. We also discuss the effectiveness of the rolling band flagging for downstream users and illustrate with some affected light curves.

Clarke, B. D.↗

IN13B-1660: Analytics and Visualization Pipelines for Big Data on the NASA Earth Exchange (NEX) and OpenNEX

We are developing capabilities for an integrated petabyte-scale Earth science collaborative analysis and visualization environment. The ultimate goal is to deploy this environment within the NASA Earth Exchange (NEX) and OpenNEX in order to enhance existing science data production pipelines in both high-performance computing (HPC) and cloud environments. Bridging of HPC and cloud is a fairly new concept under active research and this system significantly enhances the ability of the scientific community to accelerate analysis and visualization of Earth science data from NASA missions, model outputs and other sources. We have developed a web-based system that seamlessly interfaces with both high-performance computing (HPC) and cloud environments, providing tools that enable science teams to develop and deploy large-scale analysis, visualization and QA pipelines of both the production process and the data products, and enable sharing results with the community. Our project is developed in several stages each addressing separate challenge - workflow integration, parallel execution in either cloud or HPC environments and big-data analytics or visualization. This work benefits a number of existing and upcoming projects supported by NEX, such as the Web Enabled Landsat Data (WELD), where we are developing a new QA pipeline for the 25PB system.

visualization↗

TESS Science Processing Operations Center Pipeline and Data Products

TESS launched 18 April 2018 to conduct a two-year, near all-sky survey for at least 50 small, nearby exoplanets for which masses can be ascertained and whose atmospheres can be characterized by ground- and space-based follow-on observations. TESS just completed its survey of the southern hemisphere, identifying >600 candidate exoplanets and unveiling a plethora of exciting non-exoplanet astrophysics results, such as asteroseismology, asteroids, and supernova. The TESS Science Processing Operations Center (SPOC) processes the data downlinked every two weeks to generate a range of data products hosted at the Mikulski Archive for Space Telescopes (MAST). For each sector (~1 month) of observations, the SPOC calibrates the image data for both 30-min Full Frame Images (FFIs) and up to 20,000 pre-selected 2-min target star postage stamps. Data products for the 2-min targets include simple aperture photometry and systematic error-corrected flux time series. The SPOC also conducts searches for transiting exoplanets in the 2-min data for each sector and generates Data Validation time series and associated reports for each transit-like feature identified in the search. Multi-sector searches for exoplanets are conducted periodically to discover longer period planets, including those in the James Webb Continuous Viewing Zone (CVZ), which are observed for up to one year. Data products also include co-trending basis vectors (CBVs) and calibration files, such as the Pixel Response Functions across the field of view of each of TESS's four cameras. To maximize the usability, the TESS science data products are modeled after those for Kepler, including Target Pixel Files and Light Curve files.In this talk, I describe the SPOC pipeline and the chief differences between the TESS and the Kepler pipelines, and the major updates to the SPOC pipeline (4.0) available now to the community at MAST. I also discuss the documentation available to the community to help them in properly interpreting and analyzing the TESS data products.The TESS Mission is funded by NASA's Science Mission Directorate as an Astrophysics Explorer Mission.

Jenkins, Jon M.↗

A Planning Pipeline for Large Multi-Agent Missions

In complex multi-agent applications, human operators are often tasked with planning and managing large heterogeneous teams of humans and autonomous vehicles. Although the use of these autonomous vehicles broadens the scope of meaningful applications, many of their systems remain unintuitive and difficult to master for human operators whose expertise lies in the application domain and not at the platform level. Current research focuses on the development of individual capabilities necessary to plan multi-agent missions of this scope, placing little emphasis on the integration of these components in to a full pipeline. The work presented in this paper presents a complete and user-agnostic planning pipeline for large multiagent missions known as the HOLII GRAILLE. The system takes a holistic approach to mission planning by integrating capabilities in human machine interaction, flight path generation, and validation and verification. Components – modules – of the pipeline are explored on an individual level, as well as their integration into a whole system. Lastly, implications for future mission planning are discussed.

Chandarana, Meghan↗

Development and Demonstration of a Digital NDE Pipeline for Streamlined Analysis of Ultrasonic Data

Currently when nondestructive evaluation (NDE) is performed on composite structures, the results, although recorded digitally are often manually interpreted and indicated (drawn) on the part being inspected by hand. Following this, a determination must be made on how to disposition that part. This decision could be based on engineering guidelines and best practices, rule-of-thumb, expert opinion or finite-element analysis of the part with some approximate representation of the damage. In order to streamline this process for ultrasonic inspection an effort was undertaken as part of NASA Advanced Composites Project to develop the Digital NDE Pipeline. The Digital NDE Pipeline is an integrated tool suite and associated framework that streamlines the inspection and defect disposition process through model-assisted inspection optimization, automated defect analysis of NDE data, and mapping of NDE data into finite element analysis software. This paper will provide an overview of the Digital NDE Pipeline and provide details of the individual tools developed along with the results of applying these tools to a demonstration test case.

Composites↗

TESS Science Processing Operations Center Pipeline Status and Updates

The past eighteen months have seen a number of important changes for the TESS Science Processing Operations Center (SPOC) and our archival data products as TESS embarked upon its first extended mission. First, the SPOC developed and deployed a new 20-sec cadence pipeline, promising to unveil exciting new astrophysics at these short timescales for up to 1000 targets per observing sector. We also developed an FFI light curve pipeline that creates light curves and associated data products for up to 160,000 targets in each sector and archive these as High-Level Science Products (HLSP) at the Mikulski Archive for Space Telescopes (MAST). Soon we plan to perform transiting planet searches on these light curves and to release Data Validation reports and associated data products to the MAST. We also present results from the first multi-year transiting planet search of sectors 1 through 36. Finally, we discuss major changes to the SPOC pipeline that motivated the reprocessing of the first year of data, including the application of target-and cadence-specific scattered light flags, and an update to the sky background correction algorithm to mitigate bias in the original algorithm for dim and/or severely crowded stars.

TESS↗

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan Dejesus Oribello↗

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan D Oribello↗

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗

Sensor and transmitter system for communication in pipelines

A system for sensing and communicating in a pipeline that contains a fluid. An acoustic signal containing information about a property of the fluid is produced in the pipeline. The signal is transmitted through the pipeline. The signal is received with the information and used by a control.

47 OTHER INSTRUMENTATION↗

Implementing the LSST Software Stack for DESGW processing and the Integration of Convolutional Neural Networks into the DESGW Pipeline

The Dark Energy Survey Gravitational Wave (DESGW) group strives to understand thelargescale structure of the universe and galaxies by looking for electromagnetic signatures intelescope images following gravitational wave detections. The DESGW group uses a processing pipeline to perform difference imaging to look for potential candidates. In order to perform these searches more effectively, we first explored using an alternative processing pipeline, the LSST Software Stack to analyze the telescope images. Because the LSST software stack is currently transitioning between its generation 2 and generation 3 system, we decided that while the software will be usable in the future once generation 3 is complete, at the moment its incompleteness makes it impractical to use. We then decided to work on improving the current pipeline by developing a module that integrates a Convolutional Neural Network (CNN) to test difference imaging products for bad subtractions due to misalignment.

Navarro, Alexander↗

A Data Processing Pipeline for Adversarial Socio-Technical Network Analysis

With the rapid adoption of emerging technologies, there is a need to catalog and model sociotechnical interdependencies that have been historically used to influence the operation of Critical Infrastructure networks including the impacts of mergers and acquisitions, hostile takeovers, and foreign investment. Our research intends to address this need with two primary contributions. First, we have developed a data curation and processing pipeline to generate sociotechnical networks extracted from a variety of data sources including SEC filings and infrastructure asset databases. The pipeline, implemented in Apache Airflow, extracts and normalizes the representation of entities and relations, specified within ontologies. Our intent is to provide an extensible, machine-actionable approach to quickly communicate such models, reproduce previous results, and adapt them to new, unanticipated situations. Second, networks produced by our pipeline enable the development of graph-theoretic metrics that consider the properties of network components in addition to its topology. Metadata associated with network components---whether semantic, temporal, or geospatial---affects the alignment of generated networks with assumptions underlying complexity metrics. Validation of generated networks relative to component types defined by an ontology, may allow the research community to adapt metrics to the semantics of the domains being studied. Generated networks may be processed as knowledge, dynamic, or spatial graphs and enables a variety of analyses including automated reasoning and measures of network complexity. Automated reasoning views extracted entities and relations as a knowledge graph; this enables application of inference rules that represent historically-attested adversarial business methods and applies that behavior to a specific geographic context. Measures of network complexity, including degree distribution, reachability analyses, temporal analysis, and community detection can be adapted to indicate adversarial organizational influence.

97 MATHEMATICS AND COMPUTING↗

Utilizing the Shift Technical Engineer Pipeline to Develop the Next Generation of Engineers - 17354

As the liquid waste contractor for the U.S. Department of Energy's (DOE) Savannah River Site (SRS), Savannah River Remediation (SRR) is responsible for safely treating and dispositioning approximately 35 million gallons of liquid waste stored at SRS. To achieve this mission, SRR manages and operates several complex nuclear facilities. These nuclear facilities include the H and F Tank Farms, the Defense Waste Processing Facility (DWPF), and the Saltstone Processing and Disposal Facility. To support the operations of these complex nuclear facilities, a Shift Technical Engineer (STE) is stationed for every shift. The STE is the sole Engineering Organization representative on-shift and serves as a direct technical resource for operations, including safety basis interpretation, process monitoring and prompt troubleshooting support. The training and qualification for an STE is more intensive than for typical technical staff personnel at SRR, involving additional classes, under-instruction watches, demonstration of practical factors, and culminating in an intense oral board examination. Like many other contractors in the DOE complex, SRR faces the challenge of preparing a significant number of new generation engineers to take the place of an aging workforce. Beyond the traditional training to establish minimum technical qualifications, SRR has established a number of initiatives aimed at accelerating the process of preparing early career engineers to replace experienced senior engineers. These initiatives include an internal rotational program, technical exchange activities with companies at other DOE sites, and an STE pipeline. The STE role provides a unique mechanism for select, early career engineers to gain in-depth knowledge and significant responsibility normally not achieved until much later in their careers. Once qualification is complete, the engineers perform the STE duties, providing technical support for nuclear facility operations. In addition, creating an STE pipeline, where promising young engineers are continually being qualified to replace the on-shift STEs, provides the company with a source of experienced engineers with in-depth technical knowledge as well as valuable experience interfacing with operations and other facility support groups. This paper will discuss the STE position, the qualification process, and how SRR is using the STE qualification pipeline to benefit both the company and the individual engineer. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

An Analysis of DES Cluster Simulations through the IMCAT and Shapelets Weak Lensing Pipelines

We have run two completely independent weak lensing analysis pipelines on a set of realistic simulated images of a massive galaxy cluster with a singular isothermal sphere profile (galaxy velocity dispersion sigma_v=1250 km/ sec). The suite of images was constructed using the simulation tools developed by the Dark Energy Survey. We find that both weak lensing pipelines can accurately recover the velocity dispersion of our simulated clusters, suggesting that current weak lensing tools are accurate enough for measuring the shear profile of massive clusters in upcoming large photometric surveys. We also demonstrate how choices of some cuts influence the final shear profile and sigma_v measurement. Analogously to the STEP program, we make all of these cluster simulation images publically available for other groups to analyze through their own weak lensing pipelines.

Gill, M. S.S.↗

Evaluation of NETL’s Self-Healing Metallic Coating for Internal Corrosion Protection of Natural Gas Pipelines: Field Test

Steel pipelines are a safe, reliable, and affordable way to transport natural gas. However, the presence of impurities (e.g., water, carbon dioxide, and hydrogen sulfide) in the natural gas can cause internal corrosion, which can lead to pipeline leaks and failure. This paper reports on field tests of an innovative self-healing, corrosion-resistant, metallic coating developed at the National Energy Technology Laboratory (NETL) for protecting the interior surfaces of pipelines.

carbon dioxide (CO2)↗

The JCSG MR Pipeline: Optimized Alignments, Multiple Models And Parallel Searches

The success rate of molecular replacement (MR) falls considerably when search models share less than 35% sequence identity with their templates, but can be improved significantly by using fold-recognition methods combined with exhaustive MR searches. Models based on alignments calculated with fold-recognition algorithms are more accurate than models based on conventional alignment methods such as FASTA or BLAST, which are still widely used for MR. In addition, by designing MR pipelines that integrate phasing and automated refinement and allow parallel processing of such calculations, one can effectively increase the success rate of MR. Here, updated results from the JCSG MR pipeline are presented, which to date has solved 33 MR structures with less than 35% sequence identity to the closest homologue of known structure. By using difficult MR problems as examples, it is demonstrated that successful MR phasing is possible even in cases where the similarity between the model and the template can only be detected with fold-recognition algorithms. In the first step, several search models are built based on all homologues found in the PDB by fold-recognition algorithms. The models resulting from this process are used in parallel MR searches with different combinations of input parameters of the MR phasing algorithm. The putative solutions are subjected to rigid-body and restrained crystallographic refinement and ranked based on the final values of free R factor, figure of merit and deviations from ideal geometry. Finally, crystal packing and electron-density maps are checked to identify the correct solution. If this procedure does not yield a solution with interpretable electron-density maps, then even more alternative models are prepared. The structurally variable regions of a protein family are identified based on alignments of sequences and known structures from that family and appropriate trimmings of the models are proposed. All combinations of these trimmings are applied to the search models and the resulting set of models is used in the MR pipeline. It is estimated that with the improvements in model building and exhaustive parallel searches with existing phasing algorithms, MR can be successful for more than 50% of recognizable homologues of known structures below the threshold of 35% sequence identity. This implies that about one-third of the proteins in a typical bacterial proteome are potential MR targets.

59 BASIC BIOLOGICAL SCIENCES↗

Fatigue-Crack-Growth Behavior of Two Pipeline Steels

This paper focuses on studying the fatigue-crack-growth behavior of two types of pipeline steels, and investigating their microstructural differences, which could influence the fatigue behavior. For fatigue experiments, compact-tension (CT) specimens are employed. These two kinds of base pipeline steels are Alloy B [Fe-0.05C-1.52Mn-0.12Si-0.092Nb, weight percent (wt.%)] and Alloy C [(Fe- 0.04C-1.61Mn-0.14Si-0.096Nb, wt.%)]. They have been tested at various frequencies (10 Hz, 1 Hz, and 0.1 Hz) and different R ratios (0.1 and 0.5, R = P min. /P max. where P min. is the minimum applied load, and P max. is the maximum applied load) in air. The effects of frequencies and R ratios on crackpropagation behavior are compared. The microstructures of fracture surfaces are investigated, using both scanning-electron microscopy (SEM) and transmission-electron microscopy (TEM). It is concluded that higher R ratios lead to faster crack-growth rates, while frequency does not have much influence on the fatigue-crack-growth rates. Moreover, Alloy B (Fe-0.05C-1.52Mn-0.12Si-0.092Nb, wt.%) tends to have better fatigue resistance than Alloy C (Fe-0.04C-1.61Mn-0.14Si-0.096Nb, wt.%) under various test conditions in air.

36 MATERIALS SCIENCE↗

Automated Radiology Report Summarization Using an Open-Source Natural Language Processing Pipeline

Diagnostic radiologists are expected to review and assimilate findings from prior studies when constructing their overall assessment of the current study. Radiology information systems facilitate this process by presenting the radiologist with a subset of prior studies that are more likely to be relevant to the current study, usually by comparing anatomic coverage of both the current and prior studies. It is incumbent on the radiologist to review the full text report and/or images from those prior studies, a process that is time-consuming and confers substantial risk of overlooking a relevant prior study or finding. This risk is compounded when patients have dozens or even hundreds of prior imaging studies. Our goal is to assess the feasibility of natural language processing techniques to automatically extract asserted and negated disease entities from free-text radiology reports as a step towards automated report summarization. We compared automatically extracted disease mentions to a gold-standard set of manual annotations for 50 radiology reports from CT abdomen and pelvis examinations. The automated report summarization pipeline found perfect or overlapping partial matches for 86% of the manually annotated disease mentions (sensitivity 0.86, precision 0.66, accuracy 0.59, F1 score 0.74). The performance of the automated pipeline was good, and the overall accuracy was similar to the interobserver agreement between the two manual annotators.

62 RADIOLOGY AND NUCLEAR MEDICINE↗