Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “PIPELINE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Instrumental Response Model and Detrending for the Dark Energy Camera

We describe the model for mapping from sky brightness to the digital output of the Dark Energy Camera (DECam) and the algorithms adopted by the Dark Energy Survey (DES) for inverting this model to obtain photometric measures of celestial objects from the raw camera output. This calibration aims for fluxes that are uniform across the camera field of view and across the full angular and temporal span of the DES observations, approaching the accuracy limits set by shot noise for the full dynamic range of DES observations. The DES pipeline incorporates several substantive advances over standard detrending techniques, including principal-components-based sky and fringe subtraction; correction of the "brighter-fatter" nonlinearity; use of internal consistency in on-sky observations to disentangle the influences of quantum efficiency, pixel-size variations, and scattered light in the dome flats; and pixel-by-pixel characterization of instrument spectral response, through combination of internal-consistency constraints with auxiliary calibration data. This article provides conceptual derivations of the detrending/calibration steps, and the procedures for obtaining the necessary calibration data. Other publications will describe the implementation of these concepts for the DES operational pipeline, the detailed methods, and the validation that the techniques can bring DECam photometry and astrometry within $\approx 2$ mmag and $\approx 3$ mas, respectively, of fundamental atmospheric and statistical limits. In conclusion, the DES techniques should be broadly applicable to wide-field imagers.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Data-driven selection and spectral classification of white dwarf stars

The next generation of spectroscopic surveys is expected to provide spectra for hundreds of thousands of white dwarf (WD) candidates in the upcoming years. Currently, spectroscopic classification of white dwarfs is mostly done by visual inspection, requiring substantial amounts of expert attention. We propose a data-driven pipeline for fast, automatic selection, and spectroscopic classification of WD candidates, trained using spectroscopically confirmed objects with available Gaia astrometry, photometry, and Sloan Digital Sky Survey (SDSS) spectra with signal-to-noise ratios ≥9. The pipeline selects WD candidates with improved accuracy and completeness over existing algorithms, classifies their primary spectroscopic type with ≥ 90 % accuracy, and spectroscopically detects main sequence companions with similar performance. We apply our pipeline to the Gaia Data Release 3 cross-matched with the SDSS Data Release 17 (DR17), identifying 424 096 high-confidence WD candidates and providing the first catalogue of automated and quantifiable classification for 36 523 WD spectra. Both the catalogue and pipeline are made available online. Such a tool will prove particularly useful for the undergoing SDSS-V survey, allowing for rapid classification of thousands of spectra at every data release.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

BRAKER3: Fully automated genome annotation using RNA-seq and protein evidence with GeneMark-ETP, AUGUSTUS, and TSEBRA

Gene prediction has remained an active area of bioinformatics research for a long time. Still, gene prediction in large eukaryotic genomes presents a challenge that must be addressed by new algorithms. The amount and significance of the evidence available from transcriptomes and proteomes vary across genomes, between genes, and even along a single gene. User-friendly and accurate annotation pipelines that can cope with such data heterogeneity are needed. The previously developed annotation pipelines BRAKER1 and BRAKER2 use RNA-seq or protein data, respectively, but not both. A further significant performance improvement integrating all three data types was made by the recently released GeneMark-ETP. We here present the BRAKER3 pipeline that builds on GeneMark-ETP and AUGUSTUS, and further improves accuracy using the TSEBRA combiner. BRAKER3 annotates protein-coding genes in eukaryotic genomes using both short-read RNA-seq and a large protein database, along with statistical models learned iteratively and specifically for the target genome. We benchmarked the new pipeline on genomes of 11 species under an assumed level of relatedness of the target species proteome to available proteomes. BRAKER3 outperforms BRAKER1 and BRAKER2. The average transcript-level F1-score is increased by about 20 percentage points on average, whereas the difference is most pronounced for species with large and complex genomes. BRAKER3 also outperforms other existing tools, MAKER2, Funannotate, and FINDER. The code of BRAKER3 is available on GitHub and as a ready-to-run Docker container for execution with Docker or Singularity. Overall, BRAKER3 is an accurate, easy-to-use tool for eukaryotic genome annotation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

High-Performance Deep Learning Toolbox for Genome-Scale Prediction of Protein Structure and Function

Computational biology is one of many scientific disciplines ripe for innovation and acceleration with the advent of high-performance computing (HPC). In recent years, the field of machine learning has also seen significant benefits from adopting HPC practices. In this work, we present a novel HPC pipeline that incorporates various machine-learning approaches for structure-based functional annotation of proteins on the scale of whole genomes. Our pipeline makes extensive use of deep learning and provides computational insights into best practices for training advanced deep-learning models for high-throughput data such as proteomics data. We showcase methodologies our pipeline currently supports and detail future tasks for our pipeline to envelop, including large-scale sequence comparison using SAdLSA and prediction of protein tertiary structures using AlphaFold2.

Gao, Mu↗
DOE OSTI2024

CONSTRAINT-INDEPENDENT CONSTANT CTOA DETERMINATION FOR DUCTILE STABLE CRACK GROWTH

Crack tip opening angle (CTOA) has been used as a reliable fracture toughness parameter for decades to characterize stable ductile crack growth for thin-walled aerospace structures in the low-constraint conditions. Recently, the CTOA parameter was also applied to the pipeline industry, and a CTOA test standard ASTM E3039 was thus developed for testing a critical constant CTOA. Research showed that the constant CTOA can reasonably describe fracture toughness required to arrest a dynamic crack propagation for a modern gas pipeline. However, the CTOA fracture criterion requires constraint-independent CTOA toughness against stable ductile crack growth. ASTM E3039 recommends a drop weight tearing test (DWTT) specimen for CTOA testing. Since a shallow crack is used, DWTT measured CTOA may depend on constraint level at the crack tip. To understand if it is the case, this paper evaluates the critical CTOA for a set of fracture toughness tests on single edge notched bend (SENB) specimens with shallow and deep cracks based on four CTOA estimation models. In which, the Ln(P)-LLD linear fit model is similar to that used by ASTM E3039 in the CTOA calculation. Fracture test data for X80 pipeline steel and HY80 structural steel are considered in the CTOA evaluation. The results show that the four CTOA models can determine a crack size-independent constant CTOA over stable ductile crack growth for the SENB specimens. As a result, CTOA determined by ASTM E3039 is constraintindependent and transferable to use for an actual crack propagating in a gas pipeline.

Zhu, Xian-Kui↗
DOE OSTI2024

EXPERIMENTAL VALIDATION OF THEORETICAL BURST STRENGTH SOLUTION FOR DEFECT-FREE THICK-WALLED PIPES

The burst pressure of line pipes is an important strength property required in pipeline design and integrity management. Historically, the Barlow formula in conjunction with the ultimate tensile stress (UTS) of pipeline steels were utilized to estimate the burst strength of line pipes. However, the Barlow formula did not consider the plastic flow effect for ductile steels and is applicable only to thin-walled pipes. In 2006, the present author proposed a new multiaxial plastic yield theory and obtained a theoretical Zhu-Leis solution of burst strength for defect-free thin-walled pipes in term of UTS and strain hardening exponent n of pipeline steels. The Zhu-Leis solution has been validated by various burst test data for thin-walled pipelines for a wide range of steel grades from Grade B to X120. Recently, the present author extended the Zhu-Leis theory of plasticity to thick-walled pipes and obtained the Zhu-Leis solution of burst pressure for thick-walled pipes. The proposed burst pressure solution is applicable to both thin and thick-walled pipes. To experimentally validate the proposed theoretical burst pressure solution, this paper obtains a set of burst test data for three thick-walled pipes in Grade B carbon steel with a nominal diameter of 2.375 inches and three nominal wall thicknesses, resulting in D/t = 15.4, 10.9, 6.9. Through comparisons, these burst data validate the theoretical burst pressure solution for thick-walled pipes. Moreover, two additional burst test datasets collected from literature for thin and thick-walled pipes further validate the proposed burst pressure solution for both thin and thick-walled pipes.

Zhu, Xiankui↗
DOE OSTI2015

Microbial Community Analysis with Ribosomal Gene Fragments from Shotgun Metagenomes

Shotgun metagenomic sequencing does not depend on gene-targeted primers or PCR amplification; thus, it is not affected by primer bias or chimeras. However, searching rRNA genes from large shotgun Illumina data sets is computationally expensive, and no approach exists for unsupervised community analysis of small-subunit (SSU) rRNA gene fragments retrieved from shotgun data. We present a pipeline, SSUsearch, to achieve the faster identification of short-subunit rRNA gene fragments and enabled unsupervised community analysis with shotgun data. It also includes classification and copy number correction, and the output can be used by traditional amplicon analysis platforms. Shotgun metagenome data using this pipeline yielded higher diversity estimates than amplicon data but retained the grouping of samples in ordination analyses. We applied this pipeline to soil samples with paired shotgun and amplicon data and confirmed bias against Verrucomicrobia in a commonly used V6-V8 primer set, as well as discovering likely bias against Actinobacteria and for Verrucomicrobia in a commonly used V4 primer set. In conclusion, this pipeline can utilize all variable regions in SSU rRNA and also can be applied to large-subunit (LSU) rRNA genes for confirmation of community structure.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Streaming Analytics and Data Engineering

SADE is a software package for rapidly assembling analytic pipelines to manipulate data. The packages consists of the engine that manages the data and coordinates the movement of data between the tasks performing a function, a set of core libraries consisting of plugins that perform common tasks, and a framework to extend the system supporting the development of new plugins. Currently through configuration files, a pipeline can be defined that maps the routing of data through a series of plugins. Pipelines can be run in a batch mode or can process streaming data; they can be executed from the command line or run through a Windows background service. There currently exists over a hundred plugins, over fifty pipeline configurations, and the software is now being used by about a half-dozen projects.

Source record↗
DOE OSTI2022

Real-Time Highly Resolved Spatial-Temporal Vehicle Energy Consumption Estimation Using Machine Learning and Probe Data

Real-time highly resolved spatial-temporal vehicle energy consumption is a key missing dimension in transportation data. Most roadway link-level vehicle energy consumption data are estimated using average annual daily traffic measures derived from the Highway Performance Monitoring System; however, this method does not reflect day-to-day energy consumption fluctuations. As transportation planners and operators are becoming more environmentally attentive, they need accurate real-time link-level vehicle energy consumption data to assess energy and emissions; to incentivize energy-efficient routing; and to estimate energy impact caused by congestion, major events, and severe weather. This paper presents a computational workflow to automate the estimation of time-resolved vehicle energy consumption for each link in a road network of interest using vehicle probe speed and count data in conjunction with machine learning methods in real time. The real-time pipeline can deliver energy estimates within a couple seconds on query to its interface. The proposed method was evaluated on the transportation network of the metropolitan area of Chattanooga, Tennessee. The volume estimation results were validated with ground truth traffic volume data collected in the field. To demonstrate the effectiveness of the proposed method, the energy consumption pipeline was applied to real-world data to quantify road transportation-related energy reduction because of mitigation policies to slow the spread of COVID-19 and to measure energy loss resulting from congestion.

Severino, Joseph↗
DOE OSTI2016

TagDigger: user-friendly extraction of read counts from GBS and RAD-seq data

In genotyping-by-sequencing (GBS) and restriction site-associated DNA sequencing (RAD-seq), read depth is important for assessing the quality of genotype calls and estimating allele dosage in polyploids. However, existing pipelines for GBS and RAD-seq do not provide read counts in formats that are both accurate and easy to access. Additionally, although existing pipelines allow previously-mined SNPs to be genotyped on new samples, they do not allow the user to manually specify a subset of loci to examine. Pipelines that do not use a reference genome assign arbitrary names to SNPs, making meta-analysis across projects difficult. We created the software TagDigger, which includes three programs for analyzing GBS and RAD-seq data. The first script, tagdigger_interactive.py, rapidly extracts read counts and genotypes from FASTQ files using user supplied sets of barcodes and tags. Input and output is in CSV format so that it can be opened by spreadsheet software. Tag sequences can also be imported from the Stacks, TASSEL-GBSv2, TASSEL-UNEAK, or pyRAD pipelines, and a separate file can be imported listing the names of markers to retain. A second script, tag_manager.py, consolidates marker names and sequences across multiple projects. A third script, barcode_splitter.py, assists with preparing FASTQ data for deposit in a public archive by splitting FASTQ files by barcode and generating MD5 checksums for the resulting files. TagDigger is open-source and freely available software written in Python 3. It uses a scalable, rapid search algorithm that can process over 100 million FASTQ reads per hour. TagDigger will run on a laptop with any operating system, does not consume hard drive space with intermediate files, and does not require programming skill to use.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2018

Optimized 3D co-registration of ultra-low-field and high-field magnetic resonance images

The prototypes of ultra-low-field (ULF) MRI scanners developed in recent years represent new, innovative, cost-effective and safer systems, which are suitable to be integrated in multi-modal (Magnetoencephalography and MRI) devices. Integrated ULF-MRI and MEG scanners could represent an ideal solution to obtain functional (MEG) and anatomical (ULF MRI) information in the same environment, without errors that may limit source reconstruction accuracy. However, the low resolution and signal-to-noise ratio (SNR) of ULF images, as well as their limited coverage, do not generally allow for the construction of an accurate individual volume conductor model suitable for MEG localization. Thus, for practical usage, a high-field (HF) MRI image is also acquired, and the HF-MRI images are co-registered to the ULF-MRI ones. We address here this issue through an optimized pipeline (SWIM—Sliding WIndow grouping supporting Mutual information). The co-registration is performed by an affine transformation, the parameters of which are estimated using Normalized Mutual Information as the cost function, and Adaptive Simulated Annealing as the minimization algorithm. The sub-voxel resolution of the ULF images is handled by a sliding-window approach applying multiple grouping strategies to down-sample HF MRI to the ULF-MRI resolution. The pipeline has been tested on phantom and real data from different ULF-MRI devices, and comparison with well-known toolboxes for fMRI analysis has been performed. Our pipeline always outperformed the fMRI toolboxes (FSL and SPM). The HF–ULF MRI co-registration obtained by means of our pipeline could lead to an effective integration of ULF MRI with MEG, with the aim of improving localization accuracy, but also to help exploit ULF MRI in tumor imaging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Learning epistatic polygenic phenotypes with Boolean interactions

Detecting epistatic drivers of human phenotypes is a considerable challenge. Traditional approaches use regression to sequentially test multiplicative interaction terms involving pairs of genetic variants. For higher-order interactions and genome-wide large-scale data, this strategy is computationally intractable. Moreover, multiplicative terms used in regression modeling may not capture the form of biological interactions. Building on the Predictability, Computability, Stability (PCS) framework, we introduce the epiTree pipeline to extract higher-order interactions from genomic data using tree-based models. The epiTree pipeline first selects a set of variants derived from tissue-specific estimates of gene expression. Next, it uses iterative random forests (iRF) to search training data for candidate Boolean interactions (pairwise and higher-order). We derive significance tests for interactions, based on a stabilized likelihood ratio test, by simulating Boolean tree-structured null (no epistasis) and alternative (epistasis) distributions on hold-out test data. Finally, our pipeline computes PCS epistasis p-values that probabilisticly quantify improvement in prediction accuracy via bootstrap sampling on the test set. We validate the epiTree pipeline in two case studies using data from the UK Biobank: predicting red hair and multiple sclerosis (MS). In the case of predicting red hair, epiTree recovers known epistatic interactions surrounding MC1R and novel interactions, representing non-linearities not captured by logistic regression models. In the case of predicting MS, a more complex phenotype than red hair, epiTree rankings prioritize novel interactions surrounding HLA-DRB1 , a variant previously associated with MS in several populations. Taken together, these results highlight the potential for epiTree rankings to help reduce the design space for follow up experiments.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2014

Cascading of Fluctuations in Interdependent Energy Infrastructures. Gas-Grid Coupling

The revolution of hydraulic fracturing has dramatically increased the supply and lowered the cost of natural gas in the United States driving an expansion of natural gas-fired generation capacity in many electrical grids. Unrelated to the natural gas expansion, lower capital costs and renewable portfolio standards are driving an expansion of intermittent renewable generation capacity such as wind and photovoltaic generation. These two changes may potentially combine to create new threats to the reliability of these interdependent energy infrastructures. Natural gas-fired generators are often used to balance the fluctuating output of wind generation. However, the time-varying output of these generators results in time-varying natural gas burn rates that impact the pressure in interstate transmission pipelines. Fluctuating pressure impacts the reliability of natural gas deliveries to those same generators and the safety of pipeline operations. We adopt a partial differential equation model of natural gas pipelines and use this model to explore the effect of intermittent wind generation on the fluctuations of pressure in natural gas pipelines. The mean square pressure fluctuations are found to grow linearly in time with points of maximum deviation occurring at the locations of flow reversals.

03 NATURAL GAS↗
DOE OSTI2014

Hydrogen Embrittlement of Structural Steels

Carbon-manganese steels are candidates for the structural materials in hydrogen gas pipelines; however, it is well known that these steels are susceptible to hydrogen embrittlement. Decades of research and industrial experience have established that hydrogen embrittlement compromises the structural integrity of steel components. This experience has also helped identify the failure modes that can operate in hydrogen containment structures. As a result, there are tangible ideas for managing hydrogen embrittlement in steels and quantifying safety margins for steel hydrogen containment structures. For example, fatigue crack growth aided by hydrogen embrittlement is a well-established failure mode for steel hydrogen containment structures subjected to pressure cycling. This pressure cycling represents one of the key differences in operating conditions between current hydrogen pipelines and those anticipated in a hydrogen delivery infrastructure. Applying structural integrity models in design codes coupled with measurement of relevant material properties allows quantification of the reliability/integrity of steel hydrogen pipelines subjected to pressure cycling. Furthermore, application of these structural integrity models is aided by the development of physics-based predictive models, which provide important insights such as the effects of microstructure on hydrogen-assisted fatigue crack growth. Successful implementation of these structural integrity and physics-based models enhances confidence in the design codes and enables decisions about materials selection and operating conditions for reliable and efficient steel hydrogen pipelines.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Development of Ambient Mass Spectrometry Rapid Analysis Methods for Trace Inorganics

Ambient mass spectrometry is being investigated as a new method for rapidly characterizing trace inorganic species, including nuclear materials. SRNL is currently developing electrospray ionization (ESI) and paper spray ionization (PSI) methods for mass spectrometry for the detection of trace inorganic compounds JEOL AccuTOF™-DART ® LP 4G mass spectrometer. Sample analysis generates a complex three-dimensional data set with 10 2 -10 3 individual mass spectra spread temporally. To improve the quality and ease of data extraction from this amount of information, a simple data analysis pipeline written in R/R Studio has been improved and expanded upon. Specifically, the software now has header inputs to set base input/output filenames, isotopes of interest, and a list of concentrations analyzed on the instrument. Using list-based structures to enable batch processing of many concentrations at once, it then loads each file, extracts the relevant intensities for masses of interest from each spectrum, and iterates through each concentration to generate graphs of internal standard & analyte correlations, measured isotope ratios, and total ion chromatograms. It then filters the data to remove points at background levels or at the detector saturation limit. Using this filtered data, the pipeline re-creates the graphs to reveal high-quality information from a low-quality raw dataset. All graphs of the R software then output to PowerPoint format. Additionally, faster 3D visualization functionality was added to the pipeline for viewing the entire data file produced by the instrument as an intensity map. With these data analysis tools ready, samples of strontium nitrate/yttrium nitrate as an analyte/internal standard combination were measured on the AccuTOF. From the filtered data, calibration curves were prepared and limits of detection (LOD) of 1 ppb for ESI-MS and 20 ppb for PSI-MS were obtained. The quality available from the calibration curves and the low limits of detection demonstrate the value of the instrument/analysis pipeline system in characterizing trace inorganics.

47 OTHER INSTRUMENTATION↗
DOE OSTI2019

Gas-Electrical Grid Coordination for Cross Infrastructure Resilience Enhancement

An electric power grid is exceedingly faced with extreme events, such as physical attacks and natural disasters. In comparison, the underground natural gas pipeline-based transportation network is considered less vulnerable to such environmental threats. We consider that the overhead power transmissions can be hardened by replacing segments of electric power system with underground natural gas pipelines as an energy transportation system to counter effect extreme events which can damage interdependent infrastructures severely. This project investigates an extension of the power system planning problem considering extreme events and identifies the feasibility of solving this problem with advanced modeling and optimization methods. The proposed planning framework is formulated as a two-stage optimization problem. It is found that the integrated planning introduces the additional cost of pipeline planning and the operation constraints into the power system planning. However, gas pipelines are generally less exposed to extreme events, which can provide a more resilient power transmission for supplying the electrical loads. It is with this potential for enhancing the power grid resilience that the proposed integrated expansion of electricity and natural gas system is considered an optimal layout for the transportation of energy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

Research on the DESI Experiment at Southern Methodist University

The SMU observational cosmology group in 2017-2019 continued its work in preparing DESI for operations slated to start in 2020. There are three major developments: a) a real-time spectroscopic data reduction and monitoring software needed for DESI Operations, b) components of spectroscopic simulation needed to support DESI science, and c) techniques and procedures to estimate spectroscopic observational systematics on BAO measurements for emission line galaxies (ELGs). We successfully developed a fast, flexible spectroscopic pipeline incorporating a comprehensive set of data quality algorithms. This included substantial development of data reduction algorithms, as well as the infrastructure to build pipelines for different data types necessary for calibration and science. Effort on data quality included supporting real-time monitoring during operations and long-term data quality needs for analysis. During this period, the SMU team contributed to development of DESI’s 1D spectroscopic simulation solutions, with a flexible pipeline that also mimicked the impact of the offline pipeline. Further work to upgrade the fast, parametric simulation, QuickCat, was also performed. We have also developed an initial methodology for the forward modeling of spectroscopic systematic uncertainties on ELG BAO measurements resulting from observing conditions. Kehoe also contributed to commissioning and science requirements documents, and served in difference service roles during the reporting period.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

NETL Plastic Pipes Project - June 2022 Progress Report

Plastic or composite pipelines have been the bane of the utility locating industry because conduits are neither conductive nor magnetic, which are the properties traditionally used to locate buried utilities. Ground penetrating radar (GPR) is an effective geophysical tool for locating plastic/composite pipeline where the resistive cover allows for adequate penetration of radar energy. However, GPR cannot be used in areas where the soil cover is conductive due to significant clay and/or salt content. This study takes a comprehensive look at near-surface geophysical methods that potentially are useful for locating buried plastic/composite pipelines, either singly or in combination with other geophysical methods. Specifically, this modeling study uses computational numerical methods to forward model the response of GPR, resistivity, seismic, and gravity gradiometry methods to plastic/composite pipelines for various scenarios including: (1) pipe diameters ranging between 2 in. to 12 in.; (2) burial depths ranging between 3 ft to 4 ft; (3) various degrees in contrast in physical properties (i.e., permittivity, elasticity, electrical resistivity, density); and (4) various experimental acquisition choices (e.g., GPR radar frequencies, seismic source frequency, electrode spacing).

47 OTHER INSTRUMENTATION↗