Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “PIPELINES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Improved assemblies using a source-agnostic pipeline for MetaGenomic Assembly by Merging (MeGAMerge) of contigs

Assembly of metagenomic samples is a very complex process, with algorithms designed to address sequencing platform-specific issues, (read length, data volume, and/or community complexity), while also faced with genomes that differ greatly in nucleotide compositional biases and in abundance. To address these issues, we have developed a post-assembly process: MetaGenomic Assembly by Merging (MeGAMerge). We compare this process to the performance of several assemblers, using both real, and in-silico generated samples of different community composition and complexity. MeGAMerge consistently outperforms individual assembly methods, producing larger contigs with an increased number of predicted genes, without replication of data. MeGAMerge contigs are supported by read mapping and contig alignment data, when using synthetically-derived and real metagenomic data, as well as by gene prediction analyses and similarity searches. Ultimately, MeGAMerge is a flexible method that generates improved metagenome assemblies, with the ability to accommodate upcoming sequencing platforms, as well as present and future assembly algorithms.

59 BASIC BIOLOGICAL SCIENCES↗

Simple and Scalable Streaming: The GRETA Data Pipeline

The Gamma Ray Energy Tracking Array (GRETA) is a state of the art gamma-ray spectrometer being built at Lawrence Berkeley National Laboratory to be first sited at the Facility for Rare Isotope Beams (FRIB) at Michigan State University. A key design requirement for the spectrometer is to perform gamma-ray tracking in near real time. To meet this requirement we have used an inline, streaming approach to signal processing in the GRETA data acquisition system, using a GPU-equipped computing cluster. The data stream will reach 480 thousand events per second at an aggregate data rate of 4 gigabytes per second at full design capacity. We have been able to simplify the architecture of the streaming system greatly by interfacing the FPGA-based detector electronics with the computing cluster using standard network technology. A set of highperformance software components to implement queuing, flow control, event processing and event building have been developed, all in a streaming environment which matches detector performance. Prototypes of all high-performance components have been completed and meet design specifications.

Cromaz, Mario↗

Electrochemical impedance spectroscopy and finite element analysis modeling of a 4-electrode humidity sensor for natural gas transportation pipelines

Reliable corrosion monitoring of natural gas transmission lines is a major tool providing a foundation for safe management of natural gas infrastructures. Through the development of membrane-based electrochemical sensors which are able to function in low-conductivity gas environments, corrosion monitoring practices can be further strengthened by real-time monitoring of key risk factors such as relative humidity and corrosion rates of corrodible structures. In this work, we demonstrate and validate how a 4-electrode conductivity sensor can provide a means to monitor relative humidity in gases via electrochemical impedance spectroscopy through finite element analysis (FEA). For a relative humidity range of 5%–55%, the impedance response varied from 1 kΩ to 66 kΩ, showing a high sensitivity for gas humidity. To confirm that the measured impedance values reliably interpreted relative humidity, it was found that precise estimation of the sensor’s cell constant was needed. FEA was used to assess how the cell constant depended on the electrode geometry, membrane geometry, and electrode placement within the sensor. Through this approach, assumptions about the characteristic area and length were validated using electrolyte equipotential and current density vector mapping. This reduced possible cell constant uncertainties by 70%. Here, with a cell constant of 14.84 cm –1 , obtained via FEA, membrane conductivity values were in good agreement with published data.

47 OTHER INSTRUMENTATION↗

THE DATA REDUCTION PIPELINE FOR THE APACHE POINT OBSERVATORY GALACTIC EVOLUTION EXPERIMENT

The Apache Point Observatory Galactic Evolution Experiment (APOGEE), part of the Sloan Digital Sky Survey III, explores the stellar populations of the Milky Way using the Sloan 2.5-m telescope linked to a high resolution (R ∼ 22,500), near-infrared (1.51–1.70 μm) spectrograph with 300 optical fibers. For over 150,000 predominantly red giant branch stars that APOGEE targeted across the Galactic bulge, disks and halo, the collected high signal-to-noise ratio (>100 per half-resolution element) spectra provide accurate (∼0.1 km s{sup −1}) RVs, stellar atmospheric parameters, and precise (≲0.1 dex) chemical abundances for about 15 chemical species. Here we describe the basic APOGEE data reduction software that reduces multiple 3D raw data cubes into calibrated, well-sampled, combined 1D spectra, as implemented for the SDSS-III/APOGEE data releases (DR10, DR11 and DR12). The processing of the near-IR spectral data of APOGEE presents some challenges for reduction, including automated sky subtraction and telluric correction over a 3°-diameter field and the combination of spectrally dithered spectra. We also discuss areas for future improvement.

79 ASTRONOMY AND ASTROPHYSICS↗

NEW TECHNIQUES FOR HIGH-CONTRAST IMAGING WITH ADI: THE ACORNS-ADI SEEDS DATA REDUCTION PIPELINE

We describe Algorithms for Calibration, Optimized Registration, and Nulling the Star in Angular Differential Imaging (ACORNS-ADI), a new, parallelized software package to reduce high-contrast imaging data, and its application to data from the SEEDS survey. We implement several new algorithms, including a method to register saturated images, a trimmed mean for combining an image sequence that reduces noise by up to {approx}20%, and a robust and computationally fast method to compute the sensitivity of a high-contrast observation everywhere on the field of view without introducing artificial sources. We also include a description of image processing steps to remove electronic artifacts specific to Hawaii2-RG detectors like the one used for SEEDS, and a detailed analysis of the Locally Optimized Combination of Images (LOCI) algorithm commonly used to reduce high-contrast imaging data. ACORNS-ADI is written in python. It is efficient and open-source, and includes several optional features which may improve performance on data from other instruments. ACORNS-ADI requires minimal modification to reduce data from instruments other than HiCIAO. It is freely available for download at www.github.com/t-brandt/acorns-adi under a Berkeley Software Distribution (BSD) license.

79 ASTRONOMY AND ASTROPHYSICS↗

A MAGELLAN-IMACS-IFU SEARCH FOR DYNAMICAL DRIVERS OF NUCLEAR ACTIVITY. I. REDUCTION PIPELINE AND GALAXY CATALOG

Using the IMACS integral-field unit (IFU) on the 6.5 m Magellan telescope, we have designed the first statistically significant investigation of the two-dimensional distribution and kinematics of ionized gas and stars in the central kiloparsec regions of a well-matched sample of Seyfert and inactive control galaxies selected from the Sloan Digital Sky Survey. The goals of the project are to use the fine spatial sampling (0.2 arcsec pixel{sup -1}) and large wavelength coverage (4000-7000 A) of the IMACS-IFU to search for dynamical triggers of nuclear activity in the central region where active galactic nucleus (AGN) activity and dynamical timescales become comparable, to identify and assess the impact of AGN-driven outflows on the host galaxy and to provide a definitive sample of local galaxy kinematics for comparison with future three-dimensional kinematic studies of high-redshift systems. In this paper, we provide the first detailed description of the procedure to reduce and calibrate data from the IMACS-IFU in 'long mode' to obtain two-dimensional maps of the distribution and kinematics of ionized gas and stars. The sample selection criteria are presented, the observing strategy is described, and resulting maps of the sample galaxies are presented along with a description of the observed properties of each galaxy and the overall observed properties of the sample.

79 ASTRONOMY AND ASTROPHYSICS↗

A NOVEL, FULLY AUTOMATED PIPELINE FOR PERIOD ESTIMATION IN THE EROS 2 DATA SET

We present a new method to discriminate periodic from nonperiodic irregularly sampled light curves. We introduce a periodic kernel and maximize a similarity measure derived from information theory to estimate the periods and a discriminator factor. We tested the method on a data set containing 100,000 synthetic periodic and nonperiodic light curves with various periods, amplitudes, and shapes generated using a multivariate generative model. We correctly identified periodic and nonperiodic light curves with a completeness of ∼90% and a precision of ∼95%, for light curves with a signal-to-noise ratio (S/N) larger than 0.5. We characterize the efficiency and reliability of the model using these synthetic light curves and apply the method on the EROS-2 data set. A crucial consideration is the speed at which the method can be executed. Using a hierarchical search and some simplification on the parameter search, we were able to analyze 32.8 million light curves in ∼18 hr on a cluster of GPGPUs. Using the sensitivity analysis on the synthetic data set, we infer that 0.42% of the sources in the LMC and 0.61% of the sources in the SMC show periodic behavior. The training set, catalogs, and source code are all available at http://timemachine.iic.harvard.edu.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A path to intelligent watersheds: coordinating the data to decision pipeline

Operations of multi-reservoir systems are challenged in-part by the interplay of complex physical processes functioning within the watershed. The employment of intelligent systems can be of aid by linking environmental sensing, information technology, data analytics, simulation and decision support to achieve a data-to-decision flow of information. A further challenge is that watershed resources are managed for multiple purposes requiring some level of coordination among numerous resource managers, asset operators and users. System intelligence in this context relies on shared community platforms (data portals, community models), and coordinated communication between decision makers. Opportunities to enrich watershed intelligence has been the subject of a roadmapping exercise for the Department of Energy’s Water Power Technologies Office which has relied on broad stakeholder engagement. Initial phases of engagement involved personal interviews and a series of virtual group meetings, which focused on identifying opportunities to improve the intelligence of the physical infrastructure within our watersheds—examples of feedback include improved sensing of snowpack and runoff, data standards for facilitated data sharing, and better forecasting tools. The latter phase of engagement involved the conduct of a case study in the Upper Colorado River basin where key stakeholders were interviewed to map how their decisions are informed by intelligence from other basin stakeholders. Our presentation will highlight the interdisciplinary flow of information in complex watershed systems and identify physical and institutional opportunities toward the strategic operation of water infrastructure.

Colorado River↗

Snekmer: a scalable pipeline for protein sequence fingerprinting based on amino acid recoding

Abstract Motivation The vast expansion of sequence data generated from single organisms and microbiomes has precipitated the need for faster and more sensitive methods to assess evolutionary and functional relationships between proteins. Representing proteins as sets of short peptide sequences (kmers) has been used for rapid, accurate classification of proteins into functional categories; however, this approach employs an exact-match methodology and thus may be limited in terms of sensitivity and coverage. We have previously used similarity groupings, based on the chemical properties of amino acids, to form reduced character sets and recode proteins. This amino acid recoding (AAR) approach simplifies the construction of protein representations in the form of kmer vectors, which can link sequences with distant sequence similarity and provide accurate classification of problematic protein families. Results Here, we describe Snekmer, a software tool for recoding proteins into AAR kmer vectors and performing either (i) construction of supervised classification models trained on input protein families or (ii) clustering for de novo determination of protein families. We provide examples of the operation of the tool against a set of nitrogen cycling families originally collected using both standard hidden Markov models and a larger set of proteins from Uniprot and demonstrate that our method accurately differentiates these sequences in both operation modes. Availability and implementation Snekmer is written in Python using Snakemake. Code and data used in this article, along with tutorial notebooks, are available at http://github.com/PNNL-CompBio/Snekmer under an open-source BSD-3 license. Supplementary information Supplementary data are available at Bioinformatics Advances online.

59 BASIC BIOLOGICAL SCIENCES↗

An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Wang, Tianle [Brookhaven National Laboratory (BNL)↗