Engineering topics
Publications and source records attributed to Meyer, Folker.
Efficient response to the pandemic through the mobilization of the larger scientific community is challenged by the limited reusability of the available primary genomic data. Here, the Genomic Standards Consortium board highlights the essential need for contextual genomic data FAIRness, for empowering key data-driven biological questions.
Background The MG-RAST API provides search capabilities and delivers organism and function data as well as raw or annotated sequence data via the web interface and its RESTful API. For casual users, however, RESTful APIs are hard to learn and work with. Results We created the graphical MG-RAST API explorer to help researchers more easily build and export API queries; understand the data abstractions and indices available in MG-RAST; and use the results presented in-browser for exploration, development, and debugging. Conclusions The API explorer lowers the barrier to entry for occasional or first-time MG-RAST API users.
Despite some notable progress in data sharing policies and practices, restrictions are still often placed on the open and unconditional use of various genomic data after they have received official approval for release to the public domain or to public databases. These restrictions, which often conflict with the terms and conditions of the funding bodies who supported the release of those data for the benefit of the scientific community and society, are perpetuated by the lack of clear guiding rules for data usage. Existing guidelines for data released to the public domain recognize but fail to resolve tensions between the importance of free and unconditional use of these data and the “right” of the data producers to the first publication. This self-contradiction has resulted in a loophole that allows different interpretations and a continuous debate between data producers and data users on the use of public data. We argue that the publicly available data should be treated as open data, a shared resource with unrestricted use for analysis, interpretation, and publication.
Nicol et al. make discerning comments on issues related to data sharing and ethics, regulations, and imbalance of power. We broadly share their concerns, and we did indeed mention some of them briefly in our Policy Forum. Our discussion pertained to all types of data, many or most of which do not include any personal information or do not involve individuals or humans in any way.
Explore the source record for details and available documents.
© 2018 Nature Publishing Group. All rights reserved. In the version of this article initially published, the author A. Murat Eren was listed as A.M. Eren. The corresponding affiliation was given as the Knapp Center for Biomedical Discovery, rather than Department of Medicine, University of Chicago, Chicago, Illinois, USA, and Marine Biological Laboratory, Woods Hole, Massachusetts, USA. The errors have been corrected in the HTML and PDF versions of the article as of 29 November 2017. In the version of this article initially published, the following acknowledgment was omitted: A.L. was supported by the Russian Science Foundation (grant number 14-50-00069). The error has been corrected in the HTML and PDF versions of the article as of 7 December 2017.
Federation is a popular concept in building distributed cyberinfrastructures, whereby computational resources are provided by multiple organizations through a unified portal, decreasing the complexity of moving data back and forth among multiple organizations. Federation has been used in bioinformatics only to a limited extent, namely, federation of datastores, e.g. SBGrid Consortium for structural biology and Gene Expression Omnibus (GEO) for functional genomics. Here, we posit that it is critical to federate both computational resources (CPU, GPU, FPGA, etc.) and datastores to support popular bioinformatics portals, with fast-increasing data volumes and increasing processing requirements. A prime example, and one that we discuss here, is in genomics and metagenomics. It is vital that the processing of the data be done without having to transport the data across large network distances. We exemplify our design and development through our experience with metagenomics-RAST (MG-RAST), the most popular metagenomics analysis pipeline. Currently, it is hosted completely at Argonne National Laboratory. Yet, through a recently started collaborative National Institutes of Health project, we are taking steps toward federating this infrastructure. Being a widely used resource, we have to move toward federation without disrupting 50 K annual users. Here, we describe the computational tools that will be useful for federating a bioinformatics infrastructure and the open research challenges that we see in federating such infrastructures. It is hoped that our manuscript can serve to spur greater federation of bioinformatics infrastructures by showing the steps involved, and thus, allow them to scale to support larger user bases.
Here, we present two standards developed by the Genomic Standards Consortium (GSC) for reporting bacterial and archaeal genome sequences. Both are extensions of the Minimum Information about Any (x) Sequence (MIxS). The standards are the Minimum Information about a Single Amplified Genome (MISAG) and the Minimum Information about a MetagenomeAssembled Genome (MIMAG), including, but not limited to, assembly quality, and estimates of genome completeness and contamination. These standards can be used in combination with other GSC checklists, including the Minimum Information about a Genome Sequence (MIGS), Minimum Information about a Metagenomic Sequence (MIMS), and Minimum Information about a Marker Gene Sequence (MIMARKS). Community-wide adoption of MISAG and MIMAG will facilitate more robust comparative genomic analyses of bacterial and archaeal diversity.
Here, we present two standards developed by the Genomic Standards Consortium (GSC) for reporting bacterial and archaeal genome sequences. Both are extensions of the Minimum Information about Any (x) Sequence (MIxS). The standards are the Minimum Information about a Single Amplified Genome (MISAG) and the Minimum Information about a Metagenome- Assembled Genome (MIMAG), including, but not limited to, assembly quality, and estimates of genome completeness and contamination. These standards can be used in combination with other GSC checklists, including the Minimum Information about a Genome Sequence (MIGS), Minimum Information about a Metagenomic Sequence (MIMS), and Minimum Information about a Marker Gene Sequence (MIMARKS). Community-wide adoption of MISAG and MIMAG will facilitate more robust comparative genomic analyses of bacterial and archaeal diversity.
Metagenomics data analyses from independent studies can only be compared if the analysis workflows are described in a harmonised way. In this overview, we have mapped the landscape of data standards available for the description of essential steps in metagenomics: (1) material sampling, (2) material sequencing (3) data analysis and (4) data archiving & publishing. Taking examples from marine research, we summarise essential variables used to describe material sampling processes and sequencing procedures in a metagenomics experiment. These aspects of metagenomics dataset generation have been to some extent addressed by the scientific community but greater awareness and adoption is still needed. We emphasise the lack of standards relating to reporting how metagenomics datasets are analysed and how the metagenomics data analysis outputs should be archived and published. We propose best practice as a foundation for a community standard to enable reproducibility and better sharing of metagenomics datasets, leading ultimately to greater metagenomics data reuse and repurposing.
Despite the central role of soil microbial communities in global carbon (C) cycling, little is known about soil microbial community structure and even less about their metabolic pathways. Efforts to characterize soil communities often focus on identifying differences in gene content across environmental gradients, but an alternative question is what genes are similar in soils. These genes may indicate critical species or potential functions that are required in all soils. Here we identified the “core” set of C cycling sequences widely present in multiple soil metagenomes from a fertilized prairie (FP). Of 226,887 sequences associated with known enzymes involved in the synthesis, metabolism, and transport of carbohydrates, 843 were identified to be consistently prevalent across four replicate soil metagenomes. This core metagenome was functionally and taxonomically diverse, representing five enzyme classes and 99 enzyme families within the CAZy database. Though it only comprised 0.4% of all CAZy-associated genes identified in FP metagenomes, the core was found to be comprised of functions similar to those within cumulative soils. The FP CAZy-associated core sequences were present in multiple publicly available soil metagenomes and most similar to soils sharing geographic proximity. In soil ecosystems, where high diversity remains a key challenge for metagenomic investigations, these core genes represent a subset of critical functions necessary for carbohydrate metabolism, which can be targeted to evaluate important C fluxes in these and other similar soils.
Explore the source record for details and available documents.
Soil microbial communities are essential for ecosystem function, but linking community composition to biogeochemical processes is challenging because of high microbial diversity and large spatial variability of most soil characteristics. We investigated soil bacterial community structure in a switchgrass stand planted on soil with a history of grassland vegetation at high spatial resolution to determine whether biogeographic trends occurred at the centimeter scale. Moreover, we tested whether such heterogeneity, if present, influenced community structure within or among ecosystems. Pronounced heterogeneity was observed at centimeter scales, with abrupt changes in relative abundance of phyla from sample to sample. At the ecosystem scale (> 10 m), however, bacterial community composition and structure were subtly, but significantly, altered by fertilization, with higher alpha diversity in fertilized plots. Moreover, by comparing these data with data from 1772 soils from the Earth Microbiome Project, it was found that 20% diverse globally sourced soil samples, while grassland soils shared approximately 40% of their operational taxonomic units with the current study. By spanning several orders of magnitude, the analysis suggested that extreme patchiness characterized community structure at smaller scales but that coherent patterns emerged at larger length scales.
MG-RAST (http://metagenomics.anl.gov) is an opensubmission data portal for processing, analyzing, sharing and disseminating metagenomic datasets. Currently, the system hosts over 200 000 datasets and is continuously updated. The volume of submissions has increased 4-fold over the past 24 months, now averaging 4 terabasepairs per month. In addition to several new features, we report changes to the analysis workflow and the technologies used to scale the pipeline up to the required throughput levels. Lastly, to show possible uses for the data from MG-RAST, we present several examples integrating data and analyses from MG-RAST into popular third-party analysis tools or sequence alignment tools.
To improve our understanding of the stability of mammalian intestinal communities, we characterized the responses of both bacterial and viral communities in murine fecal samples to dietary changes between high- and low-fat (LF) diets. Targeted DNA extraction methods for bacteria, virus-like particles and induced prophages were used to generate bacterial and viral metagenomes as well as 16S ribosomal RNA amplicons. Gut microbiome communities from two cohorts of C57BL/6 mice were characterized in a 6-week diet perturbation study in response to high fiber, LF and high-refined sugar, milkfat (MF) diets. The resulting metagenomes from induced bacterial prophages and extracellular viruses showed significant overlap, supporting a largely temperate viral lifestyle within these gut microbiomes. The resistance of baseline communities to dietary disturbances was evaluated, and we observed contrasting responses of baseline LF and MF bacterial and viral communities. In contrast to baseline LF viral communities and bacterial communities in both diet treatments, baseline MF viral communities were sensitive to dietary disturbances as reflected in their non-recovery during the washout period. Finally, the contrasting responses of bacterial and viral communities suggest that these communities can respond to perturbations independently of each other and highlight the potentially unique role of viruses in gut health.
Although plasmids and other episomes are recognized as key players in horizontal gene transfer among microbes, their diversity and dynamics among ecologically structured host populations in the wild remain poorly understood. Here, we show that natural populations of marine Vibrionaceae bacteria host large numbers of families of episomes, consisting of plasmids and a surprisingly high fraction of plasmid-like temperate phages. Episomes are unevenly distributed among host populations, and contrary to the notion that high-density communities in biofilms act as hot spots of gene transfer, we identified a strong bias for episomes to occur in free-living as opposed to particle-attached cells. Mapping of episomal families onto host phylogeny shows that, with the exception of all phage and a few plasmid families, most are of recent evolutionary origin and appear to have spread rapidly by horizontal transfer. Such high eco-evolutionary turnover is particularly surprising for plasmids that are, based on previously suggested categorization, putatively nontransmissible, indicating that this type of plasmid is indeed frequently transferred by currently unknown mechanisms. Finally, analysis of recent gene transfer among plasmids reveals a network of extensive exchange connecting nearly all episomes. Genes functioning in plasmid transfer and maintenance are frequently exchanged, suggesting that plasmids can be rapidly transformed from one category to another. The broad distribution of episomes among distantly related hosts and the observed promiscuous recombination patterns show how episomes can offer their hosts rapid assembly and dissemination of novel functions.
Metagenomic sequencing has produced significant amounts of data in recent years. For example, as of summer 2013, MGRAST has been used to annotate over 110,000 data sets totaling over 43 Terabases. With metagenomic sequencing finding even wider adoption in the scientific community, the existing web-based analysis tools and infrastructure in MG-RAST provide limited capability for data retrieval and analysis, such as comparative analysis between multiple data sets. Moreover, although the system provides many analysis tools, it is not comprehensive. By opening MG-RAST up via a web services API (application programmers interface) we have greatly expanded access to MG-RAST data, as well as provided a mechanism for the use of third-party analysis tools with MG-RAST data. This RESTful API makes all data and data objects created by the MG-RAST pipeline accessible as JSON objects. As part of the DOE Systems Biology Knowledgebase project (KBase, http:// kbase.us) we have implemented a web services API for MG-RAST. This API complements the existing MG-RAST web interface and constitutes the basis of KBase’s microbial community capabilities. In addition, the API exposes a comprehensive collection of data to programmers. This API, which uses a RESTful (Representational State Transfer) implementation, is compatible with most programming environments and should be easy to use for end users and third parties. It provides comprehensive access to sequence data, quality control results, annotations, and many other data types. Where feasible, we have used standards to expose data and metadata. Code examples are provided in a number of languages both to show the versatility of the API and to provide a starting point for users. We present an API that exposes the data in MG-RAST for consumption by our users, greatly enhancing the utility of the MG-RAST service.
Background: As the impact and prevalence of large-scale metagenomic surveys grow, so does the acute need for more complete and standards compliant metadata. Metadata (data describing data) provides an essential complement to experimental data, helping to answer questions about its source, mode of collection, and reliability. Metadata collection and interpretation have become vital to the genomics and metagenomics communities, but considerable challenges remain, including exchange, curation, and distribution. Currently, tools are available for capturing basic field metadata during sampling, and for storing, updating and viewing it. These tools are not specifically designed for metagenomic surveys; in particular, they lack the appropriate metadata collection templates, a centralized storage repository, and a unique ID linking system that can be used to easily port complete and compatible metagenomic metadata into widely used assembly and sequence analysis tools. Results: Metazen was developed as a comprehensive framework designed to enable metadata capture for metagenomic sequencing projects. Specifically, Metazen provides a rapid, easy-to-use portal to encourage early deposition of project and sample metadata. Conclusion: Metazen is an interactive tool that aids users in recording their metadata in a complete and valid format. A defined set of mandatory fields captures vital information, while the option to add fields provides flexibility.