Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data processing in systems biology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Enabling Model Organism and Commercial Astronaut Data Access Through the NASA Open Science Data Repository

NASA’s Open Science Data Repository (OSDR) brings together omics data from NASA’s GeneLab project and non-omics data, including physiological, phenotypic, imaging, and behavioral data from NASA’s Ames Life Sciences Data Archive (ALSDA) collected from decades of space biology research, providing open and FAIR (findable, accessible, interoperable, and reusable) access of these precious data to scientists world-wide. This rich source of meticulously curated metadata and data from spaceflight and analog studies has been mined by the scientific community resulting in dozens of high impact scientific publications that reveals a complex network of molecular and physiological effects of spaceflight across living systems, from microbes to plants, to mammals. Understanding how these effects translate to the human condition is critical as we move deeper into the era of commercial space travel. However, the integration of data, specifically omics data, from astronauts is particularly challenging due to their sensitive nature. OSDR has risen to this challenge by developing a mechanism to control access to identifiable levels of omics data, such as raw sequence data, while enabling public access to processed, unidentifiable, data and associated metadata that will allow the scientific community to interrogate human astronaut data alongside data from model organisms to begin answering these critical questions. The 2021 SpaceX Inspiration4 (I4) mission collected a comprehensive atlas of biological measurements from four civilian astronauts, providing a wealth of data to characterize the effects of spaceflight on the human body. These data include both non-omics and omics assays such as direct RNA sequencing (RNA-seq), single nuclei ATAC-seq and RNA-seq, metagenomics, proteomics, and comprehensive metabolic and cytokine panels, all of which have been integrated into the OSDR system across no less than 9 studies. Each study has been carefully curated using community-backed OSDR standards for sample and assay level metadata ensuring these data are findable and accessible. In addition to hosting both raw and processed data from the principal investigator team for each assay type, the GeneLab team plans to re-process the I4 omics data using GeneLab’s standard processing pipelines. The GeneLab processed data outputs will allow for comparisons across studies on OSDR and enable visualization of these data through the OSDR data visualization platform thereby enabling data reusability and interoperability. Here we describe the robust privacy and security protocols implemented by OSDR to safeguard sensitive health data from astronauts while facilitating metadata and processed data sharing for research purposes. We further provide a road map for navigating the vast amount of data provided for each I4 study on the OSDR, including experimental design, associated experiments, payloads, and missions, data generation and analysis protocols, and associated scientific articles. Additionally, we illustrate how to interrogate the standardized metadata provided in the sample and assay tables as well as instructions for how to download and access the data. The I4 datasets described here re present the first ever comprehensive collection of commercial astronaut data.

Amanda M Saravia-Butler↗

GeneLab

GeneLab collects and enables analysis of spaceflight and ground-based spaceflight simulation genomic data, RNA and protein expression, and metabolic profiles. It interfaces with other existing databases containing spaceflight omic data. The 2011 National Research Council (NRC) Decadal Survey on NASA Life and Physical Sciences called for increased opportunities for multi-investigator spaceflight opportunities and greater use of genomic approaches to meet the needs of NASA researchers. To address these recommendations of the NRC Decadal Survey, the Space Life and Physical Sciences Research and Applications Division of NASA's Human Exploration and Operations Mission Directorate has initiated a transition to an Open Science architecture to increase research opportunities, and has developed the GeneLab Platform based on highly leveraged and integrated bioinformatics analytics. GeneLab is an interactive, open-access resource where scientists can upload, download, store, search, share, transfer, and analyze omics data from spaceflight and corresponding analogue experiments. Users can explore GeneLab datasets in the Data Repository, analyze data using the Analysis Platform, visualize high-order data and create collaborative projects using the Collaborative Workspace. Our primary goal is to maximize the utilization of the valuable biological research conducted aboard the International Space Station (ISS) by collecting genomic, transcriptomic, proteomic, and metabolomics data known as “omics”. By providing a portal linking processed data to flight parameters, GeneLab enables exploration of the molecular network responses of terrestrial biology to the space environment. This allows researchers to understand the complex responses of biological systems to the space environment. This technology development activity was transferred from the Human Exploration and Operations Mission Directorate to the Science Mission Directorate Division of Biological and Physical Sciences (BPS) in October 2020.

GeneLab↗

Automatic variable selection in ecological niche modeling: A case study using Cassin’s Sparrow (Peucaea cassinii)

MERRA/Max provides a feature selection approach to dimensionality reduction that enables direct use of global climate model outputs in ecological niche modeling. The system accomplishes this reduction through a Monte Carlo optimization in which many independent MaxEnt runs, operating on a species occurrence file and a small set of randomly selected variables in a large collection of variables, converge on an estimate of the top contributing predictors in the larger collection. These top predictors can be viewed as potential candidates in the variable selection step of the ecological niche modeling process. MERRA/Max’s Monte Carlo algorithm operates on files stored in the underlying filesystem, making it scalable to large data sets. Its software components can run as parallel processes in a high-performance cloud computing environment to yield near real-time performance. In tests using Cassin’s Sparrow (Peucaea cassinii) as the target species, MERRA/Max selected a set of predictors from Worldclim’s Bioclim collection of 19 environmental variables that have been shown to be important determinants of the species’ bioclimatic niche. It also selected biologically and ecologically plausible predictors from a more diverse set of 86 environmental variables derived from NASA’s Modern-Era Retrospective Analysis for Research and Applications Version 2 (MERRA-2) reanalysis, an output product of the Goddard Earth Observing System Version 5 (GEOS-5) modeling system. We believe these results point to a technological approach that could expand the use global climate model outputs in ecological niche modeling, foster exploratory experimentation with otherwise difficult-to-use climate data sets, streamline the modeling process, and, eventually, enable automated bioclimatic modeling as a practical, readily accessible, low-cost, commercial cloud service.

John L. Schnase↗

Altered Expression of Two Small Secreted Proteins (ssp4 and ssp6) Affects the Degradation of a Natural Lignocellulosic Substrate by Pleurotus ostreatus

Pleurotus ostreatus is a white-rot fungus that can degrade lignin in a preferential manner using a variety of extracellular enzymes, including manganese and versatile peroxidases (encoded by the vp1-3 and mnp1-6 genes, respectively). This fungus also secretes a family of structurally related small secreted proteins (SSPs) encoded by the ssp1-6 genes. Using RNA sequencing (RNA-seq), we determined that ssp4 and ssp6 are the predominant members of this gene family that were expressed by P. ostreatus during the first three weeks of growth on wheat straw. Downregulation of ssp4 in a strain harboring an ssp RNAi construct (KDssp1) was then confirmed, which, along with an increase in ssp6 transcript levels, coincided with reduced lignin degradation and the downregulation of vp2 and mnp1. In contrast, we observed an increase in the expression of genes related to pectin and side-chain hemicellulose degradation, which was accompanied by an increase in extracellular pectin-degrading capacity. Genome-wide comparisons between the KDssp1 and the wild-type strains demonstrated that ssp silencing conferred accumulated changes in gene expression at the advanced cultivation stages in an adaptive rather than an inductive mode of transcriptional response. Based on co-expression networking, crucial gene modules were identified and linked to the ssp knockdown genotype at different cultivation times. Based on these data, as well as previous studies, we propose that P. ostreatus SSPs have potential roles in modulating the lignocellulolytic and pectinolytic systems, as well as a variety of fundamental biological processes related to fungal growth and development.

59 BASIC BIOLOGICAL SCIENCES↗

Martian stable isotopes: volatile evolution, climate change and exobiological implications

Measurements of the ratios of stable isotopes in the martian atmosphere and crust provide fundamental information about the evolution of the martian volatile and climate system. Current best estimates of the isotope ratios indicate that there has been substantial loss of gases to space and exchange of gases between the atmosphere and the crust throughout geologic time; exchange may have occurred through circulation of water in hydrothermal systems. Processes of volatile evolution and exchange will fractionate the isotopes in a manner that complicates the possible interpretation of isotopic data in terms of any fractionation that may have been caused by martian biota, and must be understood first. Key measurements are suggested that will enhance our understanding of the non-biological fractionation of the isotopes and of the evolution of the martian volatile system.

Review, Tutorial↗

Modeling of Biological Fluid Systems

This study presents a detailed methodology for modeling and validating three- dimensional fluid flow in realistic reconstructed vasculature geometry. Optical scanned data from a sectioned stenotic lumen having -65% area constriction within an arterial vessel of 9 mm diameter is processed and converted into a finite-element mesh. Mass and momentum transfer equations within the reconstructed stenotic vessel with a-flow rate of 200 ml/rnin is solved using the Galarkin Finite Element formulation. The calculated velocities at several axial locations along the length of the vessel are compared with the Particle Image Velocimetry (PIV) measurements. The computation results are in agreement with the initial steady state PIV measurements. In addition, initial results from pulsatile flow computations at different time points of a pulse cycle will be presented.

Banerjee, Rupak↗

Human Primary Airway Epithelium +/- Macrophages Response to HCoV-229E Infection Transcriptomics (ACS-DP3)

The purpose of this experiment was to evaluate the human host cellular response to wild-type Human coronavirus strain 229E (HCoV-299E) infection. Sample data was obtained for mock and infected (MOI 3) primary human airway epithelial cells with and without macrophages and grown in air-liquid interface conditions. Sample data was acquired using an Illumina Hi-Seq 4000 sequencer system and further processed for RNA sequencing (RNA-Seq) expression analysis.

59 BASIC BIOLOGICAL SCIENCES↗

Plant science decadal vision 2020–2030: Reimagining the potential of plants for a healthy and sustainable future

Abstract Plants, and the biological systems around them, are key to the future health of the planet and its inhabitants. The Plant Science Decadal Vision 2020–2030 frames our ability to perform vital and far‐reaching research in plant systems sciences, essential to how we value participants and apply emerging technologies. We outline a comprehensive vision for addressing some of our most pressing global problems through discovery, practical applications, and education. The Decadal Vision was developed by the participants at the Plant Summit 2019, a community event organized by the Plant Science Research Network. The Decadal Vision describes a holistic vision for the next decade of plant science that blends recommendations for research, people, and technology. Going beyond discoveries and applications, we, the plant science community, must implement bold, innovative changes to research cultures and training paradigms in this era of automation, virtualization, and the looming shadow of climate change. Our vision and hopes for the next decade are encapsulated in the phrase reimagining the potential of plants for a healthy and sustainable future. The Decadal Vision recognizes the vital intersection of human and scientific elements and demands an integrated implementation of strategies for research (Goals 1–4), people (Goals 5 and 6), and technology (Goals 7 and 8). This report is intended to help inspire and guide the research community, scientific societies, federal funding agencies, private philanthropies, corporations, educators, entrepreneurs, and early career researchers over the next 10 years. The research encompass experimental and computational approaches to understanding and predicting ecosystem behavior; novel production systems for food, feed, and fiber with greater crop diversity, efficiency, productivity, and resilience that improve ecosystem health; approaches to realize the potential for advances in nutrition, discovery and engineering of plant‐based medicines, and "green infrastructure." Launching the Transparent Plant will use experimental and computational approaches to break down the phytobiome into a "parts store" that supports tinkering and supports query, prediction, and rapid‐response problem solving. Equity, diversity, and inclusion are indispensable cornerstones of realizing our vision. We make recommendations around funding and systems that support customized professional development. Plant systems are frequently taken for granted therefore we make recommendations to improve plant awareness and community science programs to increase understanding of scientific research. We prioritize emerging technologies, focusing on non‐invasive imaging, sensors, and plug‐and‐play portable lab technologies, coupled with enabling computational advances. Plant systems science will benefit from data management and future advances in automation, machine learning, natural language processing, and artificial intelligence‐assisted data integration, pattern identification, and decision making. Implementation of this vision will transform plant systems science and ripple outwards through society and across the globe. Beyond deepening our biological understanding, we envision entirely new applications. We further anticipate a wave of diversification of plant systems practitioners while stimulating community engagement, underpinning increasing entrepreneurship. This surge of engagement and knowledge will help satisfy and stoke people's natural curiosity about the future, and their desire to prepare for it, as they seek fuller information about food, health, climate and ecological systems.

59 BASIC BIOLOGICAL SCIENCES↗

NASA biological and physical sciences databases: who’s the FAIRest of them all?

Conceptual models are a key part of the foundation of scientific study. Scientific data discovery and retrieval are often inaccurate and incomplete because these models are not sufficiently well-incorporated into data retrieval systems. Systems often don’t provide the necessary tools to those producing scientific data to fully and unambiguously annotate them and the result is consumers of the data cannot find them efficiently. The capability of data archives to provide these tools to link data to underlying conceptual models is one of dimensions of the recently developed “FAIR” principles (https://www.go-fair.org/fair-principles/ ), and is key to many automated processes being able to operate on these data, particularly analytics involving artificial intelligence. We used an open-source web service to measure the FAIR compliance of the three data archives operated by NASA for the biological and physical sciences: the Life Sciences Data Archive, the Physical Sciences Informatics database, and GeneLab. The service ingests references to data sets in these archives, and then executes domain-non-specific examinations of these data and metadata that test compliance to the FAIR principles. Of the 22 metrics tested, GeneLab passed 11 (50%), and PSI and LSDA each passed 7 (32%). These data were gathered using only one representative data set from each archive and we anticipate variability in results as we continue to apply these metrics to other data. A preliminary study of the failure traces for each metric suggests there is a wide range of effort and complexity in the enhancements required for each system to elevate FAIR compliance, and this is the subject of continued investigation. This information has been and will likely continue to be important information in planning these enhancements, with the goal of increased readiness of the data for automated processes.

database↗

Functional characterization of glycosyltransferases in duckweed to enable predictive biology

Glycosyltransferases (GTs) catalyze the formation of glycosidic linkages to produce almost all complex carbohydrates. This project used a multi-disciplinary, high-throughput (HTP) biochemical and computational biology approach focused on duckweed as a model energy crop, to study carbohydrate metabolic processes. To achieve this, developed and carried out out high-throughput (HTP) functional characterization of plant glycosyltransferases (GTs) role of enzymatic microenvironments be assessed through a combined proteomic and computational biology approach, and the combined data was used to populate deep-learning frameworks to predict plant GT function. Functional validation achieved through this research is being used to assign gene function and study plant processes at the systems level to efficiently link the genome sequence with gene function. Together, the combined approaches used within this study provide a foundation for how computational prediction, in combination with high-throughput functional validation, can be used to study plant processes at the systems level and translate knowledge gained to efficiently link genome sequence with gene function in a species agnostic manner.

09 BIOMASS FUELS↗

Substantial and overlooked greenhouse gas emissions from deep Arctic lake sediment - supporting data and code

This data package contains data, descriptions, and code-based analyses that were used to support conclusions drawn in “Substantial and overlooked greenhouse gas emissions from deep Arctic lake sediment”, by Freitas et al. (2025) (https://doi.org/10.1038/s41561-024-01614-y). The study evaluated greenhouse gas production along a deep sediment core (20 m) taken in 2018 from below Goldstream Lake, a field site approximately 15 km north of Fairbanks, Alaska.The file “ESSDive_NFreitas_2024_flmd.csv” includes an overview of all other csv files in this data package, namely: sediment descriptions (depth and type of sediment), sediment characterizations (bulk density, gravimetric water content, total carbon, etc.), calculated respiration and temperature sensitivity values associated with year-long incubations of the sediment core, and the R code used to process the dataset. Additional details regarding the content of these files and how the calculations were performed are described in the Methods section of this archive. The “data_dictionary_ESSDive_NFreitas_2024_dd.csv” is a data dictionary for all files included in the data package. Each row in the data dictionary represents a column name in a given file. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).Data package updates- August 2024: Title was updated, R code now includes a process for calculating potential production across the whole sediment column, the "sediment_column_production_potentials_ESSDive_NFreitas_2024.csv" is the associated data output file, and the file level metadata and data dictionaries were updated to reflect the contents of the additional csv file.- October 2024: Title was updated, and a statement was added to Step 2 of the Methods that describes that samples were collected (and exported) in a responsible manner and in accordance with relevant permits and local laws.- January 2025: Updated the associated manuscript details in the Abstract (manuscript title, publication date, DOI link) and in the Related References section (full citation).

54 ENVIRONMENTAL SCIENCES↗

The GREEN ‘omics of Nutrient Feedbacks to Soil Warming

The GREEN ‘omics of Nutrient Feedbacks in Soil project advanced the DOE Biological and Environmental Research (BER) mission by developing and applying isotope-enabled ’omics tools to understand how soil microbes regulate carbon and nutrient cycling. Guided by the Growth Rate, growth Efficiency, and stoichiometry of Essential Nutrients (GREEN ’omics) framework, the project aimed to build a predictive, systems-level understanding of microbial traits that control ecosystem biogeochemistry. In a collaboration among Northern Arizona University (lead), West Virginia University, Lawrence Livermore National Laboratory, and Pacific Northwest National Laboratory, we combined quantitative stable isotope probing (qSIP), Chip-SIP, NanoSIMS, and genome-resolved metagenomics across long-term experiments in Arctic, boreal, temperate, and tropical ecosystems. The project produced three key outcomes: 1) We showed that community-weighted temperature sensitivities of bacterial growth (Q10) can predict ecosystem-scale soil respiration responses across diverse soils. 2) We provided the first in situ evidence for density-dependent population dynamics in soil bacteria and demonstrated that nutrient additions intensify competition, concentrating carbon use into fewer taxa. 3) We improved and extended isotope-enabled ’omics methods by quantifying qSIP measurement error to guide experimental design and coupling SIP with genome-resolved metagenomics to reveal cross-kingdom interactions among bacteria, fungi, and viruses. Together, these results show that a small number of microbial traits and taxa exert disproportionate control over soil carbon and nutrient cycling, providing critical data and methods to improve representation of microbial processes in Earth system models.

54 ENVIRONMENTAL SCIENCES↗

How Discipline-Oriented DAACs Collaborate to Support Interdisciplinary Earth Science

The Earth system is made of multiple complex subsystems. While scientists tend to specialize in single subsystems or even portions of a subsystem, to fully understand the physical, chemical and biological processes occurring on the Earth, we must understand how these systems interact. Thus, scientists need to be able to expand beyond their chosen field and collaborate across disciplines. The NASA Earth Observing System Data and Information System (EOSDIS) is focusing on new capabilities to make interdisciplinary science easier. The EOSDIS DAACS are responsible for archiving and distributing NASA’s rapidly growing collection of satellite, airborne and field data. Like the scientists they serve, the DAACs are discipline oriented. This has many advantages. Scientists are able to interact with experts on data in their domain and receive data appropriate for their research. However, discipline-oriented services may be confusing to cross-disciplinary users who may not know how to get started. Fortunately, collaboration across scientific communities is a specialty of the DAACs who support interdisciplinary users with a collaborative culture and technology that empowers the open sharing of data, information, and knowledge within the scientific community and the wider public to accelerate scientific research and understanding. This presentation will discuss the capabilities of a discipline-oriented approach to support future Earth science goals including those of Open Science and cross-disciplinary collaboration.

Sara Lubkin↗

Design of multisample, multistep phase partitioning apparatus for use on Space Shuttle Spacelab, and Spacelab missions

Traditional separation techniques are inadequate for many new bioprocessing challenges. Innovative separation methods such as aqueous two phase partitioning are needed to perpetuate bioprocess commercialization. Aqueous two phase polymer partitioning systems provide a process for separating biological materials when combined with microgravity. An innovative space qualified apparatus developed for carrying out separations by partitioning in immiscible polymer systems under mirogravity conditions is described. The apparatus offers an innovative approach to low gravity bioseparations in general and phase partitioning in particular. These capabilities support NASA's interest in serving the biotechnology research community and providing quantitative data in the gravity dependent components of separation processes.

Deuser, Mark S.↗

Characterization of Soil and Rock Magnetic Properties along Multiple Hillslope Transects at Teller Road Site, Seward Peninsula, Alaska, 2018 and 2023

The magnetometer data was collected in multiple directions across the watershed hillslope at the NGEE Arctic Teller Road site at mile marker 27 (TL_MM27) on the Seward Peninsula, Alaska over multiple years in March 2018 and April 2023. The magnetic data were collected using a Geometrics Inc. G-858 gradiometer and G-857 base station in 2018 and the G-864 gradiometer and G857 base station in 2023. The data was collected (in all instances) by towing the gradiometer behind a snow machine around the watershed with the two sensors in a vertical profile with constant spacing during the continuous survey in that specific year. Magnetic total field measurements were collected by gradiometer and base station, and the data processing was performed in Geometrics MagMap2000 software. The processing steps were limited to removal of data spikes (despiking), reading dropouts, and correction/removal of bad GPS points. All offsets between sensors and GPS are stated within the data files and metadata, alongwith the processed and raw data. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).In this data submission there are two sets of raw magnetic data (.bin and .stn for 2018 and base for 2023; raw rover mag for 2023 is in .csv) inside two .zip files that identify the year the mag data was collected. The data are proprietary format to Geometrics and can be opened and processed with MagMap2000 which can be downloaded for free at Geometrics website. There are also two processed data files *.csv for each year and two metadata files *.csv.

54 ENVIRONMENTAL SCIENCES↗

Interrelationships among methods of estimating microbial biomass across multiple soil orders and biomes

Understanding the role of soil microbes is critical to ecosystem processes, and more thorough comparisons of measurement proxies for soil microbial biomass could broaden the inclusion of explicit microbial parameterization in soil carbon cycling and earth system models. We measured physical, chemical, and biological data from eight soil orders representing 11 major biomes and four climate regions. Four prominent methods to measure microbial abundance—chloroform fumigation extraction (CFE), total DNA yield, gene copy number by quantitative polymerase chain reaction (GCN), and phospholipid fatty acids (PLFA)—were compared to assess their relationships with each other and with soil characteristics. Correlations were observed when comparing methods, with CFE correlating strongly with total DNA yield, GCN, and PLFA; CFE with bacterial GCN and bacterial PLFA; and to a lesser extent, total PLFA and total DNA yield. Correlations improved with the removal of organic soils (Histosols, Gelisols). Comparisons involving extracted DNA were improved by correcting for clay content, due to DNA extraction inefficiencies in clay-rich soils. Correlations involving fungi (PLFA or GCN) were always less significant. These methods could serve as reliable, inter-relatable proxies for the estimation of total soil microbial biomass while recognizing that the proxies are less effective at parsing differences between bacteria and fungi. Here, we provide specific equations to relate measures of soil microbial biomass by these four different methods to enable microbial models to utilize a greater diversity of observed data sources in parameterizations and simulations. Caveats for the equations and their values are also discussed.

59 BASIC BIOLOGICAL SCIENCES↗

An improved data structure for AVIRIS-type imaging spectrometer measurements

Imaging spectrometers, such as the Airborne Visible/Infrared Imaging (AVIRIS), measure spectra in the region from 400 to 2500 nm at nominally 10-nrn sampling. This gives approximately 200 continuous spectral channels of data for every spatial element measured. These spectroscopic measurements are used to determine the composition and infer processes of the Earth system through spectroscopy analysis. Spectroscopic analysis is based on the physics, chemistry, and biology revealed through the interaction of energy with matter recorded in each measured spectrum

Perez, Rosa M.↗

Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward

The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.

Mehta, Kshitij↗