Early experience in ultra-scale E3SM land model development on SUMMIT
Explore the source record for details and available documents.
Engineering topics
Publications and source records attributed to Yuan, Fengming.
Explore the source record for details and available documents.
High-performance computing (HPC) plays an important role during the development of Earth system models. This chapter reviews HPC efforts related to Earth system models, including community Earth system models and energy exascale Earth system models. Specifically, this chapter evaluates computational and software design issues, analyzes several current HPC-related model developments, and provides an outlook for some promising areas within Earth system modeling in the era of exascale computing.
Spatial heterogeneities in soil hydrology have been confirmed as a key control on CO 2 and CH 4 fluxes in the Arctic tundra ecosystem. In this study, we applied a mechanistic ecosystem model, CLM-Microbe, to examine the microtopographic impacts on CO 2 and CH 4 fluxes across seven landscape types in Utqiaġvik, Alaska: trough, low-centered polygon (LCP) center, LCP transition, LCP rim, high-centered polygon (HCP) center, HCP transition, and HCP rim. We first validated the CLM-Microbe model against static-chamber measured CO 2 and CH 4 fluxes in 2013 for three landscape types: trough, LCP center, and LCP rim. Model application showed that low-elevation and thus wetter landscape types (i.e., trough, transitions, and LCP center) had larger CH 4 emissions rates with greater seasonal variations than high-elevation and drier landscape types (rims and HCP center). Sensitivity analysis indicated that substrate availability for methanogenesis (acetate, CO 2 + H 2 ) is the most important factor determining CH 4 emission, and vegetation physiological properties largely affect the net ecosystem carbon exchange and ecosystem respiration in Arctic tundra ecosystems. Modeled CH 4 emissions for different microtopographic features were upscaled to the eddy covariance (EC) domain with an area-weighted approach before validation against EC-measured CH 4 fluxes. The model underestimated the EC-measured CH 4 flux by 20% and 25% at daily and hourly time steps, suggesting the importance of the time step in reporting CH 4 flux. The strong microtopographic impacts on CO 2 and CH 4 fluxes call for a model-data integration framework for better understanding and predicting carbon flux in the highly heterogeneous Arctic landscape.
Large-scale scientific code plays an important role in scientific researches. In order to facilitate module and element evaluation in scientific applications, we introduce a unit testing framework and describe the demand for module-based experiment customization. We then develop a parallel version of the unit testing framework to handle long-term simulations with a large amount of data. Specifically, we apply message passing based parallelization and I/O behavior optimization to improve the performance of the unit testing framework and use profiling result to guide the parallel process implementation. Finally, we present a case study on litter decomposition experiment using a standalone module from a large-scale Earth System Model. This case study is also a good demonstration on the scalability, portability, and high-efficiency of the framework.
Based on previous work on in-situ data transfer infrastructure and compiler-based software analysis, we have designed a virtual observation system for real time computer simulations. This paper presents an event detection framework for a virtual observation system. By using signal processing and detection approaches to the memory-based data streams, this framework can be reconfigured to capture high-frequency events and low-frequency events. These approaches used in the framework can dramatically reduce the data transfer needed for in-situ data analysis (between distributed computing nodes or between the CPU/GPU nodes). In the paper, we also use a terrestrial ecosystem system simulation within the Earth System Model to demonstrate the practical values of this effort.
Microtopographic features, such as polygonal ground, are characteristic sources of landscape heterogeneity in the Alaskan Arctic coastal plain. In this study, we analyze the effects of snow redistribution (SR) and lateral subsurface processes on hydrologic and thermal states at a polygonal tundra site near Barrow, Alaska. We extended the land model integrated in the E3SM to redistribute incoming snow by accounting for microtopography and incorporated subsurface lateral transport of water and energy (ELM-3D v1.0). Multiple 10-year-long simulations were performed for a transect across a polygonal tundra landscape at the Barrow Environmental Observatory in Alaska to isolate the impact of SR and subsurface process representation. When SR was included, model predictions better agreed (higher R 2 , lower bias and RMSE) with observed differences in snow depth between polygonal rims and centers. The model was also able to accurately reproduce observed soil temperature vertical profiles in the polygon rims and centers (overall bias, RMSE, and R 2 of 0.59°C, 1.82°C, and 0.99, respectively). The spatial heterogeneity of snow depth during the winter due to SR generated surface soil temperature heterogeneity that propagated in depth and time and led to ~ 10 cm shallower and ~ 5 cm deeper maximum annual thaw depths under the polygon rims and centers, respectively. Additionally, SR led to spatial heterogeneity in surface energy fluxes and soil moisture during the summer. Excluding lateral subsurface hydrologic and thermal processes led to small effects on mean states but an overestimation of spatial variability in soil moisture and soil temperature as subsurface liquid pressure and thermal gradients were artificially prevented from spatially dissipating over time. The effect of lateral subsurface processes on maximum thaw depths was modest, with mean absolute differences of ~ 3 cm. Our integration of three-dimensional subsurface hydrologic and thermal subsurface dynamics in the E3SM land model will facilitate a wide range of analyses heretofore impossible in an ESM context.
This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.
Explore the source record for details and available documents.
Investigating and evaluating physical-chemical-biological processes within an Earth system model (EMS) can be very challenging due to the complexity of both model design and software implementation. A virtual observation system (VOS) is presented to enable interactive observation of these processes during system simulation. Based on advance computing technologies, such as compiler-based software analysis, automatic code instrumentation, and high-performance data transport, the VOS provides run-time observation capability, in-situ data analytics for Earth system model simulation, model behavior adjustment opportunities through simulation steering. A VOS for a terrestrial land model simulation within the Accelerated Climate Modeling for Energy model is also presented to demonstrate the implementation details and system innovations.
Explore the source record for details and available documents.
Abstract. We explore coupling to a configurable subsurface reactive transport code as a flexible and extensible approach to biogeochemistry in land surface models. A reaction network with the Community Land Model carbon–nitrogen (CLM-CN) decomposition, nitrification, denitrification, and plant uptake is used as an example. We implement the reactions in the open-source PFLOTRAN (massively parallel subsurface flow and reactive transport) code and couple it with the CLM. To make the rate formulae designed for use in explicit time stepping in CLMs compatible with the implicit time stepping used in PFLOTRAN, the Monod substrate rate-limiting function with a residual concentration is used to represent the limitation of nitrogen availability on plant uptake and immobilization. We demonstrate that CLM–PFLOTRAN predictions (without invoking PFLOTRAN transport) are consistent with CLM4.5 for Arctic, temperate, and tropical sites.Switching from explicit to implicit method increases rigor but introduces numerical challenges. Care needs to be taken to use scaling, clipping, or log transformation to avoid negative concentrations during the Newton iterations. With a tight relative update tolerance (STOL) to avoid false convergence, an accurate solution can be achieved with about 50 % more computing time than CLM in point mode site simulations using either the scaling or clipping methods. The log transformation method takes 60–100 % more computing time than CLM. The computing time increases slightly for clipping and scaling; it increases substantially for log transformation for half saturation decrease from 10-3 to 10-9 mol m-3, which normally results in decreasing nitrogen concentrations. The frequent occurrence of very low concentrations (e.g. below nanomolar) can increase the computing time for clipping or scaling by about 20 %, double for log transformation. Overall, the log transformation method is accurate and robust, and the clipping and scaling methods are efficient. When the reaction network is highly nonlinear or the half saturation or residual concentration is very low, the allowable time-step cuts may need to be increased for robustness for the log transformation method, or STOL may need to be tightened for the clipping and scaling methods to avoid false convergence.As some biogeochemical processes (e.g., methane and nitrous oxide reactions) involve very low half saturation and thresholds, this work provides insights for addressing nonphysical negativity issues and facilitates the representation of a mechanistic biogeochemical description in Earth system models to reduce climate prediction uncertainty.
Empirical and dynamic biogeochemical modelling are complementary approaches for determining the critical load (CL) of atmospheric nitrogen (N) or other constituent deposition that an ecosystem can tolerate without causing ecological harm. The greatest benefits are obtained when these approaches are used in combination. Confounding environmental factors can complicate the determination of empirical CLs across depositional gradients, while the experimental application of N amendments for estimating the CL does not realistically mimic the effects of chronic atmospheric N deposition. Biogeochemical and vegetation simulation models can provide CL estimates and valuable ecosystem response information, allowing for past and future scenario testing with various combinations of environmental factors, pollutants, pollutant control options, land management, and ecosystem response parameters. Even so, models are fundamentally gross simplifications of the real ecosystems they attempt to simulate. Empirical approaches are vital as a check on simulations and CL estimates, to parameterize models, and to elucidate mechanisms and responses under real world conditions. In this chapter, we provide examples of empirical and modelled N CL approaches in ecosystems from three regions of the United States: mixed conifer forest, desert scrub and pinyon- juniper woodland in California; alpine catchments in the Rocky Mountains; and lakes in the Adirondack region of New York state.
Permafrost thaw and the subsequent mobilization of carbon stored in previously frozen soil organic matter (SOM) would be a strong positive feedback to climate1. As the northern permafrost region experiences double the rate of warming as the rest of the Earth2, the vast amount of carbon in permafrost soils3 is vulnerable to thaw, decomposition and release as atmospheric greenhouse gases (GHG). Here, we employ a process-based model simulation experiment to assess the net effect of this so-called permafrost carbon feedback (PCF) in recent decades. Results show a wide-spread increase in the depth to permafrost between 1990 and 2006, with simulated active layer thickness (ALT) capturing the mean and spatial variability of the observational data. Analysis of the simulation experiment provides an estimate of a 2.8 mm/yr increase in permafrost depth, which translates to 281 TgC/yr thawed from previously frozen SOM. Overall, we estimate a net GHG forcing of 534 MtCO2eq/yr directly tied to ALT dynamics, while accounting for CO2 (562 MtCO2eq/yr) and CH4 (52 MtCO2eq/yr) release as well as CO2 uptake by vegetation (-80 MtCO2eq/yr). This net forcing represents a significant factor in the estimated 640 MtCO2eq/yr pan-arctic GHG source4, and an additional 6.9% contribution on top of the combined 7792 MtCO2eq/yr fossil fuel emissions from the eight Arctic nations over this time period5.
Microbial assimilation of soil organic carbon is one of the fundamental processes of global carbon cycling and it determines the magnitude of microbial biomass in soils. Mechanistic understanding of microbial assimilation of soil organic carbon and its controls is important for to improve Earth system models ability to simulate carbon-climate feedbacks. Although microbial assimilation of soil organic carbon is broadly considered to be an important parameter, it really comprises two separate physiological processes: one-time assimilation efficiency and time-dependent microbial maintenance energy. Representing of these two mechanisms is crucial to more accurately simulate carbon cycling in soils. In this study, a simple modeling framework was developed to evaluate the substrate and environmental controls on microbial assimilation of soil organic carbon using a new term: microbial annual active period (the length of microbes remaining active in one year). Substrate quality has a positive effect on microbial assimilation of soil organic carbon: higher substrate quality (lower C:N ratio) leads to higher ratio of microbial carbon to soil organic carbon and vice versa. Increases in microbial annual active period from zero stimulate microbial assimilation of soil organic carbon; however, when microbial annual active period is longer than an optimal threshold, increasing this period decreases microbial biomass. The simulated ratios of soil microbial biomass to soil organic carbon are reasonably consistent with a recently compiled global dataset at the biome-level. The modeling framework of microbial assimilation of soil organic carbon and its controls developed in this study offers an applicable ways to incorporate microbial contributions to the carbon cycling into Earth system models for simulating carbon-climate feedbacks and to explain global patterns of microbial biomass.
Permafrost is tightly coupled to the organic layer, an interaction that mediates permafrost degradation in response to regional warming. We analyzed changes in permafrost occurrence (PF) and organic layer thickness (OLT) in more than 3000 soil pedons across a mean annual temperature (MAT) gradient. Cause and effect relationships between PF, OLT, and other topographic factors were investigated using structural equation modeling in a multi-group analysis. Groups were defined by slope, soil texture type, and shallow v. deep organic layers. Permafrost probability sharply increased by 0.32 for every 10-cm OLT increase in shallow OLT soils (OLTs) due to an insulation effect, but PF decreased in deep OLT soils (OLTd) by 0.06 for every 10-cm increase. As temperature warmed, sandy soils varied little in PF or OLT, but PF in loamy and sandy soils decreased substantially. The change in OLT was more heterogeneous across soil types in some there was no change while in others OLTs soils thinned and/or OLTd soils thickened as temperature warmed. Furthermore, the rate of thickening with warming for OLTd soils was on average almost 4 times greater than the rate of thinning for OLTs soils across all soil types. If soils follow a trajectory of warming that mimics the spatial gradients found today, then heterogeneities of permafrost degradation and organic layer thinning and thickening should be considered in the regional carbon balance.
There is a substantial amount of carbon stored in the permafrost soils of boreal forest ecosystems, where it is currently protected from decomposition. The surface organic horizons insulate the deeper soil from variations in atmospheric temperature. The removal of these insulating horizons through consumption by fire increases the vulnerability of permafrost to thaw, and the carbon stored in permafrost to decomposition. In this study we ask how warming and fire regime may influence spatial and temporal changes in active layer and carbon dynamics across a boreal forest landscape in interior Alaska. To address this question, we (1) developed and tested a predictive model of the effect of fire severity on soil organic horizons that depends on landscape-level conditions and (2) used this model to evaluate the long-term consequences of warming and changes in fire regime on active layer and soil carbon dynamics of black spruce forests across interior Alaska. The predictive model of fire severity, designed from the analysis of field observations, reproduces the effect of local topography (landform category, the slope angle and aspect and flow accumulation), weather conditions (drought index, soil moisture) and fire characteristics (day of year and size of the fire) on the reduction of the organic layercaused by fire. The integration of the fire severity model into an ecosystem process-based model allowed us to document the relative importance and interactions among local topography, fire regime and climate warming on active layer and soil carbon dynamics. Lowlands were more resistant to severe fires and climate warming, showing smaller increases in active layer thickness and soil carbon loss compared to drier flat uplands and slopes. In simulations that included the effects of both warming and fire at the regional scale, fire was primarily responsible for a reduction in organic layer thickness of 0.06 m on average by 2100 that led to an increase in active layer thickness of 1.1 m on average by 2100. The combination of warming and fire led to a simulated cumulative loss of 9.6 kgC m 2 on average by 2100. Our analysis suggests that ecosystem carbon storage in boreal forests in interior Alaska is particularly vulnerable, primarily due to the combustion of organic layer thickness in fire and the related increase in active layer thickness that exposes previously protected permafrost soil carbon to decomposition.
Carbon (C) dynamics of boreal forest ecosystems have substantial implications for efforts to mitigate the rise of atmospheric CO2 and may be substantially influenced by warming and changing wildfire regimes. In this study we applied a large-scale ecosystem model that included dynamics of organic soil horizons and soil organic matter characteristics of multiple pools to assess forest C stock changes of the Yukon River Basin (YRB) in Alaska, USA, and Canada from 1960 through 2006, a period characterized by substantial climate warming and increases in wildfire. The model was calibrated for major forests with data from long-term research sites and evaluated using a forest inventory database. The regional assessment indicates that forest vegetation C storage increased by 46 Tg C, but that total soil C storage did not change appreciably during this period. However, further analysis suggests that C has been continuously lost from the mineral soil horizon since warming began in the 1970s, but has increased in the amorphous organic soil horizon. Based on a factorial experiment, soil C stocks would have increased by 158 Tg C if the YRB had not undergone warming and changes in fire regime. The analysis also identified that warming and changes in fire regime were approximately equivalent in their effects on soil C storage, and interactions between these two suggests that the loss of organic horizon thickness associated with increases in wildfire made deeper soil C stocks more vulnerable to loss via decomposition. Subbasin analyses indicate that C stock changes were primarily sensitive to the fraction of burned forest area within each subbasin and that boreal forest ecosystems in the YRB are currently transitioning from being sinks to sources at ;0.7% annual area burned. We conclude that it is important for international mitigation efforts focused on controlling atmospheric CO2 to consider how climate warming and changes in fire regime may concurrently affect the CO2 sink strength of boreal forests. It is also important for large-scale biogeochemical and earth system models to include organic soil dynamics in applications to assess regional C dynamics of boreal forests responding to warming and changes in fire regime.