Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “simulation decomposition”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Time-dependent-bases with local CUR decomposition method for accelerating turbulent combustion simulations

Here, this study presents a novel reduced-order modeling framework, Time-Dependent Bases with Local CUR decomposition (TDB-L-CUR), designed to efficiently and accurately approximate the species transport equations in reacting flow simulations. The method extends the existing TDB-CUR approach for chemically reacting flows (Jung et al. Comput. Methods Appl. Mech. Engrg. 437 (2025) 117758), which leverages matrix decomposition techniques to form a global-in-space, time-dependent low-dimensional manifold. While TDB-CUR performs well in homogeneous systems, it may be less well-suited to spatially heterogeneous systems such as turbulent flames, where higher-rank approximations are typically required. The proposed TDB-L-CUR framework introduces two methodological extensions to the baseline approach. First, it applies unsupervised clustering to partition the physical domain into distinct regions, enabling spatially localized manifold construction, thereby reducing the rank required for the reduced-order representation. Second, it incorporates a computational singular perturbation (CSP)-based scheme for identifying and penalizing fast species, allowing for spatio-temporally adaptive mitigation of chemical stiffness. The proposed framework is validated on a hierarchy of test cases, including a one-dimensional premixed flame, a two-dimensional nonpremixed ignition case with vortex interaction, and a three-dimensional turbulent premixed flame. TDB-L-CUR significantly improves accuracy over TDB-CUR while further reducing computational cost. The fully on-the-fly formulation of TDB-L-CUR (i.e., requiring no offline training or prior knowledge) makes it a robust and scalable tool for reduced-order modeling of reactive flows.

Local manifold↗
DOE OSTI2019

Wood Decomposition: Understanding Processes Regulating Carbon Transfer to Soil Carbon Pools Using FACE Wood at Multiple Scales

Dead wood comprises approximately 20% of forest biomass. While it is widely recognized for its role in ecosystem functions relative to habitat and nutrient cycling, there are considerable uncertainties associated with its role in the forest carbon (C) cycle. The principal uncertainty is whether dead wood contributes to the soil C pool. With little supporting evidence, the presumption is that C in wood is respired back to the atmosphere through decomposition processes. The inability to attribute forest soil C to its source components has been a barrier to addressing this important forest carbon cycle question. The U.S. Dept. of Energy’s Free Air CO 2 Enrichment (FACE) experiment produced wood that has a distinct δ 13 C signature that can be used to trace its fate through organic matter decomposition processes. Using two hardwood species (aspen, birch) from the Rhinelander FACE site and pine from the Duke FACE site, the FACE Wood Decomposition Experiment (FWDE) was established in 2011 to provide the foundation for addressing wood decomposition and its associated processes at the continental scale. Plots were installed on nine experimental forests representing a range in bioclimatic forest conditions within the continental U.S. Each identical plot has logs of the three species incubating on the soil surface and pine logs suspended in the air, representing standing dead wood. Abiotic conditions are monitored continuously, and soil and forest conditions characterized. Logs on the sites have been measured every two years to assess decay. This project was initiated in 2016 with specific objectives to: (a) determine whether dead wood C contributes to the soil C pool, (b) assess the fungal community composition mediating the decay, (c) assess the interactions of termites with microbial decay, and (d) develop a mechanistic model to simulate wood decomposition. The principal field sampling was conducted in 2017; correspondingly, the results reported here represent six years of decomposition. Wood decomposition varied from 13 to 75 % loss across the bioclimatic gradient. Sites in the southeastern U.S. coastal plain and piedmont had fastest decomposition rates for each of the species. In contrast sites in the Rocky Mountains had the slowest decay rates, which were associated with cold temperatures. Standing deadwood decomposes at a much slower rate than logs on the soil surface; a response due to differences in wood moisture content. Wood carbon was measured in the mineral soil across each of the sites; accordingly, this is the first study to affirm that a portion of the wood C does become part of the soil C pool. Averaged across the FWDE sites, approximately 1.4% of the soil C in the upper 10 cm of the mineral soil directly beneath the log was derived from the deadwood; within 50 cm of the log, wood C accounted for an average of 2.1%. These results demonstrate that the contribution of deadwood to the soil C pool is a soil volume surrounding the log; and it’s likely a larger volume than what we sampled. We also hypothesized that there would be more wood C in the mineral soil when termites were present. However, that hypothesis wasn’t supported by measurements on the Santee Experimental Forest termite exclusion (SEF-TX) plots under FACE logs that incubated for 6 years. Across all the sites, the majority of the wood C was transferred to the atmosphere. Approximately 7,000 fungal species were sequenced. The fungal communities were different among each of the sites suggesting local colonization; interestingly, within sites the three species also exhibited differences in fungal community composition. White rot was the dominant fungal form in pine and hardwood, which yields less lignin in the wood residue. Brown rot occurred in approximately 20% of the pine and aspen logs, and it did not occur in birch logs. The occurrence of brown rot was only weakly correlated to site factors. We conducted two additional experiments to assess the effects of termites on wood decomposition and the associated interactions with the fungal community. Pine and birch logs were incubated on the Santee Experimental Forest termite exclusion plots in upland and riparian settings. There was no difference in decay between areas where termites were able colonize and plots where they had been excluded. Highly variable foraging by termites was a primary contributor the result. Correspondingly, fungal communities did not vary on logs incubated among the two treatments, but they did reflect site and species differences. The second experiment was conducted on each of the nine FWDE sites where pine and birch wood blocks were incubated for two years with and without a wrapping designed to exclude termites and other arthropods. The effects of termites on wood decomposition could be confirmed on two of the four sites known to have subterranean termites. On the two southeastern sites, termites contributed approximately 8% to mass loss in pine blocks over two years. A major factor clouding the role of termites in wood decomposition is the inability to assess the time, duration, and intensity of the foraging with the logs. Another uncertainty is related to whether a piece of wood will be foraged. A process-based wood decomposition model was developed representing biological processes mediated by fungi, bacteria, termites, beetles, as well as physical and chemical processes. The Coarse Wood Decomposition Model (CWDM) was parameterized based on a thorough review of the literature and it was validated using data from the FWDE. In the initial setup, the model considers softwoods and hardwoods groups, as opposed to individual species. Sensitivity analyses incorporating climate data from 89 locations in North America demonstrated that CWDM was stable from the tropics to the boreal zone. Model calibration and validation showed it to perform very well. The CWDM is poised for additional testing with subsequent FWDE data as well as other data sets, and it is available for incorporation into forest biogeochemical models or DOEs Earth System Modeling framework.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A learned closure method applied to phase mixing in a turbulent gradient-driven gyrokinetic system in simple geometry

We present a new method for formulating closures that learn from kinetic simulation data. We apply this method to phase mixing in a simple gyrokinetic turbulent system – temperature-gradient-driven turbulence in an unsheared slab. The closure, called the learned multi-mode (LMM) closure, is constructed by, first, extracting an optimal basis from a nonlinear kinetic simulation using singular value decomposition. Subsequent nonlinear fluid simulations are projected onto this basis and the results are used to formulate the closure. We compare the closure with other closures schemes over a broad range of the relevant two-dimensional parameter space (collisionality and gradient drive). We find that the turbulent kinetic system produces phase-mixing rates much lower than the linear expectations, which the LMM closure is capable of capturing. We also compare radial heat fluxes. A Hammett–Perkins closure, generalized to include collisional effects, is quite successful throughout the parameter space, producing ${\sim }14\,\%$ root-mean-square (r.m.s.) error. The LMM closure is also very effective: when trained at three (two) points (in a 35 point parameter grid), the LMM closure produces $8\,\%$ ( $12\,\%$ ) r.m.s. errors. The LMM procedure can be readily generalized to other closure problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

CPU and memory efficient coherent mode decomposition for the partially coherent x-ray simulations

In this work, the method of the Coherent Mode Decomposition (CMD) is applied to numerical wave propagation calculations for partially-coherent X-rays, using the Fourier optics and compatible methods. Its CPU and memory efficiency is discussed in various cases of the wavefront at the source and the beam waist. With the absence of the quadratic phase terms, the required sampling density of the electric fields is effectively reduced. The problem size is thus moderate and the method is feasible to be implemented on a single-node CPU server. In other cases, the same argument holds with proper treatments of the quadratic phase terms. Tests on CMD and the modes propagation are done for the case of the Coherent Hard X-ray beamline of the National Synchrotron Light Source II, using the Synchrotron Radiation Workshop software. We observe a few hundred or less dominant decomposed modes that resemble the electric fields converge to the wavefront intensity at a high accuracy of over 99%.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Compression of tokamak boundary plasma simulation data using a maximum volume algorithm for matrix skeleton decomposition

This report demonstrates satisfactory data compression of SOLPS-ITER simulation output ranging from 2D fields, 1D profiles, and 0D scalar variables with a novel matrix decomposition approach. The singular value decomposition (SVD) scales poorly for large matrix sizes and is unsuited to the application on high dimensional data common to fusion plasma physics simulation. In this work, we employ the columns-submatrix-rows (CUR) matrix factorization technique in order to compute a low-rank approximation up to two orders of magnitude faster than the SVD, but within a nominal L2-norm relative error of ε = 10 –2 . In addition, the CUR approach maintains the original format of the data, in its extracted columns and rows, allowing for interpretable data storage at the original resolution of the simulation. We utilize an iterative algorithm to compute the CUR decomposition of simulation output by maximizing the volume, or linearly independent information content, of a low-rank submatrix contained within the data. Experiments over $\textit{n} × \textit{n}$ randomized test matrices with embedded rank-deficient features show that this maximum volume implementation of CUR matrix approximation has reduced asymptotic computational complexity on the order of n compared to the SVD, which scales approximately as $n^3$. These results show that the CUR technique can be used to effectively select time step snapshots (columns) of over 140 SOLPS-ITER output variables and the associated discretized coordinate timeseries (rows) allowing for reconstruction of the complete simulation dynamics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Scalable algorithms for domain decomposition of Monte Carlo neutral particle transport simulations on unstructured mesh

Monte Carlo simulations are the gold standard in particle transport simulations, for shielding and nuclear reactor simulations Full core simulations can require terabytes of memory for tracking pin-resolved (up to 50k pins), nuclide (200 tracked), axial (20-200 zones) reaction rates Domain decomposition, splitting the calculation and the memory storage, among several nodes is challenging as it involves communication of particles across domain boundaries We created a new MOOSE application, MaCaw, to study this problem Algorithms developed for ray tracing in MOOSE were adapted for simulating neutral particle transport Physics and tally routines in OpenMC, dynamically linked, are called from the application

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Train small, model big: Scalable physics simulators via reduced order modeling and domain decomposition

Numerous cutting-edge scientific technologies originate at the laboratory scale, but transitioning them to practical industry applications is a formidable challenge. Traditional pilot projects at intermediate scales are costly and time-consuming. An alternative, the pilot-scale model, relies on high-fidelity numerical simulations, but even these simulations can be computationally prohibitive at larger scales. To overcome these limitations, we propose a scalable, physics-constrained reduced order model (ROM) method. The ROM identifies critical physics modes from small-scale unit components, projecting governing equations onto these modes to create a reduced model that retains essential physics details. We also employ Discontinuous Galerkin Domain Decomposition (DG-DD) to apply ROM to unit components and interfaces, enabling the construction of large-scale global systems without data at such large scales. Here this method is demonstrated on the Poisson and Stokes flow equations, showing that it can solve equations about 15–40 times faster with only ~1% relative error. Furthermore, ROM takes one order of magnitude less memory than the full order model, enabling larger scale predictions at a given memory limitation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Reparameterizing Litter Decomposition Using a Simplified Monte Carlo Method Improves Litter Decay Simulated by a Microbial Model and Alters Bioenergy Soil Carbon Estimates

Litter decomposition determines soil organic matter (SOM) formation and plant-available nutrient cycles. Therefore, accurate model representation of litter decomposition is critical to improving soil carbon (C) projections of bioenergy feedstocks. Soil C models that simulate microbial physiology (i.e., microbial models) are new to bioenergy agriculture, and their parameterization is often based on small datasets or manual calibration to reach benchmarks. Here, we reparameterized litter decomposition in a microbial soil C model (CORPSE - Carbon, Organisms, Rhizosphere, and Protection in the Soil Environment) using the continental-scale Long-term Inter-site Decomposition Experiment Team (LIDET) dataset which documents decomposition across a range of litter qualities over a decade. Here, we conducted a simplified Monte Carlo simulation that constrained parameter values to reduce computational costs. The LIDET-derived parameters improved modeled C and nitrogen (N) remaining, decomposition rates, and litter mean residence times as compared to Baseline parameters. We applied the LIDET litter decomposition parameters to a microbial bioenergy model (Fixation and Uptake of Nitrogen – Bioenergy Carbon, Rhizosphere, Organisms, and Protection) to examine soil C estimates generated by Baseline and LIDET parameters. LIDET parameters increased estimated soil C in bioenergy feedstocks, with even greater increases under elevated plant inputs (i.e., by increasing residue, N fertilization). This was due to the integrated effects of plant litter quantity, quality, and agricultural practices (tillage, fertilization). Collectively, we developed a simple framework for using large-scale datasets to inform the parameterization of microbial models that impacts projections of soil C for bioenergy feedstocks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Temperature-based reactive flow model for triaminotrinitrobenzene (TATB) plastic bonded explosives

A new reactive flow model is presented for triaminotrinitrobenzene (TATB)-based plastic bonded explosives, applicable to shock initiation and steady detonation problems of differing initial temperature. Temperature disequilibrium is assumed between unreacted explosive, material in the vicinity of compressed defects (called hot spots), and reaction products. The model incorporates temperature-dependent decomposition reaction rates. Particularly, Arrhenius model parameters were derived from quantum-based molecular dynamics simulations of TATB decomposition. Further, a model of detonation carbon aggregation is incorporated, describing the slow release of energy inherent to detonation in TATB-based materials. Model parameters were calibrated against gas gun shock initiation experiments and steady detonation rate stick tests. The predictive ability of the model in the shock initiation regime is tested against recent thin pulse experiments. The model is found to perform equally well in predicting the size-effect curve of ambient, cold, and hot rate sticks. The present work demonstrates the viability of incorporating results from subscale simulations into a continuum-scale reactive flow model.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Flexible User-Defined Domain Decomposition in Kilometer-Scale E3SM Land Model Simulation

The Energy Exascale Earth System Model (E3SM) Land Model (ELM) has been extended to kilometer-scale (km-ELM) resolutions, enabling high-fidelity simulations of terrestrial processes at 1 km x 1 km grid spacing. In ELM, domain decomposition partitions the computational domain across processors, ensuring efficient parallel execution. Currently, round-robin decomposition is applied, providing a straightforward way to distribute computational workload. As ELM continues evolving at the kilometer-scale (km-scale), particularly with integrating lateral flow modeling, decomposition strategies must also account for the increased workload and data movement. This paper introduces a flexible user-defined domain decomposition framework, allowing users to customize domain partitioning based on application requirements. The impact of different decomposition strategies is evaluated across various applications concerning computation, communication, and I/O. Results demonstrate that while 1D partitioning yields superior I/O performance, k-nearest neighbors (KNN) clustering effectively reduces inter-process communication overhead. This study lays the groundwork for scalable partitioning in large-scale land surface simulations, enhancing next-generation Earth system modeling.

Wang, Dali [ORNL] (ORCID:0000000168065108)↗
DOE OSTI2024

Extended dynamic mode decomposition for model reduction in fluid dynamics simulations

High computational cost and storage/memory requirements of fluid dynamics simulations constrain their usefulness as a predictive tool. Reduced-order models (ROMs) provide a viable solution to this challenge by extracting the key underlying dynamics of a complex system directly from data. We investigate the efficacy and robustness of an extended dynamic mode decomposition (xDMD) algorithm in constructing ROMs of three-dimensional cardiovascular computations. Focusing on the ROMs' accuracy in representation and interpolation, we relate these metrics to the truncation rank of singular value decomposition, which underpins xDMD and other approaches to ROM construction. Our key innovation is to relate the truncation rank to the singular values of the original flow problem. This result establishes a priori guidelines for the xDMD deployment and its likely success as a means of data compression and reconstruction of the system's dynamics from dominant spatiotemporal structures present in the data.

Mechanics↗
DOE OSTI2021

Parametrized Hamiltonian simulation using quantum optimal control

Analog quantum simulation offers a hardware-specific approach to studying quantum dynamics, but mapping a model Hamiltonian onto the available device parameters requires matching the hardware dynamics. In this work, we introduce a paradigm for quantum Hamiltonian simulation that leverages digital decomposition techniques and optimal control to perform analog simulation. We validate this approach by constructing the optimal analog controls for a superconducting transmon device to emulate the dynamics of an extended Bose-Hubbard model. We demonstrate the role of control time, digital error, and pulse complexity, and we explore the accuracy and robustness of these controls. We conclude by discussing the opportunity for implementing this paradigm in near-term quantum devices.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Advanced modeling and simulation of research reactors using dynamic mode decomposition

Full text of publication follows. Due to the ever-increasing safety requirements, the current trend of nuclear reactor analysis is shifting towards high-fidelity multi-physics models, which have a very high computational cost and modelling complexity. As the cost of even a single model run makes it impossible to analyse the behaviour and performance of these models on large-scale commercial plants, it has become even more significant to provide suitable benchmarks to validate and test them extensively. In this sense, research reactors offer a promising solution for the initial validation of high-fidelity models, as they are significantly smaller than commercial reactors and their characteristics are well known. In particular, the reactors of the TRIGA family have been used to assess and validate models and methods for Generation-IV designs, as they have some similar features (such as the dominance of natural convection as cooling mechanism and the difficulties in performing sub-channel analysis using standard codes). Still, the computational requirements of high-fidelity models make them unsuitable for real-time analysis, even following their assessment on research reactors. In this sense, Model Order Reduction (MOR) techniques give an additional strategy to reduce the computational cost of high-fidelity models (whilst preserving sufficient accuracy). In particular, this work focuses on Dynamic Mode Decomposition (DMD), a non-intrusive MOR technique that aims at representing models with explicit temporal dynamics by extracting the time-varying characteristics and the governing structures based only on a set of available data, thus without needing any underlying knowledge of the governing equations. In addition, DMD also computes a low-dimensional surrogate of the dynamic matrix of the system, making it suited for stability analysis and real-time evaluations. This work focuses on the application and validation of the DMD method on the Computational Fluid-Dynamics (CFD) model TRIGA Mark II reactor, also discussing in detail the potentiality of this algorithm as an advanced modelling tool for nuclear reactor analysis. (author)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

CORPSE model with litter decomposition parameters derived from the LIDET dataset

This is a version of the CORPSE model (Carbon, Organisms, Rhizosphere and Protection in the Soil Environment, Sulman et al. 2014) that uses litter decomposition parameters derived from a modified Monte Carlo simulation using the LIDET litter decomposition dataset (Long-term Intersite Decomposition Experiment Team, Harmon 2013). The code also includes the Baseline parameters, and the eight other best parameter sets identified in a modified Monte Carlo simulation. Related publication:Juice, S.M., Ridgeway, J.R., Hartman, M.D., Parton, W.J., Berardi, D.M., Sulman, B.N., Allen, K.E., & Brzostek, E.R. Reparameterizing litter decomposition using a simplified Monte Carlo method improves litter decay simulated by a microbial model and alters bioenergy soil carbon estimates. Description of files:The folder "Input Files" contains one folder for each LIDET site with data necessary to run the model. Note that "(site)" in the filenames below indicates where the LIDET site code appears (see Table 1 for site codes). Data streams include: CORPSE_full_spinup_litter.csv, CORPSE_full_spinup_rhizo.csv, CORPSE_full_spinup_bulk.csv, litterbag_init_100g_6spp.csv: initial C and N (kg C or N/m2) pool values for each soil layer, the litterbag_init_100_6spp.csv file is for the litterbag layer and is the same file for all sites. All initial C and N files have the same columns (Column - Description - Units) uFastC - Unprotected fast decomposing carbon - kg carbon/m2 uSlowC - Unprotected slow decomposing carbon - kg carbon/m2 uNecroC - Unprotected necromass carbon - kg carbon/m2 pFastC - Protected fast decomposing carbon - kg carbon/m2 pSlowC - Protected slow decomposing carbon - kg carbon/m2 pNecroC - Protected necromass carbon - kg carbon/m2 livingMicrobeC - Carbon in living microbial biomass - kg carbon/m2 uFastN - Unprotected fast decomposing nitrogen - kg nitrogen/m2 uSlowN - Unprotected slow decomposing nitrogen - kg nitrogen/m2 uNecroN - Unprotected necromass nitrogen - kg nitrogen/m2 pFastN - Protected fast decomposing nitrogen - kg nitrogen/m2 pSlowN - Protected slow decomposing nitrogen - kg nitrogen/m2 pNecroN - Protected necromass nitrogen - kg nitrogen/m2 inorganicN - Inorganic nitrogen - kg nitrogen/m2 CO2 - Carbon in carbon dioxide - kg carbon/m2 livingMicrobeN - Nitrogen in living microbial biomass - kg nitrogen/m2 soilT (site) DOY274start.csv: Average daily soil temperature (oC) interpolated from previously calculated monthly values used in DayCent LIDET simulations (Bonan et al., 2013). soilT (site) DOY274start.csv: Average daily soil volumetric water content (VWC) scalar interpolated from previously calculated monthly values used in DayCent LIDET simulations (Bonan et al., 2013). litter production.csv: Average daily litter production values for each site, data sources listed in Table S3 of related publication. litter (site) CN.csv: C:N ratio for each species from LIDET dataset (Table 2, Harmon 2013). (site).csv: Table indicating number of observations for each species decomposed at each site. Instructions: Save the model code ("CORPSE_LIDET.R") and "Input Files" folder in the same folder. Also make a folder for the model output (e.g., "results_Baseline") in the same folder. Set the working directory (setwd) in the model code to the folder with the files saved in step #1. Select the parameter set to use for the litter and litterbag compartments, comment out all other parameter sets. Run code. Output will be saved in the folder made in step 1. Output destination can be changed as necessary in code section called "Running the model." Table 1 LIDET sites and site codes used in model files. Site Code - Site AND - H.J. Andrews Experimental Forest BNZ - Bonanza Creek Experimental Forest BSF - Blodgett Research Forest CDR - Cedar Creek Natural History Area CPR - Central Plains Experimental Range HBR - Hubbard Brook Experimental Forest HFR - Harvard Forest JUN - Juneau KBS - Kellogg Biological Station KNZ - Konza Prairie Research Natural Area NWT - Niwot Ridge/Green Lakes Valley OLY - Olympic National Park OLY Conifer forest SEV - Sevilleta National Wildlife Refuge SMR - Santa Margarita Ecological Reserve UFL - University of Florida VCR - Virginia Coast Reserve Table 2 LIDET species and species codes used in model files (6 common species). Species - Species Code Sugar maple (Acer saccharum) - ACSA Drypetes (Drypetes glauca) - DRGL Red pine (Pinus resinosa) - PIRE Chestnut oak (Quercus prinus) - QUPR Western redcedar (Thuja plicata) - THPL Wheat (Triticum aestivum) - TRAE References:Bonan, G. B., Hartman, M. D., Parton, W. J., & Wieder, W. R. (2013). Evaluating litter decomposition in earth system models with long-term litterbag experiments: an example using the Community Land Model version 4 (CLM4). Global Change Biology, 19(3), 957-974. https://doi.org/https://doi.org/10.1111/gcb.12031 Harmon, M. (2013). LTER Intersite Fine Litter Decomposition Experiment (LIDET), 1990 to 2002. Long-Term Ecological Research. Forest Science Data Bank, Corvallis, OR. [Data set]. Accessed http://andlter.forestry.oregonstate.edu/data/abstract.aspx?dbcode=TD023. https://doi.org/10.6073/pasta/f35f56bea52d78b6a1ecf1952b4889c5. Sulman, B. N., Phillips, R. P., Oishi, A. C., Shevliakova, E., & Pacala, S. W. (2014). Microbe-driven turnover offsets mineral-mediated storage of soil carbon under elevated CO2. Nature Climate Change, 4, 1099 - 1102. https://doi.org/10.1038/nclimate2436

Juice, Stephanie↗
DOE OSTI2021

Error-controlled, progressive, and adaptable retrieval of scientific data with multilevel decomposition

Extreme-scale simulations and high-resolution instruments have been generating an increasing amount of data, which poses significant challenges to not only data storage during the run, but also post-processing where data will be repeatedly retrieved and analyzed for a long period of time. The challenges in satisfying a wide range of post-hoc analysis needs while minimizing the I/O overhead caused by inappropriate and/or excessive data retrieval should never be left unmanaged. In this paper, we propose a data refactoring, compressing, and retrieval framework capable of 1) fine-grained data refactoring with regard to precision; 2) incrementally retrieving and recomposing the data in terms of various error bounds; and 3) adaptively retrieving data in multi-precision and multi-resolution with respect to different analysis. With the progressive data re-composition and the adaptable retrieval algorithms, our framework significantly reduces the amount of data retrieved when multiple incremental precision are requested and/or the downstream analysis time when coarse resolution is used. Experiments show that the amount of data retrieved under the same progressively requested error bound using our framework is 64% less than that using state-of-the-art single-error-bounded approaches. Parallel experiments with up to 1, 024 cores and ~ 600 GB data in total show that our approach yields 1.36× and 2.52× performance over existing approaches in writing to and reading from persistent storage systems, respectively.

Liang, Xin↗