Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “simulated data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Synapse v1.0

Synapse (SYNergistic software platform for AI, Physics Simulations, and Experiments) is a software package meant to deploy real-time guidance from simulations during experimental campaigns, The software package contains functionalities to collect data from simulations (e.g. running at NERSC) and experiments (e.g. from the BELLA facility at LBNL) into a database, train ML surrogate models from this data, and display the predictions of the surrogate model in the control room of an experimental facility, so as to guide on-going experimental campaign. This software was developed as part of an on-going LDRD.

Lehe, Remi [Lawrence Berkeley National Laboratory ↗
DOE OSTI2024

Power System Operational Impacts of Electric Vehicle Dynamic Wireless Charging

The electrification of the transportation sector poses an opportunity for reducing greenhouse gas (GHG) emissions from passenger vehicles. Electric vehicle (EV) charging through dynamic wireless power transfer (DWPT), known as roadway electrification, could shift EV demand profiles to better coincide with renewable electricity generation. However, this would be a very large new load and few studies evaluate the regional impacts of DWPT charging in a power transmission system. This paper defines methods that address dataset generation for passenger vehicle trips and models to evaluate regional impacts for this emerging technology. Household vehicle miles traveled (VMT) data form localized EV demand profiles through discrete-event simulation. This data serves as exogenous inputs for a Production Cost Model (PCM) of a synthetic transmission system based on the Electric Reliability Council of Texas's (ERCOT) network. EV charging methods are compared for both a 2018 baseline generation mixture and a high-renewable generation case incorporating 20 GW of installed solar photovoltaic (PV) capacity. The PCM employs unit commitment and economic dispatch (UC&ED) models to compare financial, environmental, and grid reliability impacts from EV charging across passenger EV adoption levels. In-transit charging could reduce grid operational costs by as much as 1.49%, with up to $13.7B saved in annual vehicle operational costs for consumers compared to gas-powered vehicles. Health impacts analysis from power plant and vehicle tailpipe emissions from this study show net health benefits increase by 40% for in-transit charging coupled with high renewable generation. Renewable resources provide an avenue for cost-effective in-transit charging with reduced emissions. The combination of dataset generation and open-source power system modeling establish a foundation for the holistic evaluation of regional DWPT impacts.

dynamic wireless power transfer↗
DOE OSTI2026

IK-Frag: Frag data generator for the PHITS simulation with the inverse kinematic reaction producing a focused neutron beam

IK-Frag has been developed for the creation of the nuclear cross-section data format, which is named frag data and can be used in PHITS, a Monte Carlo simulation code. IK-Frag focuses on the inverse kinematic reactions between a lithium or beryllium ion and a proton target. These reactions achieve naturally collimated neutrons and potentially reduce the necessity of radiation shielding. IK-Frag enables PHITS users to conduct simulations for the inverse kinematic reactions. The present software aims to contribute to future development of the neutron source system using the inverse kinematic reactions.

43 PARTICLE ACCELERATORS↗
DOE OSTI2026

Designing resilient IoT and Edge Computing with federated tinyML

The rapid growth of the Internet of Things (IoT) and Edge Computing (EC) has brought significant conveniences to modern society but has also greatly expanded the cyber attack surfaces, particularly as these technologies are being increasingly integrated into critical systems such as power grids, healthcare, and smart homes. Here, to improve IoT/EC’s cybersecurity posture, we leveraged Artificial Intelligence (AI) and Machine Learning (ML) by employing tinyML to monitor voluminous IoT data for cyber threats while addressing devices’ resource constraints, and utilizing Federated Learning (FL) to share local detection knowledge across the system while preserving privacy. Building on our three-layer architecture combining tinyML and FL to enhance autonomous cyber attack detection, this paper demonstrated that the architecture improves detection accuracy, reduces resource consumption, and enables lightweight, secure IoT device monitoring. These results were validated using the public N-BaIoT dataset as well as real IoT network traffic data collected under multiple attack scenarios from our testbeds. Additionally, we introduced an enhanced FL methodology with a novel preprocessing stage, including federated feature selection and global preprocessor construction, to address IoT/EC data heterogeneity. We developed a physical IoT testbed for attack simulations and data collection, implemented a tinyML-powered detector for realistic model validation, and also built a virtual testbed for scalable evaluations of FL models across diverse network environments.

Cognitive cyber↗
DOE OSTI2025

Active Learning Surrogates for Integrating Electron Microscopy and Computational Insights from Simulations in Autonomous Experiments

Artificial Intelligence (AI) combined with simulations and experiments has great potential to accelerate scientific discovery across technology and pharmaceuticals. However, the gap between simulations and experiments is challenging due to disparities in time and scale, making it difficult to estimate properties like energy and electronic states from experiments, and to provide feedback based on theoretical insights.Our research addresses the challenge by developing unique deep kernel based surrogate models that learns from microscopic images, mapping structural features to energy differences from defect formation. We start with full-training using simulated images to determine optimal settings, establishing a baseline for active learning. Using these settings from the baseline, active learning is trained, and predicts structures along simulation trajectories based on uncertainty and energetic stability, thus reducing data requirements, simulation time and computational costs. The results demonstrate that the model achieves a low average error margin of approximately 0.03 meV, indicating good performance. To enhance feature extraction and reconstruction capabilities, we developed an autoencoder-decoder as additional surrogate to create latent space to capture essential features, enabling precise comparisons between simulations and experiments. The results from this model achieved a reconstruction loss of around 0.2 and accurately reconstructed molecular structures.Overall, this work advances the steering of experiments through computational simulations by employing a surrogate models that actively predicts the trajectories of structural evolution, achieving time-to-solution comparable to experimental measurements.

Saranathan, Gayathri [Hewlett-Packard]↗
DOE OSTI2024

Estimating soil thermal inertia profiles from the passive equilibration of a temperature probe: Supporting data and code

This dataset contains measurements of soil thermal properties and organic layer thickness at three field sites on the Seward Peninsula near Nome, Alaska, USA. Code is also provided for estimation of soil thermal properties from temperature data, including a forward heat transfer mode, a library of simulated temperature data, and an inversion scheme to retrieve thermal properties from temperature measurements (Lamb et al., 2025). This dataset and code is published in support of the manuscript “Estimating Soil Thermal Inertia Profiles from the Passive Equilibration of a Temperature Probe” by Lamb et al., 2025, in which an experimental thermal measurements technique is developed. The dataset contains measurements of soil thermal properties and organic layer thickness obtained with an industry standard thermal properties analyzer and direct observations, as well as data generated using the experimental technique to estimate thermal properties and soil structure from temperature observations. All data is reported in .csv format, and all code is written in Python 3.7 using standard packages. This research was performed as a part of the NGEE Arctic project, which aims to advance model predictions of arctic carbon cycle responses to a changing climate over the 21st Century.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Machine Learning assisted optimization and parameter space exploration dataset of spin ice Hamiltonian

This repository contains both simulated and experimental structure factor data for the data challenge involving the inverse scattering problem. The simulated data were generated during a machine-learning-assisted optimization routine described in ref[1]. The experimental structure factor was measured on a rare-earth oxide, Dy2Ti2O7 using diffuse neutron scattering from time-of-flight techniques on the CORELLI instrument at the Spallation Neutron Source, Oak Ridge National Laboratory. A Metropolis Monte Carlo code implemented to run in a High-performance computing setting was used to calculated simulated structure factors for the spin-ice Hamiltonian at 680 mK, which is the same temperature as for the experimental data. The total size of all the files in this repository is 5.12 GB. A detailed description of the files is given below. ExperimentalData_630mK.dat – A linearized version of 3-dimensional experimental data of size 61×81×21. This data was processed to remove an estimation of non-magnetic background, including nuclear scattering signal and instrumentation background. Parameters.dat – 6700 samples were evaluated over the 4-dimensional parameter space (J_1, J_2, J_3 and J_(3^' )). There is an additional parameter, D in the spin Hamiltonian to mimic the dipolar interaction between magnetic ions. However, this parameter, D was fixed to a value determined by prior work. This file contains five columns for the parameters J_1, J_2, J_3, J_(3^' ) and D respectively. 3D_Simulation_Data.dat – The simulated structure factor, S(Q) data are included in this file. Each raw contains a linearized array of 3D volumes of S(Q) calculated for the parameter set given in the corresponding row of the file Parameters.dat. The size of the volume data was matched to the experimental data. Qx(h,-h,0).dat, Qy(k,k,-2k).dat, Qz(l,l,l).dat – These files contain the h, k, and l values along with the reciprocal vectors [h,-h,0], [k,k,-2k] and [l,l,l] respectively.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Harnessing Uncertainty through Functional Data Analysis in Gas Breakthrough Data

Detecting subsurface explosions from radionuclide gas migration through rock fractures is an effective way to identify nuclear activity. Los Alamos National Laboratory (LANL) has developed simulation methods, based on data from the 1962 Hardhat underground nuclear test, to predict gas breakthrough times at the surface. However, these methods rely on an imperfect understanding of the relationship between rock damage and fracture permeability. Our clinic project studies methods for predicting breakthrough curves that characterize total mass produced as a function of time, as well as quantifying the uncertainty associated with these predictions. The model that is currently employed to relate damage to permeability uses an empirically motivated power-law expression, with a range of parameter values that are compatible with the experimental Hardhat data. We develop emulators, built from functional data analysis techniques and trained on simulation data, that rapidly predict the gas breakthrough curve given a damage field and given the parameter values of the power-law equation. Using Bayesian regression, we address the problem of uncertainty quantification in our emulators. Finally, in order to test the robustness of the model, we further validate it on a damage field representing different physical conditions.

58 GEOSCIENCES↗
DOE OSTI2023

Characterizing Reactor Operations from Realistic Simulated Environmental Samples: Combining High-Performance Computing and Data Analytics

Environmental sampling is a common technique employed by inspectors and facility operators in nuclear safeguards, proliferation detection, and process monitoring contexts. Interpreting measurements performed on samples or collections of samples and ensuring the information extracted is accurate and precise is difficult. To date, these analyses have relied on simulated data to enable systematic studies; however, these models are inherently limited by the fidelity of the models and the implicit spatial averaging of isotopic composition or other signatures of interest. To advance this capability, we have refined the spatial discretization and expanded the range of physics in the simulation codes we use to perform reactor simulations and depletion calculations. This allows us to generate data that are more representative of real environmental samples, especially for the length scale of the isotopic composition and associated variation. Accordingly, these new data allow a more realistic assessment of traditional and new data analytic analysis methods. Here we present motivation for developing reactor simulations using high-performance computing methods and resources, impacts of these new simulations on our assessment of data analysis and interpretation methods, and initial results of developing and systematically testing data analytic methods designed to overcome the challenges expected of real-world samples. We also quantify the performance of these analyses using defensible statistical methods.

Dayman, Ken J.↗
DOE OSTI2025

Predictions for the Detectability of Milky Way Satellite Galaxies and Outer-Halo Star Clusters with the Vera C. Rubin Observatory

We predict the sensitivity of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) to faint, resolved Milky Way satellite galaxies and outer-halo star clusters. We characterize the expected sensitivity using simulated LSST data from the LSST Dark Energy Science Collaboration (DESC) Data Challenge 2 (DC2) accessed and analyzed with the Rubin Science Platform as part of the Rubin Early Science Program. We simulate resolved stellar populations of Milky Way satellite galaxies and outer-halo star clusters over a wide range of sizes, luminosities, and heliocentric distances, which are broadly consistent with expectations for the Milky Way satellite system. We inject simulated stars into the DC2 catalog with realistic photometric uncertainties and star/galaxy separation derived from the DC2 data itself. We assess the probability that each simulated system would be detected by LSST using a conventional isochrone matched-filter technique. We find that assuming perfect star/galaxy separation enables the detection of resolved stellar systems with $M_V$ = 0 mag and $r_{1/2}$ = 10 pc with >50% efficiency out to a heliocentric distance of ~250 kpc. Similar detection efficiency is possible with a simple star/galaxy separation criterion based on measured quantities, although the false positive rate is higher due to leakage of background galaxies into the stellar sample. When assuming perfect star/galaxy classification and a model for the galaxy-halo connection fit to current data, we predict that 89 +/- 20 Milky Way satellite galaxies will be detectable with a simple matched-filter algorithm applied to the LSST wide-fast-deep data set. Different assumptions about the performance of star/galaxy classification efficiency can decrease this estimate by ~7%-25%, which emphasizes the importance of high-quality star/galaxy separation for studies of the Milky Way satellite population with LSST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Transfer learning-based soybean LAI estimations by integrating PROSAIL, UAV, and PlanetScope imagery

Accurate Leaf Area Index (LAI) estimations at the soybean plot scale is achievable using high-resolution Unmanned Aerial Vehicle (UAV) imagery and field measurement samples. However, the limited coverage of UAV flights restricts large-scale remote sensing monitoring in expansive soybean fields. This study leverages the broad coverage and 3-m resolution of PlanetScope satellite imagery to extend LAI prediction from UAV to satellite scales through transfer learning, using UAV-scale LAI estimates as a benchmark to validate cross-scale consistency. To address this challenge, this study proposed the LAI-TransNet, a two-stage transfer learning framework designed for precise and scalable soybean LAI prediction across large areas, demonstrating its effectiveness in cross-scale monitoring. In Stage 1, a UAV-scale benchmark is established using PROSAIL-simulated UAV reflectance data (UAV-Sim) and field-measured soybean LAI. Traditional machine learning, deep learning, and transfer learning models are trained on a hybrid UAV-Sim and field-measured dataset (UAV-Sim_Measured), with the transfer learning model CNN-TL, fine-tuned using pre-trained weights derived from UAV-Sim, achieving the highest accuracy (R 2 = 0.81, RMSE = 0.64 m 2 /m 2 , rRMSE = 11.5 %). In Stage 2, LAI-TransNet is developed by fine-tuning the CNN-TL model on PlanetScope simulated data (PS-Sim), preprocessed via cross-domain mapping to align UAV and satellite spectral features. Real PlanetScope imagery is corrected for reflectance consistency with reference to UAV imagery spectral profiles. LAI-TransNet outperforms other deep learning models trained directly on PS-Sim (R 2 = 0.69 vs. 0.60–0.63), ensuring robust cross-scale consistency. In conclusion, by bridging UAV and satellite scales, LAI-TransNet enables large-scale soybean LAI monitoring, enhancing precision agriculture management through improved monitoring with the PlanetScope imagery.

Leaf area index (LAI)↗
DOE OSTI2021

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗
DOE OSTI2024

Once in a blue stream: Detection of recent star formation in the NGC 7241 stellar stream with MEGARA

Aims.In this work we study the striking case of a narrow blue stream with a possible globular cluster-like progenitor around the NGC 7241 galaxy and its foreground dwarf companion. We want to figure out if the stream was generated by tidal interaction with NGC 7241 or if it first interacted with the foreground dwarf companion and later both fell together toward NGC 7241. Methods.We used four sets of observations, including a follow-up spectroscopic study of this stream based on data taken with the MEGARA instrument at the 10.4 m Gran Telescopio Canarias using the integral field spectroscopy mode, the Mount Lemmon 0.80 m telescope, the Telescopio NazionaleGalileo, the DESI Imaging Legacy surveys, and GALEX archival data. We also used high-resolution zoomed-in cosmological simulations. Results.Our data suggest that the compact object we detected in the stream is a foreground Milky Way halo star. Near this compact object we detect emission lines overlapping a less compact, bluer, and fainter blob of the stream that is clearly visible in both ultraviolet and optical deep images. From its heliocentric systemic radial velocity derived from the [O III]λ5007 Å lines (V syst = 1548.58 ± 1.80 km s −1 ) and new UV and optical broadband photometry, we conclude that this overdensity could be the actual core of the stream, with an absolute magnitude ofM g ∼ −10 and ag − r = 0.08 ± 0.11, consistent with a remnant of a low-mass dwarf satellite undergoing a current episode of star formation. From the width of the stream and assuming a circular orbit, we calculate that the progenitor mass can be typical of a dwarf galaxy, but it could also be substantially lower if the stream is on a very radial orbit or if it was created by tidal interaction with the companion dwarf instead of with NGC 7241. These estimates also suggest that this is one of the lowest mass streams detected to date beyond the Local Group. Finally, we find that blue stellar streams containing star formation regions are commonly predicted by high-resolution cosmological simulations of galaxies lighter than the Milky Way. This scenario is consistent with the processes explaining the bursty star formation history of some dwarf satellites, which are followed by a gas depletion and a fast quenching once they enter within the virial radius of their host galaxies for the first time. Thus, it is likely that the stream’s progenitor is undergoing a star formation burst comparable to those that have shaped the star formation history of several Local Group dwarfs in the last few gigayears.

Astronomy & Astrophysics↗
DOE OSTI2021

Diagnosing nuclear power plant pipe wall thinning due to flow accelerated corrosion using a passive, thermal non-destructive evaluation method: Feasibility assessment via numerical experiments

Flow accelerated corrosion (FAC) in nuclear power plant pipes is one of the leading causes of accidents, fatalities, damage and outages. Current FAC identification methods employ expensive sensing technology and are “active” methods, where the response of the piping system to an externally-generated thermal, mechanical or optical excitation must be measured. As a result, these techniques require a disruptive and time-consuming setup. Here we propose a method that utilizes pipe surface temperature measurements to passively monitor for FAC-induced pipe wall thinning without the need for expensive equipment or post-installation setup time. This diagnostic method utilizes a simulation data-driven diagnostic model to estimate the amount of thickness reduction in a pipe based on changes in measured steady-state pipe temperatures. In order to reduce the computational burden of generating large, simulation-based datasets, the behavior of the insulation of the pipe was modeled using a suitably calibrated heat transfer boundary parameter. Additionally, global sensitivity analysis was performed to determine system parameter(s), such as the temperature of water flowing inside the pipe, which significantly affect the steady state pipe wall temperature and could cause errors in diagnosis. Two diagnostic models, one using only the change in steady-state temperature as an indicator for FAC-induced pipe wall thinning and the other using water temperature as an additional diagnostic model input were evaluated for their ability to estimate thickness reductions in a pipe using simulated pipe wall temperature data. For the numerical experiments conducted in this work, both models estimated wall thickness with errors within 0.5 mm, indicating that the proposed technique can potentially be used as a low-cost, first-pass method for FAC monitoring.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Magnetoelastic coupling and spin contributions to entropy and thermal transport in biferroic yttrium orthochromite

Direct engineering of material properties through exploitation of spin, phonon, and charge-coupled degrees of freedom is an active area of development in materials science. However, the relative contribution of the competing orders to controlling the desired behavior is challenging to decipher. In particular, the independent role of phonons, magnons, and electrons, quasiparticle coupling, and relative contributions to the phase transition free energy largely remain unexplored, especially for magnetic phase transitions. Here, we study the lattice and magnetic dynamics of biferroic yttrium orthochromite using Raman, infrared, and inelastic neutron spectroscopy techniques, supporting our experimental results with first-principles lattice dynamics and spin-wave simulations across the antiferromagnetic transition at T N ~ 138 K. Spectroscopy data and simulations together with the heat capacity (C p ) measurements, allow us to quantify individual entropic contributions from phonons (0.01 ± 0.01k B atom -1 ), dilational (0.03 ± 0.01k B atom -1 ), and magnons (0.11 ± 0.01k B atom -1 ) across TN. High-resolution phonon measurements conducted in a magnetic field show that anomalous T-dependence of phonon energies across TN originates from magnetoelastic coupling. Additionally, phonon scattering is primarily governed by the phonon–phonon coupling, with little contribution from magnon–phonon coupling, short-range spin correlations, or magnetostriction effects; a conclusion further supported by our thermal conductivity measurements conducted up to 14 T, and phenomenological modeling.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Deep learning for in situ data compression of large turbulent flow simulations

As the size of turbulent flow simulations continues to grow, in situ data compression is becoming increasingly important for visualization, analysis, and restart checkpointing. For these applications, single-pass compression techniques with low computational and communication overhead are crucial. In this paper we present a deep-learning approach to in situ compression using an autoencoder architecture that is customized for three-dimensional turbulent flows and is well suited for contemporary heterogeneous computing resources. The autoencoder is compared against a recently introduced randomized single-pass singular value decomposition (SVD) for three different canonical turbulent flows: decaying homogeneous isotropic turbulence, a Taylor-Green vortex, and turbulent channel flow. Our proposed fully convolutional autoencoder architecture compresses turbulent flow snapshots by a factor of 64 with a single pass, allows for arbitrarily sized input fields, is cheaper to compute than the randomized single-pass SVD for typical simulation sizes, performs well on unseen flow configurations, and has been made publicly available. The results reported here show that the autoencoder dramatically outperforms a randomized single-pass SVD with similar compression ratio and yields comparable performance to a higher-rank decomposition with an order of magnitude less compression in regard to preserving a number of important statistical quantities such as turbulent kinetic energy, enstrophy, and Reynolds stresses.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Improving The Simulation Of Muons In The Rock Around The NOvA Near Detector

This thesis investigates the viability of the PROPOSAL lepton propagation tool as an alternative to the commonly used GEANT4 simulation for muon propagation within the NOvA experiment. The main objectives of this research were twofold: first, to validate the accuracy of PROPOSAL data in simulating rock muons when compared to GEANT4 data; second, to explore the practical application of transporting cosmic muons through the NOvA rock region using PROPOSAL. Additionally, an 1D analysis was conducted to determine the optimal altitude at which to hand over the simulation task from PROPOSAL to GEANT4, should we choose to use PROPOSAL for rock simulations and GEANT4 for detector simulations in the NOvA experiment. By thoroughly examining these aspects, this study aims to contribute to the advancement of muon simulation techniques and their implementation in high-energy experiments such as NOvA.

Parvez, Radwan↗
DOE OSTI2022

Sim-Situ: A Framework for the Faithful Simulation of in situ Processing

The amount of data generated by numerical simulations in various scientific domains led to a fundamental redesign of how the analysis and visualization of simulation outputs are performed. The throughput and capacity of storage subsystems have not evolved as fast as the computing power in extreme-scale supercomputers, making the classical post-hoc approach highly inefficient. In situ processing has then emerged as a solution in which simulation and data analysis/visualization are intertwined for better performance and greater interactivity.Determining the best allocation, i.e., how many resources to allocate to simulation and analysis respectively, mapping, i.e., where and at which frequency to run the analysis/visualization, and data transfer mode is a complex task whose performance assessment is crucial to the efficient execution of in situ processing. However, such a performance evaluation of different strategies usually relies either on directly running them on the targeted execution environments, which can rapidly become extremely time- and resource-consuming, or on resorting to simplified models of the components of an in situ application, which can lack of realism. In both cases, the validity of the performance evaluation is limited.In this paper, we present Sim-Situ, a simulation-based framework for the faithful performance evaluation of in situ processing strategies. We designed Sim-Situ to reflect the typical features of in situ processing systems. Thanks to its modular design, Sim-situ has the necessary flexibility to easily and faithfully evaluate the behavior and performance of various allocation, mapping, and data transfer strategies. We illustrate the simulation capabilities of Sim-Situ on a Molecular Dynamics use case. We study the impact of different strategies on performance and show how users can leverage Sim-Situ to determine interesting tradeoffs when adding analysis/visualization components to their application.

Honoré, Valentin↗