Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data statistics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Fuel Property Effects on Stochastic Preignition Events During Engine Load Transitions

Stochastic preignition (SPI) is an abnormal combustion phenomenon that can cause catastrophic engine damage. There have been several proposed mechanisms of SPI, where a uniform source is still not certain, however, SPI tendencies have been shown to be influenced by engine operating conditions, oil composition, engine age, and fuel chemical and physical properties. Laboratory research and testing for SPI propensity is challenging given the stochastic nature of events, as well as the potential for significant degradation of the engine platform and measuring equipment over time. Thus, SPI specific experiments are generally conducted under either sustained or cyclic patterning of steady-state operating conditions to avoid the influence of transient engine boundary conditions on test parameters of interest (e.g. oil additive package, fuel properties, engine speed/load, etc.). In this work a cyclically varying SPI test sequence involves a 5 min engine warmup period at a low engine load of around 4 bar gross indicated mean effective pressure (IMEPg), followed by a transition to high load (~20 bar IMEPg) at a constant 2000 rev/min engine speed for a total of 25 min. This individual test sequence load schedule is then sequentially repeated 10 times to generate significant statistical data for analysis. This work examines the influence of fuel chemical and physical properties on SPI tendency during the unsteady portion of the 10-cycle sequence (the first 5 min of the high load operation in each sequence of the loading cycle) which has been discarded from previous analyses due to the uncertainty in engine operating and thermal boundary conditions. Results from this analysis suggest an increasing trend in the ratio of SPI events during the unsteady test period relative to the steady test period with increasing fuel Reid Vapor Pressure (RVP), implying differences in uncontrolled ignition source terms, possibly from, fuel wall interactions and retention during the load transition phase of the test.

Splitter, Derek [ORNL] (ORCID:0000000174044047)↗
DOE OSTI2024

Out-of-Distribution Detection and Radiological Data Monitoring Using Statistical Process Control

Abstract Machine learning (ML) models often fail with data that deviates from their training distribution. This is a significant concern for ML-enabled devices as data drift may lead to unexpected performance. This work introduces a new framework for out of distribution (OOD) detection and data drift monitoring that combines ML and geometric methods with statistical process control (SPC). We investigated different design choices, including methods for extracting feature representations and drift quantification for OOD detection in individual images and as an approach for input data monitoring. We evaluated the framework for both identifying OOD images and demonstrating the ability to detect shifts in data streams over time. We demonstrated a proof-of-concept via the following tasks: 1) differentiating axial vs. non-axial CT images, 2) differentiating CXR vs. other radiographic imaging modalities, and 3) differentiating adult CXR vs. pediatric CXR. For the identification of individual OOD images, our framework achieved high sensitivity in detecting OOD inputs: 0.980 in CT, 0.984 in CXR, and 0.854 in pediatric CXR. Our framework is also adept at monitoring data streams and identifying the time a drift occurred. In our simulations tracking drift over time, it effectively detected a shift from CXR to non-CXR instantly, a transition from axial to non-axial CT within few days, and a drift from adult to pediatric CXRs within a day—all while maintaining a low false positive rate. Through additional experiments, we demonstrate the framework is modality-agnostic and independent from the underlying model structure, making it highly customizable for specific applications and broadly applicable across different imaging modalities and deployed ML models.

Zamzmi, Ghada↗
DOE OSTI2025

Hot Droughts and Forest Tree Dynamics in the Amazon - Statistical Models, Scripts, Data, and Outputs

This package contains data, outputs, equations, and R scripts for analyses for manuscript entitled "Hot droughts in the Amazon: A window to a future hypertropical climate" by J. Chambers et al., in particular it contains statistical models and analyses for the INPA BIONTE tree mortality study. The Models folder contains details for all statistical models in PDF files. The Scripts folder contains the R scripts for Bayesian Hierarchical Models (two text files) and SEMs (one text file) are separate and reasonably annotated. All data associated with these scripts are in the data folder. The Data folder contains two of the three CSV files used for the analyses and are called by the R scripts. Two of them are part of published datasets (`BIONTE_mortality-rates.csv` from Lima et al. 2024, DOI:10.15486/ngt/1898910 and `SPEI.csv` from Pastorello et al. 2023 DOI:10.15486/ngt/1958257) and also provided in this package for convenience (please see the corresponding datasets for usage and citation terms). The third dataset (`BIONTE_gapfilled_wd.csv`) contains sensitive information and can be obtained by contacting the manuscript lead author. The Outputs folder contains the two output files that provide extra information about the analyses. The file `figuresFeb2025d.pdf` contains all the figures from the manuscript - captions are in the manuscript. The file `ChambersMS.pdf` contains primary results from Bayesian statistical models, regression analyses, and validation steps applied to the tree mortality data from the INPA experiments. The document includes visual summaries, model diagnostics, and leave-one-out (LOO) validation results. A breakdown of file contents can be found in the README file that is part of this package.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Critical statistical assessment of data in metal additive manufacturing

Obtaining high quality data reflecting the relationships between the additive manufacturing (AM) process parameters, material microstructure and mechanical properties is crucial for the use of machine learning in AM. A database of over 4,000 data entries of metal AM was created thanks to a large number of literature studies on key process parameters and indicators of build quality. Meta-analysis reveals critical biases in the literature. Firstly, majority of studies report only high quality builds, these imbalances in reporting result in weak correlation between process parameters, properties and consolidation, limiting the ability of machine learning models to generalize beyond optimized conditions. Nevertheless, the trained models accurately predict yield strength ($R^2 = 0.85$), suggesting that certain process–property relationships are effectively captured within these models. Secondly, quantitative microstructural data are largely absent, limiting the learning of the microstructure-mechanical properties relationships. Finally, current process window identification is based largely on the consolidation, despite significant uncertainty in its measurement. It is important to identify the process map on the basis of not only the consolidation, but also mechanical behaviour under loading. Such a identification shows that 316 L and Inconel have much larger process map (i.e. highly printable) in comparison to the AlSi10Mg and Ti6Al4V.

Additive manufacturing↗
DOE OSTI2024

DESI 2024: reconstructing dark energy using crossing statistics with DESI DR1 BAO data

Here, we implement Crossing Statistics to reconstruct in a model-agnostic manner the expansion history of the universe and properties of dark energy, using DESI Data Release 1 (DR1) BAO data in combination with one of three different supernova compilations (PantheonPlus, Union3, and DES-SN5YR) and Planck CMB observations. Our results hint towards an evolving and emergent dark energy behaviour, with negligible presence of dark energy at z ≳ 1, at varying significance depending on data sets combined. In all these reconstructions, the cosmological constant lies outside the 95% confidence intervals for some redshift ranges. This dark energy behaviour, reconstructed using Crossing Statistics, is in agreement with results from the conventional w 0 –w a dark energy equation of state parametrization reported in the DESI Key cosmology paper. Our results add an extensive class of model-agnostic reconstructions with acceptable fits to the data, including models where cosmic acceleration slows down at low redshifts. We also report constraints on H 0 r d from our model-agnostic analysis, independent of the pre-recombination physics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Multidimensional scaling informed by F -statistic: Visualizing grouped microbiome data with inference

Multidimensional scaling (MDS) is a widely used dimensionality reduction technique in microbial ecology data analysis that captures the multivariate structure of the data while preserving pairwise distances between samples. While improvements in MDS have enhanced the ability to reveal group-specific data patterns, these MDS-based methods require prior assumptions for inference, limiting their application in general microbiome analysis. Here, in this study, we introduce a new MDS-based ordination method, “F-informed MDS,” which configures the data distribution based on the F-statistic, the ratio of dispersion between groups sharing common and different characteristics. Using semisynthetic datasets, we demonstrate that the proposed method is robust to hyperparameter selection while maintaining statistical significance throughout the ordination process. Various quality metrics for evaluating dimensionality reduction confirm that F-informed MDS is comparable to state-of-the-art methods in preserving both local and global data structures. Its application to a diatom-associated bacterial community suggests the role of this new method in interpreting the community’s response to the host. Our approach offers a well-founded refinement of MDS that aligns with statistical test results, which can be beneficial for broader multidimensional data analyses in microbiology and ecology. This new visualization tool can be incorporated into standard microbiome data analyses.

Biological and medical sciences↗
DOE OSTI2025

Data-driven high-dimensional statistical inference with generative models

Crucial to many measurements at the LHC is the use of correlated multi-dimensional information to distinguish rare processes from large backgrounds, which is complicated by the poor modeling of many of the crucial backgrounds in Monte Carlo simulations. In this work, we introduce HI-SIGMA, a method to perform unbinned high-dimensional statistical inference with data-driven background distributions. In contradistinction to many applications of Simulation Based Inference in High Energy Physics, HI-SIGMA relies on generative ML models, rather than classifiers, to learn the signal and background distributions in the high-dimensional space. These ML models allow for interpretable inference while also incorporating model errors and other sources of systematic uncertainties. We showcase this methodology on a simplified version of a di-Higgs measurement in the bbγγ final state, where the di-photon resonance allows for background interpolation from sidebands into the signal region. We demonstrate that HI-SIGMA provides improved sensitivity as compared to standard classifier-based methods, and that systematic uncertainties can be straightforwardly incorporated by extending methods which have been used for histogram based analyses.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Searches for New Long-Lived Particles and Upgrade to the ATLAS Inner Detector (Final Technical Report)

The search for new fundamental particles is one of the defining goals of the Large Hadron Collider (LHC). The discovery of the Higgs Boson by the ATLAS and CMS collaborations provided the capstone of the Standard Model of particle physics, but outstanding questions remain. Why does the Higgs boson have a mass of 125 GeV when its natural mass would be many orders of magnitude larger? Is there a universal symmetry which unites all three forces described by the Standard Model? Can that symmetry be extended to include gravity? Is dark matter, evidenced by astronomical observations, made of a particle that interacts via Standard Model forces with the rest of matter? Together, these motivations provide compelling arguments that new physical processes await discovery. This project addressed some outstanding questions about the fundamental particles and their interactions with the ATLAS experiment at the Large Hadron Collider. In particular, the project improved the discovery potential for new, long- lived particles produced via electroweak processes in proton-proton collisions and set world-leading limits on their existence for certain values of their potential mass and lifetime. To achieve this, the project developed new data analysis methods, developed new triggers to select events with new long-lived particles during data-taking of the ATLAS experiment, and analyzed the largest proton–proton collision dataset ever produced. The project also supported significant development of the data acquisition software for the upgrade to the ATLAS inner detector, the Inner TracKer (ITk). The upgrade of the ATLAS inner detector is essential to the success of the entire Phase II physics program on ATLAS. Personnel supported by the project provided support for integration, assembly, and testing of the inner two layers of the ITk pixel system during its prototype and pre-production phase. Four PhD students and two post-doctoral scholars were supported by the grant and received invaluable scientific training as part of the research endeavor. The students and postdocs gained essential professional skills in the areas of advanced data analysis techniques, statistical analysis of data and simulation, programming in C++ and Python, hardware and instrumentation development, and presentation and collaboration skills. Additionally, approximately ten undergraduate students supported through other funding sources participated in research activities synergistic with the goals of this project, receiving essential mentorship from the personnel supported by this project.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Statistical Validation of Multiple Related Data Sets—Case Study Using Interstellar Boundary Explorer Satellite Data

Abstract Space scientists often face the question of whether data collected by different instruments are measurements of the same source population. This paper proposes a statistical validation method for evaluating the agreement between such related data sets. It offers a detailed case study focused on validating a new data set from the Interstellar Boundary Explorer (IBEX) mission, which serves as a practical how-to guide for similar analyses. Since 2008, the IBEX satellite has been gathering data on heliospheric energetic neutral atoms (ENAs) while being exposed to various sources of background noise, such as cosmic rays and solar energetic particles. The IBEX mission initially released only a qualified triple-coincidence (qABC) data product, which was designed to provide observations of ENAs free of background contamination. Further measurements revealed that the qABC data were in fact susceptible to contamination, having relatively low ENA counts and high background rates. To mitigate this issue, the mission team recently considered releasing a certain qualified double-coincidence (qBC) data product, which has roughly twice the detection rate of the qABC data product. This paper presents a simulation-based validation of the new qBC data product against the already-released qABC data product. The results show that the qBCs can plausibly be said to be measuring the same source population as the qABCs up to an average absolute deviation of 3.6%. Visual diagnostics provide additional confirmation of source rate coherence across data products. The framework introduced here is general and can be applied to other validation problems both within and outside the field of space physics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Impacts of PV Module Connector Failures on Cost and Performance of Utility Scale Photovoltaic Systems

The reliability, cost and performance of electrical connectors are a concern in all types of electrical systems, and demands on connectors used on photovoltaic (PV) systems include that connectors maintain electrical conductivity and physical strength, endure ultraviolet sunlight and high ambient temperature, and resist moisture and chemical intrusion over a very long (>25 year) performance period. Connector failures increase operation and maintenance (O&M) costs and reduce plant production, but connector failure can also cause safety and liability problems, which are of greater concern. This work results from a three-year collaboration between Sandia National Laboratories (SNL), the Electric Power Research Institute (EPRI), and the National Renewable Energy Laboratory (NREL) and funded by the U.S. Department of Energy (DOE) Solar Energy Technology Office (SETO) under Agreements #39035 and #38531 "Connector Reliability Across the US Solar Sector." a multi-pronged investigation of PV connector health across the US (see https://energy.sandia.gov/pvconnectors/). This report presents derivation of a Techno-Economic Analysis (TEA) that models failure modes and frequencies (how often failure occurs), estimates O&M costs and lost production associated with connector failures, and then calculates the effect that PV module connectors can have on Levelized Cost of Energy (LCOE). The model is informed with initial data from quantitative assessment of failure rates, root causes and mechanisms, in-situ diagnostics and data collection, lab-based forensics, and interviews with PV connector manufacturers and plant operators. SNL conducted site inspections at multiple utility-scale sites in different climates and subjected field samples of new, used, and degraded connectors to visual and electrical characterization. EPRI conducted metallurgical analysis of the pin and sleeve conductors to study failure-induced morphological and compositional changes. There is in general a shortage of statistically valid data, but data from PVROM database maintained by SNL was sufficient to ascertain failure rates and lost production as well as provide qualitative insight in its curated maintenance records. This report details the structure of the mathematical model but the sources of data to inform the model will continue to evolve. Analysis of a 100 MW PV plant is provided as an example of the use of the model, with results indicating that connectors are responsible for Annualized O&M Costs of $\$$71,933/year; Annualized Unit O&M Costs of $\$$0.72/kW/year; that a Reserve Account of $\$$187,220 should be available to fund repairs related to connectors; that connectors add $\$$1,494,004 to the Net Present Value of the O&M Costs (project life); and that O&M related to connectors adds about $\$$0.00088/kWh to the Levelized Cost of Energy. The impact of this model is to provide a tool to make the US solar sector more robust by quantifying and monetizing the reliability risks to utility-scale PV systems posed by poorly installed, mismatched and/or poorly designed and manufactured connectors. The TEA provides a model incorporating failure statistics, O&M cost data, and lost production into a single figure of merit, informing decisions and enabling practitioners to optimize cost and performance trade-offs. Stakeholders include connector manufacturers, system designers and equipment specifiers, standards bodies, installers and O&M providers, investors and insurance underwriters. This report supports continued growth of PV predicated on assurances that properly installed and maintained PV system connectors are safe and reliable. The project team is proposing future work including accelerated testing of connectors and expanding the approach taken here to other PV system components, such as TEA for rapid shut-down devices.

14 SOLAR ENERGY↗
DOE OSTI2026

Data and scripts associated with “Moisture content modulates DOM thermodynamic regulation of oxygen consumption in drying streambed sediments”

This data package is associated with the publication “Moisture content modulates DOM thermodynamic regulation of oxygen consumption in drying streambed sediments” published in Scientific Reports (Garayburu-Caruso et al., 2026). The package contains processed data products and scripts used to quantify how drying and re-inundation of riverbed sediments influence dissolved organic matter (DOM) thermodynamic properties and their relationship with sediment oxygen (O₂) consumption across 33 stream sites in the contiguous United States. The data package contains DOM thermodynamic metrics (e.g., Gibbs free energy of carbon oxidation and thermodynamic efficiency), and O₂ consumption along with watershed-scale climate and land-cover metrics used as explanatory variables in the analyses. Underlying unprocessed and processed ultrahigh-resolution mass spectrometry data, oxygen consumption rates from laboratory moisture-manipulation experiments, within-sample environmental properties, sediment moisture content and contextual field measurements are archived separately at https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2428003 (Laan et al., 2024) and https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1923689 (Forbes et al.,2023). A preliminary version of this data package was published in February 2026 at the time of manuscript submission. It was updated in June 2026, at the time of manuscript acceptance, to include the finalized data and additional metadata (readme, data dictionary, and file level metadata). For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. At the top level, the data package is organized into five main folders: (1) Data, (2)Figures, (3) Map, (4) GAM_Reulsts, and (5) src. The Data folder contains analysis-ready tabular files with oxygen consumption rates, DOM thermodynamic properties by site and treatment, site-level environmental variables, watershed-scale metrics, and other derived variables referenced in the manuscript. The Figures folder contains static image files associated with the main text and supplemental figures, while the Map folder includes spatial data and map-layer files used to create the sampling-location map. The GAM results folder contains the results for each of the general additive model (GAM).The src folder contains R scripts used to perform data processing, statistical analyses (including clustering, generalized additive models, and threshold analysis), and figure generation. This data package is associated with a GitHub repository found at https://github.com/WHONDRS-Hub/ECA_DOM_Thermodynamics.

Dissolved organic matter↗
DOE OSTI2024

Reconstruction and Selection of Neutrino Interactions in MicroBooNE using Deep Convolutional Neural Networks

In this document, we describe a new reconstruction workflow developed for the MicroBooNE experiment. It features the use of Deep Convolutional Neural Networks trained to recognize key structures within the data sufficient for the 3D reconstruction of neutrino interactions within the detector. As a test of the reconstruction utility, the products of the reconstruction workflow are used to select inclusive charged-current (CC) $\nu_e$ and $\nu_\mu$ interactions in both simulated and real MicroBooNE data. In simulation, our $\nu_e$ and $\nu_\mu$ selections achieve an efficiency of 57% and 68\%, respectively, with a purity of 91% and 96%, respectively. We find that these selections are competitive with the inclusive selections used for the most recent MicroBooNE LEE searches. In particular, the CC-$\nu_e$ inclusive selection efficiency improves by over 20% while also improving sample purity. As a first step in quantifying potential bias, the data and Monte Carlo expectati ons are compared for both selections using the MicroBooNE open data. Within statistical and systematic uncertainties, both the electron and muon CC-inclusive event samples agree. A comparison of the real data events chosen by our work and another reconstruction framework shows that the two analyses each identify a sizeable fraction of events the other does not. This suggests that future analyses integrating the strengths of each could lead to combined gains. This work demonstrates, for the first time on real LArTPC data, state-of-the-art neutrino interaction reconstruction centered around deep learning algorithms.

43 PARTICLE ACCELERATORS↗
DOE OSTI2026

An evaluation of air quality in major urban areas of India

Rapid economic growth and burgeoning population have contributed to enhanced levels of PM 2.5 concentrations in urban regions of India. Evaluation of ambient air quality facilitates the assessment of effectiveness of emission control measures and early identification of new sources. This study provides a comprehensive statistical analysis of PM 2.5 concentrations in key urban areas across India, including Delhi, Kolkata, Mumbai, Chennai, Hyderabad, and several regional centers. Data from 2017 to 2023 was analyzed using trend analysis, cluster analysis, principal component analysis, and geostatistical interpolation to understand spatiotemporal variations and sources. The analysis reveals significant differences in spatial distribution of PM 2.5 concentrations with high annual averages in urban regions in Indo-Gangetic plain (82–123 μg m −3 ) and relatively lower concentrations (29–46 μg m −3 ) in southern urban areas of Kerala, Tamil Nadu and Andhra Pradesh. Delhi state had the highest 24-averaged PM 2.5 concentrations (112 μg m −3 ) followed by urban regions in Uttar Pradesh, Bihar and West Bengal (94 μg m −3 ). Trend analysis from 2017 to 2023 revealed an overall 2.5% decline in site-wide PM2.5 concentrations, with the exception of Ludhiana, which exhibited a consistent annual increase of 10%. Principal component analysis (PCA) attributes 30% of the variance to wintertime emissions, 13% to biomass burning, and 18% to the regional haze in the northern Indo-Gangetic Plain. Different analyses clearly demonstrates the contribution of biomass burning to pollution in Delhi and surrounding cities. Transboundary pollution to Kolkata is likely from the highly polluted region in Indo-Gangetic Plain. Coastal cities of Mumbai and Chennai has relatively lower pollution attributed to the influence of sea breeze dilution, with mostly local contribution and some potential transport from upwind industry clusters. Hyderabad also has local contribution due to high density of vehicular traffic and local small industries. This study shows that mitigation efforts targeting clusters of regions should be undertaken to curb the high PM2.5 pollution. Policy measures should be implemented both at local and the intra-state level to address shared sources and transport of pollution.

Hysplitbacktrajectories↗
DOE OSTI2024

All-sky Search for Transient Astrophysical Neutrino Emission with 10 Years of IceCube Cascade Events

Abstract Neutrino flares in the sky are searched for in data collected by IceCube between 2011 and 2021 May. This data set contains cascade-like events originating from charged-current electron neutrino and tau neutrino interactions and all-flavor neutral-current interactions. IceCube’s previous all-sky searches for neutrino flares used data sets consisting of track-like events originating from charged-current muon neutrino interactions. The cascade data set is statistically independent of the track data sets, and while inferior in angular resolution, the low-background nature makes it competitive and complementary to previous searches. No statistically significant flare of neutrino emission was observed in an all-sky scan. Upper limits are calculated on neutrino flares of varying duration from 1 hr to 100 days. Furthermore, constraints on the contribution of these flares to the diffuse astrophysical neutrino flux are presented, showing that multiple unresolved transient sources may contribute to the diffuse astrophysical neutrino flux.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Alaska Observed Hydropower Generation

This dataset contains compiled observed hydropower generation for hydropower plants in Alaska. Data have been compiled from data provided to the Energy Information Administration by asset owners, data contained in annual reports produced by the Institute of Social and Economic Research at the University of Alaska Anchorage (Alaska Electric Power Statistics and Alaska Energy Statistics) and data provided to the Federal Energy Regulatory Commission by asset owners. This dataset provides available generation data from all sources in monthly and annual files, with quality flags, and generation data identifying the highest quality source in monthly and annual files.

hydropower datasets↗
DOE OSTI2026

Alaska Observed Hydropower Generation

This dataset contains compiled observed hydropower generation for hydropower plants in Alaska. Data have been compiled from data provided to the Energy Information Administration by asset owners, data contained in annual reports produced by the Institute of Social and Economic Research at the University of Alaska Anchorage (Alaska Electric Power Statistics and Alaska Energy Statistics) and data provided to the Federal Energy Regulatory Commission by asset owners. This dataset provides available generation data from all sources in monthly and annual files, with quality flags, and generation data identifying the highest quality source in monthly and annual files.

Broman, Daniel [Pacific Northwest National Laborat↗
DOE OSTI2024

Verification, Validation, and Calibration Through a Causal Lens

While typical validation and verification approaches focus on identifying the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on identifying causal relationships between data elements. Statistical and machine-learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between data sets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify, and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles, it is known as a directed acyclic graph. A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts can identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗