Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Design verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Requirements Description of DASSH-F

This report reviews the modeling and simulation capabilities of Argonne National Laboratory’s DASSH code that is used in present reactor analysis activities. These capabilities will be used to establish the set of verification tasks necessary to verify DASSH for use on commercial projects. A similar approach was taken for the PERSENT, REBUS and DIF3D software packages. The DASSH program is a thermal analysis code designed to rapidly allow a reactor design engineer to obtain flow rates requirements that satisfy peak temperature constraints in the domain. DASSH is a follow-on development to the SE2-ANL software and SUPERENERGY-2 software that it is based upon. DASSH was designed to account for both neutron and gamma heating and is inherently connected to the GAMSOR part of the ARC suite of fast reactor analysis software. SE2-ANL is a developed piece of software from the 1980s while DASSH is a modern implementation with notable improvements in geometry handling. The most important upgrade of DASSH relative to SE2-ANL is that it can analyze multiple time points in a single run where SE2-ANL can only treat a single time point. This allows the user to understand the impact of and search the flow distribution for the entire operational period of a reactor design considering pressure drop, peak coolant and fuel temperatures, and thermal striping. DASSH has three input paths that have to be verified. The first input path builds the geometry and power distribution based upon the DIF3D model but ignores the gamma heating aspects of the problem. The second input path also builds the geometry from the DIF3D model but it takes the neutron and gamma heating distributions from GAMSOR. The third input path is to take the geometry and power distribution directly from user input (i.e. not coupled to DIF3D or GAMSOR). DASSH also has many built in correlations for material properties along with a user defined specification of the fuel, structure, and coolant properties. There are correlations for flow split, mixing, pressure drop, and heat transfer coefficients (subchannel rather than a direct methodology). In total, verification of DASSH will require an extensive testing to cover all possible user features of the software.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Band-Selective Spin-Charge Separation across the Charge Density Wave Transition in Quasi-1D NbSe 3

Non-Fermi liquid (non-FL) phase is a pivotal enigma in understanding intriguing quantum phases in strongly correlated systems, such as high-temperature superconductivity. Tomonaga-Luttinger liquid (TLL) theory, designed for one-dimensional (1D) systems, serves as one of the microscopic frameworks that elucidates non-FL behavior. Despite its theoretical concreteness, comprehensive experimental verification has remained incomplete. In particular, addressing the persistence of the TLL nature within ordered phases, such as charge density wave (CDW), has posed a significant challenge. We report the observation of TLL characteristics across the CDW transitions in a quasi-1D material NbSe 3 using angle-resolved photoemission spectroscopy. Spin-charge separation, a disentanglement of spin and charge degrees of freedom of electrons, is clearly observed within the band, accompanied by the suppression of spectral weight following a power-law behavior with anticipated temperature scaling. Surprisingly, the spin-charge separation persists even below the CDW transition temperatures, indicating the validity of the TLL nature in the CDW phase. In conclusion, our findings offer a unique opportunity to explore the interplay between the TLL and ordered phases, which could be connected to the interplay between non-FL and ordered phases in strongly correlated systems.

1-dimensional systems↗
DOE OSTI2022

Supplemental material for: Verification, validation, and results of an approximate model for the stress of a Tokamak toroidal field coil at the inboard midplane

This is the supplemental material for the manuscript "Verification, validation, and results of an approximate model for the stress of a Tokamak toroidal field coil at the inboard midplane" submitted to Fusion Engineering and Design. This material includes PDF writeups of the derivations of the axisymmetric extended plane strain model, the elastic properties smearing model, and 20+ MATLAB scripts and functions which implement the model and generate the figures in the paper.

Swanson, CPS↗
DOE OSTI2023

Evaluation of the first version of the new RFPI system dedicated to PIP-II project accelerator

The Low-Level Radio Frequency (LLRF) control system is one of the most critical superconducting linac infrastructures responsible for the parameters of the beam acceleration. The LLRF system mainly focuses on the electromagnetic field parameters inside the cavity. While it incorporates fast feedback algorithms to optimize energy transfer to the passing particle beam it does not follow other cryomodule or cavity parameter changes. The Radio Frequency Protection Interlock (RFPI) system closely monitors various factors (like cryomodule vacuum, beamline vacuum, field emission probe current level, temperature, RF signal leakage, etc). Its simple but reliable logic has to provide an instant decision about the LLRF system or high-power amplifier output signal blocking in case of safety region excitation. This contribution presents a new version of the RFPI system which logic is implemented in the FPGA chip. The initial work on the prototype of the new system design resulted in the PoC (Proof of Concept) device. The PoC offers the possibility of various protection logic configurations, input signals parameters evaluation, and modularity aspects verification. The structure and test results from the device evaluation are summarized and discussed in this contribution.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Experimental demonstration and modeling of a robotic neutron detector with spectral and directional sensitivity for treaty verification

International safeguards and arms control agreements often require labor-intensive, intrusive onsite inspections to perform verification tasks. The ability to localize a neutron source and/or characterize a neutron field may be imperative for identifying anomalies. We are interested in the role of autonomous mobile robots, which, if designed properly, may be more effective and efficient and less intrusive than their human counterparts. Toward developing such a capability, we, for this study, have constructed the N-SpecDir Bot, comprised of three boron-coated straw detectors azimuthally-distributed within a cylinder of high-density polyethylene, which is mounted on an omni-directional robotic platform. Our N-SpecDir Bot is specifically designed to provide spectral and directional sensitivity, in addition to gross counts, by utilizing the signals from the three detectors. The detection system has been extensively characterized by MCNP modeling, which has been benchmarked to experiments conducted at the Princeton Plasma Physics Laboratory. We demonstrate the spectral and directional sensitivity experimentally and in simulation, and provide a simple yet robust method for directional measurements.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2022

SULI Research Report Paper

The Intelligence and Space Research Group 4 (ISR-4) at the Los Alamos National Laboratory focuses on signal processing and developing space electronics for various national security missions. The Space and Atmospheric Burst Reporting System (SABRS) is a sensing payload designed for nuclear detonation detection. As the sensors from previous payloads become obsolete, the requirement for a novel payload design increases. SABRS Prime is the successor to the previous SABRS payloads and is scheduled for delivery in 2028. As part of the SABRS Prompt Gamma Instrument Interface Board, high frequency signals from silicon photomultiplier tubes need to be split and subsequently transmitted to the Analog to Digital Converter. Prior to developing the Space Grade Instrument Interface Board, evaluation modules with Commercial off-the-shelf (COTS) components are evaluated and tested under rigorous conditions. Verification of power levels and gain specifications are some of the important steps taken during the evaluation stage. Working on the Signal Splitting Board has enabled me to work with various schematic software applications and learn more about the processes behind designing space grade electronic components. In addition, I conducted research on efficiency improvements for space-based power systems. Changes in design approaches were found to be the most significant sources of improvement. Being able to interact with researchers and peers across various scientific disciplines has given me a better insight into my future pursuits. Working in a R&D environment has cemented my decision to work on space applications for Quantum Sensing and pursue a PhD in Electrical Engineering. The SULI internship program at the Los Alamos National Laboratory has helped me develop the skillset of a researcher, understand the rigorous processes behind developing space grade hardware, and improve my critical thinking abilities to support the development of reliable instrumentation for deep space missions.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2021

A propagation-based fault detection and discrimination method and the optimization of sensor deployment

Industrial processes can be affected by faults having a serious impact on operation when not promptly detected and diagnosed. Here in this paper, a propagation-based fault detection and discrimination(PFDD) method is proposed to develop a strategy for fault diagnosis while in the design phase of a system. The PFDD method constructs the system model using the Integrated System Fault Analysis(ISFA) technique. Based on the system model, the propagation of hardware and software faults are simulated qualitatively. Given the results of the simulation, the process by which a fault propagates can be characterized using the qualitative features of system variables including the deviation of the system variables from their expected values, the variation of the system variables over time, and the order in which each variable is influenced during the propagation of the fault. The strategy by which a fault can be detected and discriminated is defined using those features. The PFDD method supports the detection and discrimination of faults in both steady states and transient states. Based on the PFDD method, the optimization of sensor deployment in a system is discussed. A brute force algorithm is developed to examine the system’s capability at diagnosing faults and the cost of sensor deployment for all possible configurations of sensors. The optimal sensor deployment strategy can be derived accordingly. However, the brute force method is only applicable to small-scale systems due to its high computational cost. A genetic algorithm is used to optimize sensor deployment in large-scale systems. The PFDD and sensor deployment optimization methods are applied to the Experimental Breeder Reactor II (EBR-II) for verification.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Use of Grid-Forming Medium-Voltage Power Electronics Hub in a Microgrid Setting

This paper presents the application of a new design of a multiport, modular, medium-voltage power electronics hub (M3PE-HUB) in a microgrid setting. The M3PE-HUB system was modeled in a digital real-time simulator (DRTS) and integrated into the Banshee microgrid test system. This paper presents the preliminary DRTS simulation-based results of the M3PE-HUB system connected in a test microgrid system. Verification and validation of the M3PE-HUB architecture and controls in the test microgrid setting are the primary contributions of this work. The results of the operation during the islanding and resynchronization process indicate the feasibility of the proposed architecture in a microgrid setting. This paper also presents results for a system reconfiguration use case where the M3PE-HUB was used to reconfigure the system under a fault condition.

grid forming↗
DOE OSTI2023

Fiscal Year 2023 Software Quality Assurance Activities for the ARC Software

The Argonne Reactor Code (ARC) software suite [1-17] has been developed by Argonne researchers for fast reactor design and analysis since the 1970s. With the ARC software suite, a user can quickly build a model of a proposed or existing fast spectrum reactor and carry out fuel cycle, nominal thermal analysis and flow requirements, and assess, as is appropriate, whether the core design and constraint system yield an acceptable mechanical behavior. For transient reactor analysis with SAS4A [18], the ARC software suite can be used to generate reactivity coefficients and kinetics parameters at any modeled fuel cycle time point which forms part of the input to SAS4A. The ARC suite was consistently being developed until the 1990s and followed a software QA program which was an appropriate standard for the time. In the 1990s, the DOE funding to fast reactor research and development was all but eliminated and the ARC software was put into maintenance mode. In the early 2000s, the software quality assurance (SQA) program for ARC was still in place to define an official version, but by 2005 it all but was abandoned as there were insufficient staff to fill the work roles. Since 2005, there has been a considerable increase in research and design work on fast spectrum reactors. The ARC software as a whole has since been exported to many universities and commercial companies and ANL support has been given to the various projects over the years [19-23]. Further, MC 2 -3, PERSENT, and DASSH were all developed after 2005 without any adherence to a software standard. In recent time, the DOE VTR project [22] paid for verification work to be done on the ARC software as part of the goal of making it NQA-1 complaint. The VTR project was not considered the appropriate pathway to fund and maintain a SQA program for the ARC software and while software developments (DASSH) were made and several manuals were updated and software verification work was carried out, the ARC software is not NQA-1 compliant. More recently the Advanced Reactor Development Program (ARDP [23]) has funded the creation of manuals for some ARC utility programs and funded additional software verification work on DIF3D [6, 7] and MC 2 -3 [2-5] for the purpose of commercial grade dedication. Because of the VTR and ARDP projects, software verification work was completed on MC 2 -3 and DIF3D, and detailed reports were created for each piece of software, which discuss the inputs and outputs from the codes that are covered by the verification work and link various analytic, code-to-code, and hand calculation based verification work presented in the report with verification test problems provided with the software. This is a key part of the commercial grade dedication work and constitutes the bulk of the cost to get the ARC software to commercial grade. The ARC software suite is a valuable asset as a fast reactor design and analysis tool set that has been reasonably well verified and validated with various fast reactor benchmark problems and experiments over decades. Some or all of the ARC software suite has been utilized for designing the IFR [20], PGSFR [21], VTR [22], and Natrium [23] reactors and we can expect it to continue to be used for advanced fast reactor design and/or confirmatory calculation purposes in the future. Due to increased interest by commercial companies and regulatory bodies, it is becoming more important to make the ARC software suite complete and ready-to-use in terms of its SQA pedigree and commercial grade dedication needs. This report discusses the achievements made towards building a new SQA program for the ARC software and dealing with outstanding identified QA gaps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Verification, Validation, and Calibration Through a Causal Lens

While typical validation and verification approaches focus on identifying the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on identifying causal relationships between data elements. Statistical and machine-learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between data sets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify, and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles, it is known as a directed acyclic graph. A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts can identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A methodology for corrosion testing ODS steels in liquid tin under reactor conditions in HFIR

Liquid metal–based divertor concepts have promising attributes for application in fusion reactors. Liquid tin is being considered as one of those liquid metals due to its excellent thermophysical properties. However, tin is extremely corrosive to steels at elevated temperatures, and the coupled effects of corrosion with neutron irradiation for application in fusion reactors have not been quantified. Researchers at Oak Ridge National Laboratory (ORNL) have designed and irradiated a series of capsules in the High Flux Isotope Reactor (HFIR) to help address gaps in the literature with respect to coupled corrosion and neutron irradiation effects. The capsules are filled with solid tin designed to melt due to the gamma heating in HFIR and allow interaction with the specimens. The capsules contain a SiC thermometer for post-irradiation temperature verification. Five capsules were irradiated in HFIR for 10.5 days, accumulating 0.93 dpa in the FeCrAl specimens. Post-irradiation temperature verification was performed on the SiC thermometers, and temperatures were observed to be higher than what was expected from thermal models. An explanation for the model underprediction is given herein. In conclusion, the capsules described within this article are the first to irradiate liquid metal within the HFIR flux trap and demonstrated the safety basis for continued liquid coolant studies in HFIR.

FeCrAl↗
DOE OSTI2021

Advance Reactor Operational Technology Architecture Categorization

Seven generation III+ and generation IV nuclear reactor types, based on twelve reactor concepts surveyed, are examined using functional decomposition to extract relevant operational technology (OT) architecture information. This information is compared to existing nuclear power plants (NPPs) OT architectures to highlight novel and emergent cyber risks associated with next generation NPPs. These insights can help inform operational technology architecture requirements that will be unique to a given reactor type. Next generation NPPs have streamlined OT architectures relative to the current generation II commercial NPP fleet. Overall, without compensatory measures that provide sufficient and efficient cybersecurity controls, next generation NPPs will have increased cyber risk. Verification and validation of cyber-physical testbeds and cyber risk assessment methodologies may be an important next step to reduce cyber risk in the OT architecture design and testing phase. Coordination with safety requirements can result in OT architecture design being an iterative process.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Qualification of SAS4A/SASSYS-1 for Sodium-Cooled Fast Reactor Authorization and Licensing (Final CRADA Report)

This project focuses on assessment and qualification of the SAS4A/SASSYS-1 advanced reactor safety analysis code and its capacity to fulfill requirements associated with license/authorization of a sodium-cooled fast reactor (SFR). Specific objectives of this proposal include: development of a formal documentation requirements matrix that complies with a vendor’s NQA-1 program to identify needs for qualification documentation; completion of software methods qualification gap analysis to identify and close gaps in the verification and validation basis of the software; generation of an Acceptance Test Report to confirm compliance with the software requirements specification; and closure of gaps in Software Design Description (SDD) to assist with the eventual generation of a Modeling Licensing Topic Report (LTR).

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Optimization-Based Resiliency Verification in Microgrids via Maximal Adversarial Set Characterization

Critical energy infrastructures are increasingly relying on advanced sensing and control technologies for efficient and optimal utilization of flexible energy resources. Algorithmic procedures are needed to ensure that such systems are designed to be resilient to a wide range of cyber-physical adversarial events. This paper provides a robust optimization framework to quantify the largest adversarial perturbation that a system can accommodate without violating pre-specified resiliency metrics. We formulate the maximal adversarial set characterization as a bi-level optimization problem which is solved via Lagrangian relaxations.We illustrate the proposed algorithm on an islanded microgrid example: a modified IEEE 123-node feeder with distributed energy resources. Simulations are carried out to characterize the tolerable adversarial perturbations for varying levels of available flexibility (energy reserves).

Nazir, Mohammad Nawaf↗
DOE OSTI2020

Economic Model for Estimation of GDP Losses in the MACCS Offsite Consequence Analysis Code

The MACCS (MELCOR Accident Consequence Code System) code is the U.S. Nuclear Regulatory Commission (NRC) tool used to perform probabilistic health and economic consequence assessments for atmospheric releases of radionuclides. It is also used by international organizations, both reactor owners and regulators. It is intended and most commonly used for hypothetical accidents that could potentially occur in the future rather than to evaluate past accidents or to provide emergency response during an ongoing accident. It is designed to support probabilistic risk and consequence analyses and is used by the NRC, U.S. nuclear licensees, the Department of Energy, and international vendors, licensees, and regulators. This report describes the modeling framework, implementation, verification, and benchmarking of a GDP-based model for economic losses that has recently been developed as an alternative to the original cost-based economic loss model in MACCS. The GDP-based model has its roots in a code developed by Sandia National Laboratories for the Department of Homeland Security to estimate short-term losses from natural and manmade accidents, called the Regional Economic Accounting analysis tool (REAcct). This model was adapted and modified for MACCS and is now called the Regional Disruption Economic Impact Model (RDEIM). It is based on input-output theory, which is widely used in economic modeling. It accounts for direct losses to a disrupted region affected by an accident, indirect losses to the national economy due to disruption of the supply chain, and induced losses from reduced spending by displaced workers. RDEIM differs from REAcct in its treatment and estimation of indirect loss multipliers, elimination of double counting associated with inter-industry trade in the affected area, and that it is designed to be used to estimate impacts for extended periods that can occur from a major nuclear reactor accident, such as the one that occurred at the Fukushima Daiichi site in Japan. Most input-output models do not account for economic adaptation and recovery, and in this regard RDEIM differs from its parent, REAcct, because it allows for a user-definable national recovery period. Implementation of a recovery period was one of several recommendations made by an independent peer review panel to ensure that RDEIM is state-of-practice. For this and several other reasons, RDEIM differs from REAcct. Both the original and the RDEIM economic loss models account for costs from evacuation and relocation, decontamination, depreciation, and condemnation. Where the original model accounts for an expected rate of return, based on the value of property, that is lost during interdiction, the RDEIM model instead accounts for losses of GDP based on the industrial sectors located within a county. The original model includes costs for disposal of crops and milk that the RDEIM model currently does not, but these costs tend to contribute insignificantly to the overall losses. This document discusses three verification exercises to demonstrate that the RDEIM model is implemented correctly in MACCS. It also describes a benchmark study at five nuclear power plants chosen to represent the spectrum of U.S. commercial sites. The benchmarks provide perspective on the expected differences between the RDEIM and the original cost-based economic loss models. The RDEIM model is shown to consistently predict larger losses than the original model, probably in part because it accounts for national losses by including indirect and induced losses; whereas, the original model only accounts for regional losses. Nonetheless, the RDEIM model predicts losses that are remarkably consistent with the original cost-based model, differing by 16% at most for the five sites combined with three source terms considered in this benchmark.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Demonstration and Verification of Thermo-Mechanical Bowing in a Limited Free-Bow SFR Concept Using MOOSE

Core bowing due to thermal gradients and irradiation induced swelling and creep introduces significant reactivity feedback effects in liquid metal-cooled fast spectrum reactors. During startup, normal operations, and transient events, differential thermal and flux gradients cause expansion, creep, and swelling, which results in bowing in preferential directions depending on local material properties and load-pad and restraint-ring design. The bowing phenomenon produces negative reactivity during accident conditions provided the restraint system has been properly designed to optimally guide the deformation outwards in the active core region. Under the Department of Energy Nuclear Energy Advanced Modeling and Simulation program, a Multiphysics Object Oriented Simulation Environment (MOOSE)-based multiphysics approach to model core bowing is being developed. The present work expands on previous modeling of simpler International Atomic Energy Agency (IAEA) verification problems with these tools by modeling IAEA Verification Problem 4, which involves a symmetric sector of a reactor core with ducted assemblies undergoing differential thermal expansion due to thermal gradients bowing outward with duct-to-duct contact. This model is available on the National Reactor Innovation Center Virtual Test Bed repository. This example was verified against benchmark participant results, which includes bowing deformation evaluation and duct-to-duct interactions at load pads with mechanical contact.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

A process to verify numerical models for seismic fluid-structure interaction in advanced reactor vessels

Seismic design and qualification of a liquid-filled advanced nuclear reactor will have to account for fluid-structure interaction (FSI). Interaction between the tank, internal components, and contained liquid will rely on analysis of numerical models that must be verified and validated. Here this study demonstrates a verification process for models of a base-supported cylindrical tank by comparing numerical predictions and analytical solutions. The numerical models are consistent with the assumptions made to derive analytical solutions, namely, either a rigid or a linear elastic tank, ideal fluid, and small-amplitude, unidirectional, horizontal inputs. One software platform is used to illustrate the process. Seismic FSI analysis is performed using the Arbitrary Lagrangian-Eulerian (ALE) and Incompressible Computational Fluid Dynamics (ICFD) solvers in LS-DYNA. Reported responses are those used for design, including hydrodynamic pressures on the tank wall, shear forces and moments at the tank base, and wave heights of the contained liquid. The accuracy of the numerical results is discussed. The numerical models are verified for calculating the pressures on the tank wall and reactions at its base. Accurate simulation of wave action is challenging for both solvers. Recommendations for modeling, code development, and steps for verification are provided. Although focused on reactor vessels and one software platform, the verification process described herein is broadly applicable to liquid-filled vessels and other finite element codes.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

VALIDATION, VERIFICATION, AND CALIBRATION THROUGH A CAUSAL LENS

This paper presents an alternative method based on causal inference to perform validation, verification, and calibration of simulation models. While classical validation and verification approaches focus on the identification of the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on the identification of causal relationships between data elements. Statistical and machine learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between datasets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, then the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles it is known as a directed acyclic graph (DAG). A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and from experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts have a means to identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗