SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

All sources NASA NTRS DOE OSTI

NASA NTRS2016

Crew Exploration Vehicle (CEV) (Orion) Occupant Protection

The purpose of this study was to determine the similarity between the response of the THUMS model and the Hybrid III Anthropometric Test Device (ATD) given existing Wright-Patterson (WP) sled tests. There were four tests selected for this comparison with frontal, spinal, rear, and lateral loading. The THUMS was placed in a sled configuration that replicated the WP configuration and the recorded seat acceleration for each test was applied to model seat. Once the modeling simulations were complete, they were compared to the WP results using two methods. The first was a visual inspection of the sled test videos compared to the THUMS d3plot files. This comparison resulted in an assessment of the overall kinematics of the two results. The other comparison was a comparison of the plotted data recorded for both tests. The metrics selected for comparison were seat acceleration, belt forces, head acceleration and chest acceleration. These metrics were recorded in all WP tests and were outputs of the THUMS model. Once the comparison of the THUMS to the WP tests was complete, the THUMS model output was also examined for possible injuries in these scenarios. These outputs included metrics for injury risk to the head, neck, thorax, lumbar spine and lower extremities. The metrics to evaluate head response were peak head acceleration, HIC15, and HIC36. For the neck, N (sub ij) was calculated. The thorax response was evaluated with peak chest acceleration, the Combined Thoracic Index (CTI), sternal deflection, chest deflection, and chest acceleration- 3 ms clip. The lumbar spine response was evaluated with lumbar spine force. Finally the lower extremity response was evaluated by femur and tibia force. The results of the simulation comparisons indicate the THUMS model had a similar response to the Hybrid III dummy given the same input. The primary difference seen between the two was a more flexible response of the THUMS compared to the Hybrid III. This flexibility was most pronounced in the neck flexion, shoulder deflection and chest deflection. Due to the flexibility of the THUMS, the resulting head and chest accelerations tended to lag the Hybrid III acceleration trace and have a lower peak value. The results of the injury metric comparison identified possible injury trends between simulations. Risk of head injury was highest for the lateral simulations. The risk of chest injury was highest for the rear impact. However, neck injury risk was approximately the same for all simulations. The injury metric value for lumbar spine force was highest for the spinal impact. The leg forces were highest for the rear and lateral impacts. The results of this comparison indicate the THUMS model performs in a similar manner as the Hybrid III ATD. The differences in the responses of model and the ATD are primarily due to the flexibility of the THUMS. This flexibility of the THUMS would be a more human like response. Based on the similarity between the two models, the THUMS should be used in further testing to assess risk of injury to the occupant.

Currie-Gregg, Nancy J.↗

NASA NTRS2014

Development of Super Ensemble-Based Aviation Turbulence Guidance (SEATG) for Air Traffic Management

A new method for forecasting turbulence is developed and evaluated using the high resolution weather model and in situ turbulence observations from commercial aircraft. The new method is an ensemble of various turbulence metrics from multiple time-lagged ensemble forecasts created using a sequence of four procedures. These include weather modeling, calculation of turbulence metrics, mapping the metrics into a common turbulence-scale, and production of final forecast. The new method uses similar methodology as current operational turbulence forecast with three improvements. First, it uses a higher resolution ((delta)x = 3 km) weather model to capture cloud resolving scale phenomena. Second, it computes the metrics for multiple forecasts that are combined at the same valid time resulting in a time-lagged ensemble of multiple turbulence metrics. Finally, it provides both deterministic and probabilistic turbulence forecasts. Results show the new forecasts match well with observed radar reflectivity along a surface front as well as convectively induced turbulence outside the clouds on research period. Overall performance skill of the new turbulence forecast compared with the observed EDR data during the research period is superior to any single turbulence metric. The probabilistic turbulence forecast is used in an example air traffic management application for creating a wind-optimal route considering turbulence information. The wind-optimal route passing through areas of 50% potential for moderate-or-greater turbulence and the lateral turbulence avoidance routes starting from three different waypoints along the wind-optimal route from Los Angeles international airport to John F. Kennedy international airport are calculated using different turbulence forecasts. This example shows additional flight time is required to avoid potential turbulence encounters.

modeling↗

NASA NTRS2014

Investigating Surveillance Performance for UAS Detect-and-Avoid Systems

Most unmanned aircraft systems will be required to be equipped with a detect-and-avoid system that is capable of maintaining appropriate separation from other aircraft. One of the critical components of detect-and-avoid systems is a surveillance system that identifies potential threat aircraft in real time and tracks these aircraft so that their future trajectories may be used to predict conflicts. The performance of the detect-and-avoid system generally depends on technical parameters of the surveillance system, such as the surveillance range. The quantitative requirements for detect-and-avoid systems will be determined to meet safety metrics for the operation of unmanned aircraft systems in the National Airspace System. This study employs a sensor model comprised of the surveillance range, and horizontal and vertical fields of regard that mainly characterize the overall performance of a surveillance system. In this study, potential metrics for evaluating the performance of a surveillance system were investigated through fast-time simulation with a traffic scenario that included both proposed unmanned aircraft flights and historical visual flight rule aircraft tracks. Using the simulation results, an overall analysis of encounter geometry highlights the encounter characteristics that relate surveillance parameters to safety metrics and detect-and-avoid system performance. Then, given several candidate surveillance volumes, performance and safety metrics are derived; these metrics include the ratio of undetected and late-detected violations and the time to violation at first detection. These example metrics demonstrate the utility of the database of encounters created in this work, a database which will be useful in the derivation of required detect-and-avoid surveillance system requirements.

Lee, Seung Man↗

NASA NTRS2015

Simulation and Analysis of Technology and Operational Procedures to Reduce the Combined Effects of Emissions and Contrails

The development and evaluation of concepts and technology to support future air traffic management systems require a hierarchy of models ranging from real-time simulations to extensive field evaluations. Air traffic simulation models such as Airspace Concept Evaluation System, Center Tracon Automation System, Future Air traffic management Concept Evaluation Tool and others are used to design air traffic systems balancing the conflicting objectives of maximizing safety, meeting future demands for airports and airspace and increase efficiency of traffic flows in the presence of uncertain weather. The impact of aviation emissions and contrails on climate imposes another constraint on the design of aircraft and aviation operations. The understanding of the complex interaction between physical climate system, the carbon and other greenhouse gas emissions and aviation activity can be improved by the development of integrated assessment models that include emission and climate models together with air traffic simulations. The modeling of aircraft emissions and their interaction with each other to change the concentration levels of different gasses in the atmosphere and the resulting impact of the radiative forcing on the equilibrium of the Earth's atmosphere is complex and requires the use of coupled atmosphere-ocean general circulation models together with three-dimensional models of carbon cycle and chemistry of other non-CO2 greenhouse gases. These models are computationally intensive and unsuitable for studies involving the generation of multiple scenarios. Simple emission and climate models, based on the input-output relations of linear systems, capture the fundamental emission to climate impact behavior by careful selection of key variables and their dynamics. The impact of various greenhouse gases depends on the total concentration, effect per unit change in atmospheric concentration and the spatial distribution of the gas. All these quantities are influenced by the lifetime of the gas. The impact of a greenhouse gas depends on the interval of assessment, which may vary from a few decades to a few centuries. Climate metrics are aimed at providing a common scale to compare different greenhouse gases. If the metrics are to be used as a tool in developing and evaluating aviation operations, they should be transparent and easy to apply. Global Warming Potential and Aggregate Global Temperature Potential are some of the commonly used metrics. This paper integrates a national-level air traffic simulation and optimization capability with simple climate models and carbon cycle models, and climate metrics to assess the impact of aviation on climate. The capability brings together metrics, which are useful in aviation operations together with metrics used in climate studies. The capability can be used to make trade-offs between extra fuel cost and reduction in climate impact. There is considerable uncertainty in our understanding of the radiative forcing associated with emissions and contrails. The parameters in the simulation can be used to evaluate the effect of various uncertainties in emission models and contrails. It can also be used to evaluate the impact of different decision horizons. Alternatively, the optimization results from the simulation can be used as inputs to other tools that monetize global climate impacts like the FAA's Aviation Environmental Portfolio Management Tool for Impacts.

impact of aviation on the environment↗

NASA NTRSDate not supplied

Wildfire Emergency Response Hazard Extraction and Analysis of Trends (HEAT) through Natural Language Processing and Time Series

A methodology for Hazard Extraction and Analysis of Trends (HEAT) is proposed and conducted on a data set of wildfire incident response forms, known as ICS-209-PLUS.The HEAT processes: (1) extract a set of hazards from a data set, (2) calculate hazard-relevant metrics in a primary analysis, (3) analyze trends over time in metrics using timeseries, and (4) examine potential explanations for metric trends using a secondary analysis. Hazards are extracted from narrative data in the ICS-209-PLUS based on a framework previously developed by the authors, using natural language processing. Metrics examined for each hazard include operational time to occurrence, rate of occurrence, frequency, and severity. Primary results include a taxonomy of hazards present in the data set with relevant quantitative metrics. The most frequent hazards identified are environmental and include hazardous terrain. Most hazards occur on average between 35-55% containment. Incidents with hazards tend to have a higher average severity score when compared to the average score for all incidents. Time series of the metrics and relevant predictors, including fire characteristics, fire intensity, and operations, are created to facilitate further analysis. Secondary results used to determine which factors best predict hazard frequency include a correlation matrix and regression analysis. These findings are relevant to safety for current, as well as emerging wildfire operations, and are an exploratory first step in developing historical data-driven risk assessment models.

Sequoia R. Andrade↗

NASA NTRS2021

Vegetation structural complexity and biodiversity in the Great Smoky Mountains

Vegetation structural complexity and biodiversity tend to be positively correlated, but understanding of this relationship is limited in part by structural metrics tending to quantify only horizontal or vertical variation, and that do not reflect internal structure. We developed new metrics for quantifying internal vegetation structural complexity using terrestrial LiDAR scanning and applied them to 12 NEON forest plots across an elevational gradient in Great Smoky Mountains National Park, USA. We asked (1) How do our newly developed structure metrics compare to traditional metrics? (2) How does forest structure vary with elevation in a high-biodiversity, high topographic complexity region? (3) How do forest structural metrics vary in the strength of their relationships with vascular plant biodiversity? Our new measures of canopy density (Depth) and structural complexity (σDepth), and their canopy height-normalized counterparts, were sensitive to structural variations and effectively summarized horizontal and vertical dimensions of structural complexity. Forest structure varied widely across plots spanning the elevational range of GRSM, with taller, more structurally complex forests at lower elevation. Vascular plant biodiversity was negatively correlated with elevation and more strongly positively correlated with vegetation structure variables. The strong correlations we observed between canopy structural complexity and biodiversity suggest that structural complexity metrics could be used to assay plant biodiversity over large areas in concert with airborne and spaceborne platforms.

Jonathan A. Walter↗

NASA NTRSDate not supplied

Comparison of Entry Descent and Landing Aerodynamic Databases with Uncertainty Quantification Developed Using Machine Learning Techniques

When developing the aerodynamic databases for use in trajectory simulations, it is important to develop a system of metrics to qualify which aerodynamic models are best to use. Since aerodynamics are just one input into trajectory simulations, the results of these simulations do not reflect on the quality of the aerodynamic database used. This means that aerodynamic database comparisons must be done offline. While traditional metrics that focus on mean/nominal predictions are a good first step, more robust estimates of the prediction interval become important as more focused uncertainty models are developed. We explore the limitations of evaluating aerodynamic models based purely on nominal-centered response surfaces. Before elaborating and evaluating metrics based on distributed models, the value of evaluating prediction interval and confidence interval are discussed to conclude that prediction intervals are more relevant to the use of trajectory analysis. Several metrics to evaluate the prediction interval are introduced with a focus on the standard calibration metric. Finally, we compare candidate models using both mean and distributed metrics. A finalized candidate model developed using state of the art machine learning methods is compared to a baseline model developed using traditional aerodynamic database modeling techniques.

Aerodynamic Database↗

NASA NTRS2025

Uncertainty Quantification for Empirical X-59 Sonic Boom Loudness Levels

Estimates of the total uncertainty for empirically determined loudness levels are documented when GRS (Ground Recording System) noise monitors are used to record X 59 sonic boom waveforms. The total uncertainty is characterized by combining nine different sources of uncertainty that may affect the apparent gain of the measurement chain. These uncertainty estimates are presented as expected measurement error relative to the true loudness level, and separate error estimates are provided for eight different noise metrics in which NASA has interest. The behavior of the Perceived Level (PL) metric is studied within the report body, while the total uncertainties for the seven other noise metrics are summarized in appendices for brevity. The effects of four sources of uncertainty are estimated simply from information found on hardware specification sheets provided by the manufacturer. However, mock acoustic recordings are created to estimate the effects of other sources because those effects are expected to induce spectral coloration, so they may vary with noise metric type and sound level. These sources are not well modeled by simple gain adjustments. Importantly, measurement error is computable when processing mock recordings since the true levels are knowable, which is not the case when processing data recorded in the field. Specifically, the true levels are knowable because the components of the mock recordings are separable – e.g., loudness levels of booms can be computed with or without superimposed background noise. Mock acoustic recordings also have the benefit of allowing analysis of sonic booms from vehicles that are not yet flying, like the X-59, since the mock recordings are created by combining vehicle-specific predicted waveforms with other audio sources. The estimates of total measurement error are documented as a function of the signal-to-noise ratio (SNR) of the loudness level, where the corrected SNR is computed while accounting for the effects of the method that is used to correct for background noise contamination when computing the noise metric values. The corrected SNR calculations used here can be applied to both mock recordings and in-field measurements, so the uncertainty of in-field recordings can be found using pre-computed lookup tables that identify the relationship between metric type, corrected SNR, and the expected measurement error.

Sonic Boom↗

DOE OSTI2020

Quantifying Technical Diversity Benefits of Wind as a Distributed Energy Resource

Distributed energy resources are increasingly used in power distribution systems and microgrids to supply cost competitive power, improve resilience, and provide a host of grid services. Diversifying variable renewable resources (e.g., by combining wind with solar) can increase energy usage efficiency and improve system resilience. However, when grid optimization and resilience studies consider multiple renewable resources, diversity benefits are usually captured only implicitly in the results of location-specific economic optimization. In this paper, metrics are introduced to express the technical value of resource diversity independent of jurisdiction-specific market structures. Specifically, marginal energy usage efficiency metrics are developed to quantify the ability of new distributed generation to produce useful energy and an incremental sustainable ride through metric is developed to express improvement in grid outage ride through capability. While there are economic implications for each of these metrics, the metrics themselves are technically-driven and could be used as components of a technical figure of merit that could be used to inform policy actions, such as development of resource-specific incentives and time-of-use tariff designs. Each of these metrics is demonstrated using balance-of-energy simulations that highlight the benefits of improving resource diversity.

Reiman, Andrew P.↗

DOE OSTI2021

Multifaceted radiation detection and classification system

A system identifying a source of radiation is provided. The system includes a radiation source detector and a radiation source identifier. The radiation source detector receives measurements of radiation; for one or more sources, generates a detection metric indicating whether that source is present in the measurements; and evaluates the detection metrics to detect whether a source is present in the measurements. When the presence of a source in the measurements is detected, the radiation source identifier for one or more sources, generates an identification metric indicating whether that source is present in the measurements; generates a null-hypothesis metric indicating whether no source is present in the measurements; evaluates the one or more identification metrics and the null-hypothesis metric to identify the source, if any, that is present in the measurements.

Labov, Simon E.↗

DOE OSTI2022

Multifaceted radiation detection and classification system

Labov, Simon E.↗

DOE OSTI2022

Capsule network-based semantic segmentation model for thermal anomaly identification on building envelopes

Thermography technology is widely used to inspect thermal anomalies in building façade systems. Computer vision-based techniques provide opportunities to autonomously detect such heat anomalies to significantly improve the efficiency of decision-making for building envelope retrofitting and maintenance. Here, in this work, we propose a novel Capsule Network-based deep learning model – CapsLab – that detects and identifies thermal anomalies by semantic segmentation. CapsLab is built based on our proposed prediction-tuning capsule (PT-Capsule) layer. Different from a traditional capsule layer, which consists of part-whole transformation and capsule-routing process, the proposed layer is composed of a prediction and tuning process, which helps decreasing the number of model parameters significantly. While the applicability of traditional Capsule Networks (CapsNets) has been limited to simpler tasks and smaller datasets due to their scalability issue, we can leverage the lightweight of the proposed PT-Capsule layer, and apply it to the semantic segmentation task. In this work, we also employ our previously presented performance metric, referred to as the Anomaly Identification Metric (AIM) (Kakillioglua et al. 2021), to evaluate the segmentation outputs. Traditional performance metrics do not accurately reflect the true performance of the segmentation models in thermal anomaly identification due to the high subjectivity in the annotation process and higher overlap ratio sensitivity of the standard metrics. AIM, on the other hand, is robust to these drawbacks. Experimental results show, both qualitatively and quantitatively, that our proposed segmentation method can effectively segment the thermal anomalies. Specifically, our model provides 9.38% and 13.53% improvements over the baseline model – DeepLabV3+ – based on traditional mIoU score and the AIM score, respectively, while requiring less model parameters and less computation at the same time. In addition, the scores that the AIM metric generates better align with the scores provided by building performance experts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2021

Feedback density and causal complexity of simulation model structure

Measures of simulation model complexity generally focus on outputs; we propose measuring the complexity of a model’s causal structure to gain insight into its fundamental character. This article introduces tools for measuring causal complexity. First, we introduce a method for developing a model’s causal structure diagram, which characterises the causal interactions present in the code. Causal structure diagrams facilitate comparison of simulation models, including those from different paradigms. Next, we develop metrics for evaluating a model’s causal complexity using its causal structure diagram. We discuss cyclomatic complexity as a measure of the intricacy of causal structure and introduce two new metrics that incorporate the concept of feedback, a fundamental component of causal structure. The first new metric introduced here is feedback density, a measure of the cycle-based interconnectedness of causal structure. The second metric combines cyclomatic complexity and feedback density into a comprehensive causal complexity measure. Finally, we demonstrate these complexity metrics on simulation models from multiple paradigms and discuss potential uses and interpretations. These tools enable direct comparison of models across paradigms and provide a mechanism for measuring and discussing complexity based on a model’s fundamental assumptions and design.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Emulating climate extreme indices

We use simple pattern scaling and time-shift to emulate changes in a set of climate extreme indices under future scenarios, and evaluate the emulators' accuracy. We propose a metric for the error in emulation in the context of initial condition ensembles, to specifically characterize the role of internal variability in the emulation performance. Our metric separates systematic emulation errors from unavoidable discrepancies between emulated and target values due to internal variability. We compute the metricis at grid-point scale, and we show geographically resolved results, or aggregate them at global scale. We demonstrate the use of our error metric in the emulation of a suite of temperature and precipitation extreme indices. We test and compare simple pattern scaling and time-shift using a range of trajectories spanning targets inspired by the Paris agreement -- warming to 1.5C and 2.0C from the pre-industrial baseline -- and two of the longer-established trajectories, RCP4.5 and RCP8.5. With this suite of scenarios we can test the effects on the performance of the size of the temperature gap between emulation origin and target. We find that for most indices emulation the dominant source of discrepancy is internal variability. For at least one index, however, counting exceedances of a high temperature threshold, significant portions of the globally aggregated discrepancy and its regional pattern originate from the systematic emulation error. This error exceeds internal variability of both the target and the emulated quantities in large coherent regions at low latitudes, and the explanation can be found in the differential behavior of temperature distributions across latitudes. The metric also highlights a fundamental difference in the two methods related to the simulation of internal variability, which is dampened significantly by simple pattern scaling. This aspect is of consequence when using these methods for specific applications, where preserving variability for uncertainty quantification is deemed important. With this study we offer our metric as a diagnostic tool, facilitating the formulation of scientific hypotheses on the reasons for the error. In the meantime, we show that for many impact relevant indices by now traditional emulation techniques can be accurate within the variations unavoidably introduced by internal variability, establishing the fundamental condition for using their emulation in impact modeling.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Nuclear Science User Facilities FY 2020 Program Performance Summary

The Nuclear Science User Facilities (NSUF) first gathered and reported metric data for Fiscal Year (FY) 2014. The goal of the FY 2014 NSUF Metric Report was to establish metric measures and to provide baseline data to evaluate future performance of the NSUF. Each metric goal and objective was established through NSUF discussions with Department of Energy (DOE) Idaho Operations Office and Office of Nuclear Energy staff. This report provides the data, analysis, and conclusions for the FY 2020 metrics.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2024

Justice Underpinning Science and Technology Research (JUST-R) Offline Tool: A Tool for Guiding Researchers in Addressing Energy Justice Considerations in Early-Stage Research

The Justice Underpinning Science and Technology Research (JUST-R) Metrics Framework is a set of metrics for assessing the energy justice implications of technologies that are currently under research and development (R&D). The JUST-R Metrics Framework guides researchers through an analysis of the many facets of their research processes, from material inputs to knowledge sources, that may contribute to energy injustice both during the research period and when the technology is scaled. This JUST-R Offline Tool consists of an Excel file and PDF guide to aid researchers, engineers, and project managers in their application of the JUST-R Metrics Framework. This tool enables researchers to 1) evaluate the baseline energy justice implications of their research; 2) develop justice-oriented changes to the research process; and 3) track the implementation of proposed changes. The metrics included in the framework are sorted into five aspects of research: Team Dynamics, Sources & Inputs, Processes & Protocols, Waste & Hazards, and Results & Dissemination. The JUST-R Tool can be found here: https://www.nrel.gov/analysis/just-r.html.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2020

Nuclear Science User Facilities FY 2019 Program Performance Summary

The Nuclear Science User Facilities (NSUF) first gathered and reported metric data for Fiscal Year (FY) 2014. The goal of the FY 2014 NSUF Metric Report was to establish metric measures and to provide baseline data to evaluate future performance of the NSUF. Each metric goal and objective were established through NSUF discussions with Department of Energy (DOE) Idaho Operations Office and Office of Nuclear Energy staff. This report provides the data, analysis, and conclusions for the FY 2019 metrics.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2025

AAPM Truth‐based CT (TrueCT) reconstruction grand challenge

Background: This Special Report summarizes the 2022, AAPM grand challenge on Truth-based CT image reconstruction. Purpose: To provide an objective framework for evaluating CT reconstruction methods using virtual imaging resources consisting of a library of simulated CT projection images of a population of human models with various diseases. Methods: Two hundred unique anthropomorphic, computational models were created with varied diseases consisting of 67 emphysema, 67 lung lesions, and 66 liver lesions. The organs were modeled based on clinical CT images of real patients. The emphysematous regions were modeled using segmentations from patient CT cases in the COPDGene Phase I dataset. For the lung and liver lesion cases, 1–6 malignant lesions were created and inserted into the human models, with lesion diameters ranging from 5.6 to 21.9 mm for lung lesions and 3.9 to 14.9 mm for liver lesions. The contrast defined between the liver lesions and liver parenchyma was 82 ± 12 HU, ranging from 50 to 110 HU. Similarly, the contrast between the lung lesions and the lung parenchyma was defined as 781 ± 11 HU, ranging from 725 to 805 HU. For the emphysematous regions, the defined HU values were −950 ± 17 HU ranging from −918 to −979 HU. The developed human models were imaged with a validated CT simulator. The resulting CT sinograms were shared with the participants. The participants reconstructed CT images from the sinograms and sent back their reconstructed images. Further, the reconstructed images were then scored by comparing the results against the corresponding ground truth values. The scores included both task-generic (root mean square error [RMSE] and structural similarity matrix [SSIM]), and task-specific (detectability index [d’] and lesion volume accuracy) metrics. For the cases with multiple lesions, the measured metric was averaged across all the lesions. To combine the metrics with each other, each metric was normalized to a range of 0 to 1 per disease type, with “0” and “1” being the worst and best measured values across all cases of the disease type for all received reconstructions. Results: The True-CT challenge attracted 52 participants, out of which 5 successfully completed the challenge and submitted the requested 200 reconstructions. Across all participants and disease types, SSIM absolute values ranged from 0.22 to 0.90, RMSE from 77.6 to 490.5 HU, d’ from 0.1 to 64.6, and volume accuracy ranged from 1.2 to 753.1 mm3. The overall scores demonstrated that participant “A” had the best performance in all categories, except for the metrics of d’ for lung lesions and RMSE for liver lesions. Participant “A” had an average normalized score of 0.41 ± 0.22, 0.48 ± 0.32, and 0.42 ± 0.33 for the emphysema, lung lesion, and liver lesion cases, respectively. Conclusions: The True-CT challenge successfully enabled objective assessment of CT reconstructions with the unique advantage of access to a diverse population of diseased human models with known ground truth. This study highlights the significant potential of virtual imaging trials in objective assessment of medical imaging technologies.

60 APPLIED LIFE SCIENCES↗