Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Ground truth”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

A Blind Convolutional Deep Autoencoder for Spectral Unmixing of Hyperspectral Images Over Waterbodies

Harmful algal blooms have dangerous repercussions for biodiversity, the ecosystem, and public health. Automatic identification based on remote sensing hyperspectral image analysis provides a valuable mechanism for extracting the spectral signatures of harmful algal blooms and their respective percentage in a region of interest. This paper proposes a new model called a non-symmetrical autoencoder for spectral unmixing to perform endmember extraction and fractional abundance estimation. The model is assessed in benchmark datasets, such as Jasper Ridge and Samson. Additionally, a case study of the HSI2 image acquired by NASA over Lake Erie in 2017 is conducted for extracting optical water types. The results using the proposed model for the benchmark datasets improve unmixing performance, as indicated by the spectral angle distance compared to five baseline algorithms. Improved results were obtained for various metrics. In the Samson dataset, the proposed model outperformed other methods for water (0.060) and soil (0.025) endmember extraction. Moreover, the proposed method exhibited superior performance in terms of mean spectral angle distance compared to the other five baseline algorithms. The non-symmetrical autoencoder for the spectral unmixing approach achieved better results for abundance map estimation, with a root mean square error of 0.091 for water and 0.187 for soil, compared to the ground truth. For the Jasper Ridge dataset, the non-symmetrical autoencoder for the spectral unmixing model excelled in the tree (0.039) and road (0.068) endmember extraction and also demonstrated improved results for water abundance maps (0.1121). The proposed model can identify the presence of chlorophyll-a in waterbodies. Chlorophyll-a is an essential indicator of the presence of the different concentrations of macrophytes and cyanobacteria. The non-symmetrical autoencoder for spectral unmixing achieves a value of 0.307 for the spectral angle distance metric compared to a reference ground truth spectral signature of chlorophyll-a. The source code for the proposed model, as implemented in this manuscript, can be found at https://github.com/EstefaniaAlfaro/autoencoder_owt_spectral.git.

hyperspectral imaging↗
NASA NTRS1992

A system for verifying models and classification maps by extraction of information from a variety of data sources

Recent updates to a geographical information system (GIS) called VICAR (Video Image Communication and Retrieval)/IBIS are described. The system is designed to handle data from many different formats (vector, raster, tabular) and many different sources (models, radar images, ground truth surveys, optical images). All the data are referenced to a single georeference plane, and average or typical values for parameters defined within a polygonal region are stored in a tabular file, called an info file. The info file format allows tracking of data in time, maintenance of links between component data sets and the georeference image, conversion of pixel values to `actual' values (e.g., radar cross-section, luminance, temperature), graph plotting, data manipulation, generation of training vectors for classification algorithms, and comparison between actual measurements and model predictions (with ground truth data as input).

Norikane, L.↗
DOE OSTI2020

Overcoming small minirhizotron datasets using transfer learning

Minirhizotron technology is widely used to study root growth and development. Yet, standard approaches for tracing roots in minirhiztron imagery is extremely tedious and time consuming. Machine learning approaches can help to automate this task. However, lack of enough annotated training data is a major limitation for the application of machine learning methods. Transfer learning is a useful technique to help with training when available datasets are limited. In this paper, we investigated the effect of pre-trained features from the massives-cale, irrelevant ImageNet dataset and a relatively moderate-scale, but relevant peanut root dataset on switchgrass root imagery segmentation applications. We compiled two minirhizotron image datasets to accomplish this study: one with 17,550 peanut root images and another with 28 switchgrass root images. Both datasets were paired with manually labeled ground truth masks. Deep neural networks based on the U-net architecture were used with different pre-trained features as initialization for automated, precise pixel-wise root segmentation in minirhizotron imagery. We observed that features pre-trained on a closely related but relatively moderate size dataset like our peanut dataset were more effective than features pre-trained on the large but unrelated ImageNet dataset. Here, we achieved high quality segmentation on peanut root dataset with 99.04% accuracy at the pixel-level and overcame errors in human-labeled ground truth masks. By applying transfer learning technique on limited switchgrass dataset with features pre-trained on peanut dataset, we obtained 99% segmentation accuracy in switchgrass imagery using only 21 images for training (fine tuning). Furthermore, the peanut pre-trained features can help the model converge faster and have much more stable performance.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Mining and Validating Social Media Data for COVID-19–Related Human Behaviors Between January and July 2020: Infodemiology Study

Background: Health authorities can minimize the impact of an emergent infectious disease outbreak through effective and timely risk communication, which can build trust and adherence to subsequent behavioral messaging. Monitoring the psychological impacts of an outbreak, as well as public adherence to such messaging, is also important for minimizing long-term effects of an outbreak. Objective: We used social media data from Twitter to identify human behaviors relevant to COVID-19 transmission, as well as the perceived impacts of COVID-19 on individuals, as a first step toward real-time monitoring of public perceptions to inform public health communications. Methods: We developed a coding schema for 6 categories and 11 subcategories, which included both a wide number of behaviors as well codes focused on the impacts of the pandemic (eg, economic and mental health impacts). We used this to develop training data and develop supervised learning classifiers for classes with sufficient labels. Classifiers that performed adequately were applied to our remaining corpus, and temporal and geospatial trends were assessed. We compared the classified patterns to ground truth mobility data and actual COVID-19 confirmed cases to assess the signal achieved here. Results: We applied our labeling schema to approximately 7200 tweets. The worst-performing classifiers had F1 scores of only 0.18 to 0.28 when trying to identify tweets about monitoring symptoms and testing. Classifiers about social distancing, however, were much stronger, with F1 scores of 0.64 to 0.66. We applied the social distancing classifiers to over 228 million tweets. We showed temporal patterns consistent with real-world events, and we showed correlations of up to –0.5 between social distancing signals on Twitter and ground truth mobility throughout the United States. Conclusions: Behaviors discussed on Twitter are exceptionally varied. Twitter can provide useful information for parameterizing models that incorporate human behavior, as well as for informing public health communication strategies by describing awareness of and compliance with suggested behaviors.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1997

JOVE Program

Coastal reef degradation and widespread bleaching of corals, i.e. loss of pigments and/or symbiotic zooxanthellae, is increasing globally. Remote sensing has great potential for assessing the extent of reef change. However, measuring reef change from remote platforms (boats, aircraft or satellites), requires ground-truth spectral algorithms characteristic of healthy and degraded reef populations. Our laboratory experiments demonstrated, for the first time, that healthy coral colonies emit characteristic fluorescence spectra in response to excitation with short wavelength (blue or ultraviolet) light. Furthermore, following stress, e.g. increased temperature, corals lose symbiotic algae and their health is compromised. We measured changes in coral fluorescence spectra along with pigment loss in response to temperature-induced stress. Following laboratory demonstration, we tested this approach in the field. The NASA P3 aircraft with the airborne oceanographic LIDAR system was used to map transects of coral reef in Hawaii. Ground truth samples of coral; were collected and analyzed for pigment density (pigments/surface area). Fluorescence spectra along transects showed consistent chlorophyll fluorescence peaks at 685 nm from the chlorophyll of the zooxanthellae and peaks between 400 and 570 nm believed to emanate from the coral tissue. Our results suggest that remote sensing of laser-induced fluorescence represents an efficient and effective approach to monitoring the health of coral reefs. Lists of papers, conferences, proposals awarded and a summary of student involvement is included in the appendices.

Hardy, John T.↗
NASA NTRS1975

ERTS-1 DCS technical support provided by Wallops Station

Wallops Station accepted the tasks of providing ground truth to several ERTS investigators, operating a DCP repair depot, designing and building an airborne DCP Data Acquisition System, and providing aircraft underflight support for several other investigators. Additionally, the data bank is generally available for use by ERTS and other investigators that have a scientific interest in data pertaining to the Chesapeake Bay area. Working with DCS has provided a means of evaluating the system as a data collection device possibly applicable to ongoing Earth Resources Program activities in the Chesapeake Bay area as well as providing useful data and services to other ERTS investigators. The two areas of technical support provided by Wallops, ground truth stations and repair for DCPs, are briefly discussed.

Smith, R.↗
NASA NTRS1976

Wind sets from SMS images - An assessment of quality for GATE

The paper analyzes the accuracy, representativeness, and reproducibility of tracer winds in the 1974 GARP Atlantic Tropical Experiment whose data are used as ground truth. The tracer winds were generated by tracking clouds in SMS (Synchronous Meteorological Satellite) images. Data availability limits comparisons to satellite winds with ship winds at the surface and at 250 mb. Attention is focused on how accurately the cloud displacements can be measured and on the extent to which the cloud displacements represent the wind field. Operator errors in obtaining the cloud displacements are examined in a series of reproducibility tests and wind sets. Differences between proximate satellite and ship winds were all under 3 m/sec. Representativeness of cloud tracers for cumulus and cirrus level flow is found to be good within the accuracy of currently available ground truth data.

Suchman, D.↗
NASA NTRS2016

SLAPex Freeze/Thaw 2015: The First Dedicated Soil Freeze/Thaw Airborne Campaign

Soil freezing and thawing is an important process in the terrestrial water, energy, and carbon cycles, marking the change between two very different hydraulic, thermal, and biological regimes. NASA's Soil Moisture Active/Passive (SMAP) mission includes a binary freeze/thaw data product. While there have been ground-based remote sensing field measurements observing soil freeze/thaw at the point scale, and airborne campaigns that observed some frozen soil areas (e.g., BOREAS), the recently-completed SLAPex Freeze/Thaw (F/T) campaign is the first airborne campaign dedicated solely to observing frozen/thawed soil with both passive and active microwave sensors and dedicated ground truth, in order to enable detailed process-level exploration of the remote sensing signatures and in situ soil conditions. SLAPex F/T utilized the Scanning L-band Active/Passive (SLAP) instrument, an airborne simulator of SMAP developed at NASA's Goddard Space Flight Center, and was conducted near Winnipeg, Manitoba, Canada, in October/November, 2015. Future soil moisture missions are also expected to include soil freeze/thaw products, and the loss of the radar on SMAP means that airborne radar-radiometer observations like those that SLAP provides are unique assets for freeze/thaw algorithm development. This paper will present an overview of SLAPex F/T, including descriptions of the site, airborne and ground-based remote sensing, ground truth, as well as preliminary results.

BOREAS↗
DOE OSTI2024

Using AI Segmentation Models to Improve Foreign Body Detection and Triage from Ultrasound Images

Medical imaging can be a critical tool for triaging casualties in trauma situations. In remote or military medicine scenarios, triage is essential for identifying how to use limited resources or prioritize evacuation for the most serious cases. Ultrasound imaging, while portable and often available near the point of injury, can only be used for triage if images are properly acquired, interpreted, and objectively triage scored. Here, we detail how AI segmentation models can be used for improving image interpretation and objective triage evaluation for a medical application focused on foreign bodies embedded in tissues at variable distances from critical neurovascular features. Ultrasound images previously collected in a tissue phantom with or without neurovascular features were labeled with ground truth masks. These image sets were used to train two different segmentation AI frameworks: YOLOv7 and U-Net segmentation models. Overall, both approaches were successful in identifying shrapnel in the image set, with U-Net outperforming YOLOv7 for single-class segmentation. Both segmentation models were also evaluated with a more complex image set containing shrapnel, artery, vein, and nerve features. YOLOv7 obtained higher precision scores across multiple classes whereas U-Net achieved higher recall scores. Using each AI model, a triage distance metric was adapted to measure the proximity of shrapnel to the nearest neurovascular feature, with U-Net more closely mirroring the triage distances measured from ground truth labels. Overall, the segmentation AI models were successful in detecting shrapnel in ultrasound images and could allow for improved injury triage in emergency medicine scenarios.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Novel pore size-controlled, susceptibility matched, 3D-printed MRI phantoms

We report the design concept and fabrication of MRI phantoms, containing blocks of aligned microcapillaires that can be stacked into larger arrays to construct diameter distribution phantoms or fractured, to create a “powder-averaged” emulsion of randomly oriented blocks for vetting or calibrating advanced MRI methods, that is, diffusion tensor imaging, AxCaliber MRI, MAP-MRI, and multiple pulsed field gradient or double diffusion-encoded microstructure imaging methods. The goal was to create a susceptibility-matched microscopically anisotropic but macroscopically isotropic phantom with a ground truth diameter that could be used to vet advanced diffusion methods for diameter determination in fibrous tissues. Two-photon polymerization, a novel three-dimensional printing method is used to fabricate blocks of capillaries. Double diffusion encoding methods were employed and analyzed to estimate the expected MRI diameter. Susceptibility-matched microcapillary blocks or modules that can be assembled into large-scale MRI phantoms have been fabricated and measured using advanced diffusion methods, resulting in microscopic anisotropy and random orientation. This phantom can vet and calibrate various advanced MRI methods and multiple pulsed field gradient or diffusion-encoded microstructure imaging methods. We demonstrated that two double diffusion encoding methods underestimated the ground truth diameter.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS2006

The Australian National Airborne Field Experiment 2005: Soil Moisture Remote Sensing at 60 Meter Resolution and Up

Spatially-distributed soil moisture observations have applications spanning a wide range of spatial resolutions from the very local needs of individual farmers to the progressively larger areas of interest to weather forecasters, water resource managers, and global climate modelers. To date, the most promising approach for space-based remote sensing of soil moisture makes use of passive microwave emission radiometers at L-band frequencies (1-2 GHz). Several soil moisture-sensing satellites have been proposed in recent years, with the European Space Agency's Soil Moisture Ocean Salinity (SMOS) mission scheduled to be launched first in a couple years. While such a microwave-based approach has the advantage of essentially allweather operation, satellite size limits spatial resolution to 10's of km. Whether used at this native resolution or in conjunction with some type of downscaling technique to generate soil moisture estimates on a finer-scale grid, the effects of subpixel spatial variability play a critical role. The soil moisture variability is typically affected by factors such as vegetation, topography, surface roughness, and soil texture. Understanding and these factors is the key to achieving accurate soil moisture retrievals at any scale. Indeed, the ability to compensate for these factors ultimately limits the achievable spatial resolution and/or accuracy of the retrieval. Over the last 20 years, a series of airborne campaigns in the USA have supported the development of algorithms for spaceborne soil moisture retrieval. The most important observations involved imagery from passive microwave radiometers. The early campaigns proved that the retrieval worked for larger and larger footprints, up to satellite-scale footprints. These provided the solid basis for proposing the satellite missions. More recent campaigns have explored other aspects such as retrieval performance through greater amounts of vegetation. All of these campaigns featured extensive ground truth collection over a range of grid spacings, to provide a basis for examining the effects of subpixel variability. However, the native footprint size of the airborne L-band radiometers was always a few hundred meters. During the recently completed (November, 2005) National Airborne Field Experiment (NAFE) campaign in Australia, a compact L-band radiometer was deployed on a small aircraft. This new combination permitted routine observations at native resolutions as high as 60 meters, substantially finer than in previous airborne soil moisture campaigns, as well as satellite footprint areal coverage. The radiometer, the Polarimetric L-band Microwave Radiometer (PLMR) performed extremely well and operations included extensive calibration-related observations. Thus, along with the extensive fine-scale ground truth, the NAFE dataset includes all the ingredients for the first scaling studies involving very-high-native resolution soil moisture observations and the effects of vegetation, roughness, etc. A brief overview of the NAFE will be presented, then examples of the airborne observations with resolutions from 60 m to 1 km will be shown, and early results from scaling studies will be discussed.

Kim, E. J.↗
DOE OSTI2025

Experimental data for damage mechanics simulation challenge

While there are many computational approaches for simulating damage in rock and other materials, few have been ground truth tested with either known experimental data or with blind data sets. Here, in this work, we present a bench-mark laboratory data set for a damage mechanics challenge to compare computational approaches on damage evolution in brittle-ductile materials. The samples were fabricated through additive manufacturing to produce repeatable specimens designed to fail in controlled ways. The failure was induced in the samples using a 3-point bending test to produce different Modes such as Mode I and mixed Modes including I-II, I-III and I-II-III Modes to generate a calibration data set and a blind challenge data set. Data collected included spatial and temporal measurements from traditional digital load–displacement sensors, 2D digital image correlation measurement to map surface deformations, 3D X-ray microscopy to ground-truth the crack-failure geometry, and laser profilometry to capture surface roughness. The data sets are available, on a data repository, to the community to advance computational models to improve our ability to predict damage in brittle-ductile materials.

3-point bending↗
DOE OSTI2023

Real-Time, Simultaneous Soil Water Content and Meteorological Data Measurement to Support TRACER over Harris County, Texas (Field Campaign Report: Part I)

The main purpose of this project was to provide ground-truth and satellite-based soil water content data in the Houston, Texas, area to support the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s 2022 field campaign, the Tracking Aerosol Convection Interactions ExpeRiment (TRACER). This involved installing four soil monitoring stations at different locations in the Houston area, two of which were part of the ARM facility and two were ancillary. The two stations associated directly with ARM were installed alongside other facilities managed and maintained by the TRACER research team during the intensive operational period (IOP), one located at the La Porte, Texas airport and the other near Guy, Texas. Data from these stations were assimilated with similar data collected by the Harris County Flood Control District (HCFCD) to improve the spatial coverage of the real-time monitoring network. The ground-truth data were compared to the satellite-based (National Aeronautics and Space Administration [NASA]’s Soil Moisture Active Passive [SMAP] mission) data in the TRACER area of interest, and analyzed further to nowcast gridded data over the Houston area. The nowcasted, gridded data product was made available to TRACER researchers for use in climate and land-atmosphere interaction modeling that can help understand formation and persistence of convective storms in dense urban areas, and predict possible environmental events such as floods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Real-Time, Simultaneous Soil Water Content and Meteorological Data Measurement to Support TRACER over Harris County, Texas Field Campaign Report: Part II

The main purpose of this project was to provide ground-truth and satellite-based soil water content data in the Houston, Texas, area to support the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s 2022 field campaign, the Tracking Aerosol Convection Interactions ExpeRiment (TRACER). This involved installing four soil monitoring stations at different locations in the Houston area, two of which were part of the ARM facility and two were ancillary. The two stations associated directly with ARM were installed alongside other facilities managed and maintained by the TRACER research team during the intensive operational period (IOP), one located at the La Porte, Texas airport and the other located near Guy, Texas. Data from these stations were assimilated with similar data collected by the Harris County Flood Control District (HCFCD) to improve the spatial coverage of the real-time monitoring network. The ground-truth data were compared to the satellite-based (National Aeronautics and Space Administration [NASA]’s Soil Moisture Active Passive [SMAP] mission) data in the TRACER area of interest, and analyzed further to nowcast gridded data over the Houston area. The nowcasted, gridded data product was made available to TRACER researchers for use in climate and land-atmosphere interaction modeling that can help understand formation and persistence of convective storms in dense urban areas, and predict possible environmental events such as floods.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1973

Study to demonstrate the feasibility of and determine the optimum method of remote haze monitoring by satellite

The author has identified the following significant results. Ground truth data has been obtained in order to determine the amount of haze present at the time of ERTS-1 passes over the Los Angeles area. October 21, 1972 was the haziest day, that was essentially cloud free, to date. At the time of the next pass, November 8, 1972 this area was quite clear. Changes in terrain features are minimized because these were consecutive passes. Digital tapes of the ERTS-1 data from these two days are being analyzed to determine the effects of haze on the data, in order to determine the optimum method for monitoring haze from a satellite. Preliminary results indicate that a slight reduction in contrast and some distortion of spectral signatures are caused by heavy haze. This effect is thought to be sufficient to determine the presence of moderate haze without the benefit of ground truth data. It is significant to note that there is very little degradation of ERTS-1 data caused be a rather heavy haze. For many purposes the data obtained on a hazy day are just as useful as that obtained on a clear day.

Rogers, E. H.↗
DOE OSTI2021

Scalable Programming Workflows for Validation of Quantum Computers

Hybrid quantum-classical workflows have become standard methods for executing variational algorithms and other quantum simulation techniques, which are key applications for noisy intermediate scale quantum (NISQ) computers. Validating these simulations is an important task which helps gauge the progress of quantum computer development, and classical simulation can serve as a tool to this end. Both exact and more scalable approximate methods with quantifiable error bounds can be used in validation tasks where the applicable metrics include the distance from a calculable ground truth, the quality of an error model fit to data, etc. Here we present a library extension that includes methods for validation of quantum simulations based on scalable hybrid workflows executable on high performance computers. We provide examples that use approximate methods based on tensor networks and stabilizer simulators to bound the error of quantum simulations on NISQ hardware.

Nguyen, Thien↗
NASA NTRS2012

Detection Thresholds of Falling Snow from Satellite-Borne Active and Passive Sensors

Precipitation, including rain and snow, is a critical part of the Earth's energy and hydrology cycles. Precipitation impacts latent heating profiles locally while global circulation patterns distribute precipitation and energy from the equator to the poles. For the hydrological cycle, falling snow is a primary contributor in northern latitudes during the winter seasons. Falling snow is the source of snow pack accumulations that provide fresh water resources for many communities in the world. Furthermore, falling snow impacts society by causing transportation disruptions during severe snow events. In order to collect information on the complete global precipitation cycle, both liquid and frozen precipitation must be collected. The challenges of estimating falling snow from space still exist though progress is being made. These challenges include weak falling snow signatures with respect to background (surface, water vapor) signatures for passive sensors over land surfaces, unknowns about the spherical and non-spherical shapes of the snowflakes, their particle size distributions (PSDs) and how the assumptions about the unknowns impact observed brightness temperatures or radar reflectivities, differences in near surface snowfall and total column snow amounts, and limited ground truth to validate against. While these challenges remain, knowledge of their impact on expected retrieval results is an important key for understanding falling snow retrieval estimations. Since falling snow from space is the next precipitation measurement challenge from space, information must be determined in order to guide retrieval algorithm development for these current and future missions. This information includes thresholds of detection for various sensor channel configurations, snow event system characteristics, snowflake particle assumptions, and surface types. For example, can a lake effect snow system with low (approx 2.5 km) cloud tops having an ice water content (IWC) at the surface of 0.25 g / cubic m and dendrite snowflakes be detected? If this information is known, we can focus retrieval efforts on detectable storms and concentrate advances on achievable results. Here, the focus is to determine thresholds of detection for falling snow for various snow conditions over land and lake surfaces. The results rely on simulated Weather Research Forecasting (WRF) simulations of falling snow cases since simulations provide all the information to determine the measurements from space and the ground truth. Sensitivity analyses were performed to better ascertain the relationships between multifrequency microwave and millimeter-wave sensor observations and the falling snow/underlying field of view. In addition, thresholds of detection for various sensor channel configurations, snow event system characteristics, snowflake particle assumptions, and surface types were studied. Results will be presented for active radar at Ku, Ka, and W-band and for passive radiometer channels from 10 to 183 GHz.

Gail Skofronick-Jackson↗
DOE OSTI2025

AAPM Truth‐based CT (TrueCT) reconstruction grand challenge

Background: This Special Report summarizes the 2022, AAPM grand challenge on Truth-based CT image reconstruction. Purpose: To provide an objective framework for evaluating CT reconstruction methods using virtual imaging resources consisting of a library of simulated CT projection images of a population of human models with various diseases. Methods: Two hundred unique anthropomorphic, computational models were created with varied diseases consisting of 67 emphysema, 67 lung lesions, and 66 liver lesions. The organs were modeled based on clinical CT images of real patients. The emphysematous regions were modeled using segmentations from patient CT cases in the COPDGene Phase I dataset. For the lung and liver lesion cases, 1–6 malignant lesions were created and inserted into the human models, with lesion diameters ranging from 5.6 to 21.9 mm for lung lesions and 3.9 to 14.9 mm for liver lesions. The contrast defined between the liver lesions and liver parenchyma was 82 ± 12 HU, ranging from 50 to 110 HU. Similarly, the contrast between the lung lesions and the lung parenchyma was defined as 781 ± 11 HU, ranging from 725 to 805 HU. For the emphysematous regions, the defined HU values were −950 ± 17 HU ranging from −918 to −979 HU. The developed human models were imaged with a validated CT simulator. The resulting CT sinograms were shared with the participants. The participants reconstructed CT images from the sinograms and sent back their reconstructed images. Further, the reconstructed images were then scored by comparing the results against the corresponding ground truth values. The scores included both task-generic (root mean square error [RMSE] and structural similarity matrix [SSIM]), and task-specific (detectability index [d’] and lesion volume accuracy) metrics. For the cases with multiple lesions, the measured metric was averaged across all the lesions. To combine the metrics with each other, each metric was normalized to a range of 0 to 1 per disease type, with “0” and “1” being the worst and best measured values across all cases of the disease type for all received reconstructions. Results: The True-CT challenge attracted 52 participants, out of which 5 successfully completed the challenge and submitted the requested 200 reconstructions. Across all participants and disease types, SSIM absolute values ranged from 0.22 to 0.90, RMSE from 77.6 to 490.5 HU, d’ from 0.1 to 64.6, and volume accuracy ranged from 1.2 to 753.1 mm3. The overall scores demonstrated that participant “A” had the best performance in all categories, except for the metrics of d’ for lung lesions and RMSE for liver lesions. Participant “A” had an average normalized score of 0.41 ± 0.22, 0.48 ± 0.32, and 0.42 ± 0.33 for the emphysema, lung lesion, and liver lesion cases, respectively. Conclusions: The True-CT challenge successfully enabled objective assessment of CT reconstructions with the unique advantage of access to a diverse population of diseased human models with known ground truth. This study highlights the significant potential of virtual imaging trials in objective assessment of medical imaging technologies.

60 APPLIED LIFE SCIENCES↗