Evaluating the Trustworthiness of Deep Neural Networks in Deployment – A Comparative Study (Replicability Study)
As deep neural networks (DNNs) are increasingly used in safety critical applications, there is a growing concern for their trustworthiness. Even highly trained, high-performant networks are not 100% accurate. However, it is very difficult to predict their behaviour during deployment without ground truth. In this paper, we provide a comparative and replicability study on recent approaches that have been proposed to evaluate the trustworthiness of DNNs. We find that it is very difficult to run and reproduce the results for these approaches on their replication packages, and it is even more difficult to run the tools on artifacts other than their own. Further, it is difficult to compare the effectiveness of the tools, due to lack of clearly defined evaluation metrics. Our results indicate that more effort is needed in our research community to obtain sound techniques for evaluating the trustworthiness of neural networks in safety-critical domains. To this end, we contribute an evaluation framework that incorporates the considered approaches and enables evaluation on common benchmarks, using common metrics. Using this framework, we run a comparative study of the three approaches.