Binary Classifiers as Dilations

Joint with Gabriel Ziegler.

Abstract: Seidenfeld and Wasserman (1993) define the phenomenon of dilation. When a dilation occurs, any additional information only increases the uncertainty about the true state of the world. In this paper, we show that dilation may manifest in real-world scenarios when information is provided by binary classifiers, such as diagnostic tests and predictive algorithms. This can happen when classifier performance measures are partially identified due to an imperfect reference classifier, which is common in practice. We characterize when a dilation occurs and develop corresponding inference procedures based on methods for subvector inference in moment inequality models. We apply the approach to diagnostic procedures for COVID-19 detection, using CT chest scans evaluated by radiologists and AI algorithms. We cannot reject the hypothesis that the radiologists' assessments exhibit a dilation, thus showcasing a potential real-world instance of a dilation. We additionally illustrate the broader applicability of our methodology by rejecting the hypothesis that data-mining techniques for predicting the riskiness of credit card applications are non-informative in the sense of dilation.