Performance Evaluation Metrics for Biometrics-based Authentication Systems

Abstract

Research has made many advances in many fields of biometric systems. However, there is a lack of proper research in the performance evaluation field for biometric authentication systems. In this work, we synthesize key ideas from recent publications to show how the landscape for performance evaluation of biometrics-based authentication systems has developed over time. We present the general architecture and fundamentals of an authentication system. We then show why most single-number evaluation metrics have inherent flaws and limited scopes. We synthesize and extract the four significant limitations of commonly used metrics. We also discuss how each can be avoided or addressed by using metrics like Gini Coefficient or the unnormalized frequency counts ofscores (FCS) graph. Besides, we explore the limitations of metrics in the context of continuous authentication. We demonstrate how the combination of the Receiver Operating Characteristics (ROC) curve and the FCS graph can help us address the limitations of common metrics and provide a robust performance evaluation approach. From this, we present the state-of-the-art guidelines for evaluating authentication systems. In the end, we identify three critical problems with the current methodologies and propose specific future work plans to address them.