IAPR/IEEE Winter School on Biometrics 2019

Biometric Performance and Its Optimal Calibration

Slides (pdf)  Size: 7.58MB


One of the key purposes of measuring the accuracy of a biometrics system is benchmarking, how does one system compare to another. By controlling specific quality factors, it can answer question of whether one system more robust to adverse condition than another? In this talk, we will also explore means to measure if a system degrades over time, and a branch of statistical methods based on biometric menagerie, including Doddington's zoo. Collectively, they attempt to answer questions of the nature of not just quantitative, i.e., how much better or worse a system is, but also cast light on who, i.e., which subjects are causing problems and/or under what conditions. Another closely related issue is whether or not a reported performance can be generalised to another but slightly different operating conditions and/or a different set of subjects. To this end, in this talk, I will present you a set of statistical tools that help researchers and engineers in biometrics to (1) measure the system accuracy, potentially under limited samples; (2) adopt the best practices in reporting system accuracy, that is with confidence intervals, as well as using subject-specific statistical tools -- based on what I like to refer to as "the biometric score theory"; and (3) calibrate the system for optimal performance using various score calibration methods, including calibration by equal error rate, user-specific score calibration, and quality-based score calibration, to prepare for multimodal/multi-algorithmic fusion.


Norman Poh is currently with BJSS London where he works in a team of data scientists and machine learning engineers to solve a range of practical machine learning problems across different sectors. Currently, he helps a major oil and gas company to develop predictive algorithms that can automate the forecast of financial and accounting systems, as well as detect anomalies in asset valuations. Previously, he helped QuintilesIMS (now IQVIA) to detect and predict adverse drug effects. Prior to this, he was a senior lecturer at the University of Surrey and a principal investigator of a project funded by Medical Research Council (MRC) with the aim of modelling Chronic Kidney Disease. Whilst at the University of Surrey, he was named the Researcher of the Year 2011 for his contributions to research on multimodal biometrics and performance evaluation, as attested by nominations of five best paper/journal awards. His research was made possible thanks to two personal research grants from the Swiss National Science Foundation and two EU projects (Mobile Biometry - MOBIO and Biometric Evaluation and Testing - BEAT). He received his PhD from Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland.

Norman Poh

Norman Poh