Local Binary Pattern Approach to Computer Vision |
|
Matti Pietikäinen, Guoying Zhao, and Abdenour Hadid Department of Electrical and Information Engineering, University of Oulu, P.O. Box 4500, FI-90014 University of Oulu, Finland
OverviewThe local binary pattern (LBP) operator is defined as a gray-scale invariant texture measure, derived from a general definition of texture in a local neighborhood. Through its recent extensions, the LBP operator has been made into a really powerful measure of image texture, showing excellent results in many empirical studies. The LBP operator can be seen as a unifying approach to the traditionally divergent statistical and structural models of texture analysis. Perhaps the most important property of the LBP operator in real-world applications is its invariance against monotonic gray level changes. Another equally important is its computational simplicity, which makes it possible to analyze images in challenging real-time settings. The LBP method and its variants have already been used in a large number of applications all over the world. For a bibliography of LBP-related research, see http://www.ee.oulu.fi/research/imag/texture/. Recently, we have begun to study tasks which have not been previously considered as texture analysis problems. Our facial image representation based on local binary patterns, proposed at ECCV 2004, has evolved to be a growing success. It has been adopted and further developed by many research groups. We have also developed the first texture-based method for subtracting the background and detecting moving objects in real time. All these results indicate that texture and the ideas behind LBP methodology could have a much wider role in computer vision and image analysis than was earlier thought. This tutorial presents an overview of the LBP approach. First the theoretical foundations of the method are presented. An overview of applying LBP to various computer vision applications is then given, including industrial inspection, classification of 3D textured surfaces, face recognition, face detection, facial expression recognition, content-based retrieval, modeling the background and detecting moving objects, and recognition of dynamic textures. More detailed presentations on recent developments of the LBP to increase its robustness and on its use in facial image analysis and motion analysis are then given. Finally, directions for future research are discussed.
Outline
BiographiesMatti Pietikäinen received his Doctor of Science in Technology degree in electrical engineering from the University of Oulu, Finland, in 1982. In 1981, he established the Machine Vision Group at the University of Oulu. The research results of his group have been widely exploited in industry. Currently, he is Professor of Information Engineering, Scientific Director of Infotech Oulu research center, and Leader of Machine Vision Group at the University of Oulu. From 1980 to 1981 and from 1984 to 1985, he visited the Computer Vision Laboratory at the University of Maryland, USA. His research interests are in machine vision and image analysis. His current research focuses on texture analysis, facial image analysis, and machine vision for sensing and understanding human actions. He has authored over 180 papers in international journals, books, and conference proceedings, and about 100 other publications or reports. He is Associate Editor of Pattern Recognition journal, and was Associate Editor of IEEE Transactions on Pattern Analysis and Machine Intelligence from 2000 to 2005. He was Chairman of the Pattern Recognition Society of Finland from 1989 to 1992. Since 1989, he has served as a member of the governing board of the International Association for Pattern Recognition (IAPR) and became one of the founding fellows of the IAPR in 1994. He has also served on committees of several international conferences. He is a senior member of the IEEE and was Vice-Chair of IEEE Finland Section. Guoying Zhao received her Ph.D. from the Chinese Academy of Sciences in 2005. Currently she is a postdoctoral research scientist at the University of Oulu. Her research area is dynamic texture recognition with LBP, and its applications in analysing facial expressions and other dynamic events. She has authored about 30 papers in journals and conferences, and has served as a reviewer for some journals and conferences. Abdenour Hadid received his Engineer Diploma (Master of Science in Computing) from the National Institute of Informatics (INI, Algiers), in 1997, and the Doctor of Science in Technology degree in electrical and information engineering from the University of Oulu, Finland, in 2005. Now, he is a postdoctoral researcher in the Machine Vision Group, University of Oulu. His research interests include: biometrics and facial image analysis, local binary patterns, manifold learning, human-machine interaction, and mobile applications. He has authored several papers in international conferences and journals, and served as a reviewer for many international conferences and journals. He is a member of the Pattern Recognition Society of Finland, and served as a member of the organizing committee of an international workshop on Processing Sensory Information for Proactive Systems (PSIPS2004). In 2005, he gave, with Prof. Pietikäinen, a tutorial on "Local Binary Patterns in Facial Image Analysis" at SCIA 2005 (the Scandinavian Conference on Image Analysis). He has been visiting the Institute of Automation at the Chinese Academy of Science (Beijing) in spring 2006. |