HONG KONG BAPTIST UNIVERSITY
FACULTY OF SCIENCE

Department of Computer Science Seminar
2015 Series

Random Projections as Regularizers: Learning a Linear Discriminant from Fewer Observations than Dimensions

Dr. Bob Durrant
Senior Lecturer
University of Waikato
New Zealand

Date: November 20, 2015 (Friday)
Time: 2:30 - 3:30 pm
Venue: SCT716, Cha Chi Ming Science Tower, Ho Sin Hang Campus

Abstract
'Random projection' is a simple non-adaptive dimensionality reduction scheme which is finding increasing use in application as a result of its computational cheapness, effectiveness, and pleasing theoretical properties. In this talk I will present some theoretical guarantees for the performance of an averaging-ensemble of randomly projected Fisher Linear Discriminant (FLD) classifiers, focusing on the case when there are far fewer training observations than data dimensions. I will also discuss the attractive computational properties of the algorithm this theory implies. The specific form and simplicity of this ensemble permits a direct and detailed analysis of its performance and, in particular, one can show that the randomly projected ensemble is equivalent to implementing a sophisticated regularization scheme to the linear discriminant learned in the original data space and this helps prevent overfitting in conditions of small sample size where pseudo-inverse FLD learned and the space is provably poor. I will also present some experimental results, which corroborate our theoretical findings and demonstrate the utility of our approach on some very high-dimensional datasets from the bioinformatics and drug discovery domains, both settings in which fewer observations than dimensions are the norm.

Biography
Bob Durrant is a Senior Lecturer in the Department of Statistics at the University of Waikato, New Zealand. He has a BSc(Hons) in Mathematical Sciences from the Open University UK, an MSc in Natural Computation from the University of Birmingham, and a PhD in Computer Science also from the University of Birmingham.

His published conference papers include work at ICML, ACML, ECML, AISTATS, ICPR and GECCO, of which three of his submissions won "best paper" awards. He has served on the program committees of several conferences and workshops, given technical and non-technical invited talks, presented an invited tutorial on Random Projections at ECML/PKDD 2012 in Bristol, and organized and ran the workshop on "Learning from Small Sample Sizes" at KDD 2014 in Sydney.

Bob's main research interests are in the areas of Machine Learning and Statistical Pattern Recognition; in particular using theory to better understand existing techniques, and to develop efficient, effective, and principled methods for big data settings with performance guarantees. His current research focus is on the problem of learning from small samples of high-dimensional data.

He is a member of the New Zealand Statistical Association, the ANZ chapter of SIGKDD, and the Royal Society of New Zealand.

********* ALL INTERESTED ARE WELCOME ***********
(For enquiry, please contact Computer Science Department at 3411 2385)

http://www.comp.hkbu.edu.hk/v1/?page=seminars&id=355&lang=sc
Photos  Slides