Robust Information Fusion and its Applications in Video Surveillance (P C Yuen, et al.)

While surveillance video camera networks have been installed in public areas in many countries, the development of intelligent video software technology lags far behind that of the associated hardware. Surveillance video processing has thus become a research topic of great interest in the computer vision and pattern recognition community over the last decade because of a number of potential applications such as human behaviour understanding, person identification from videos, event detection for video retrieval and gait-based criminal investigations. However, all these applications are very complicated. It may be difficult to achieve a good level of accuracy using a single piece of information from a video recording. Instead, it has been shown that combining different pieces of information from the same or different video recordings could significantly improve accuracy.

Combining multiple sources of information to solve computer vision and pattern recognition problems was proposed in the 1980s, and has been successfully applied in character recognition, biometrics, image retrieval, event detection and social media data analysis. While encouraging results have been reported in the literature, existing fusion algorithms do rely on strong assumptions concerning information dependency and data distribution. However, most surveillance video based applications do not satisfy all these assumptions. In turn, fusion performance will be degraded, and the degree of degradation will depend on the level of deviation from these assumptions. As a result, existing information fusion algorithms cannot perform at a consistent level. We have conducted a simple experiment that shows that performance can be degraded by 30%. This robustness issue must be resolved to deploy fusion methods in practical applications.

To address the forgoing robustness issue arising in information fusion, the proposed project will develop a distribution-free dependency model. The proposed dependency model does not require that the pieces of information to be fused are independent of each other and does not assume any data distribution. To reduce the distribution estimation error introduced by the presence of outliers, the model learning process will incorporate an outlier handling procedure. The proposed project will also investigate how to generalise the fusion model learnt from training data to query data by minimising the maximum mean discrepancy.

The proposed project will develop a new theory and algorithm for robust information fusion. The theory and algorithm to be developed will be generic in nature and will be applicable to computer vision and pattern recognition applications, as well as to other domains.


Grant Support:

This project is supported by the Research Grants Council (RGC), Hong Kong SAR, China (Project HKBU212313).


For further information on this research topic, please contact Prof. P C Yuen.