Learning Bayesian Networks for Pattern Classification

Boaz Lerner

Department of Electrical & Computer Engineering, Ben-Gurion University, Israel

Overview

In the recent years, the Bayesian network (BN) has become one of the most studied machine learning models for knowledge representation, probabilistic inference and pattern classification. The BN is a graphical model that efficiently encodes the joint probability distribution for a set of variables representing a domain. The BN enhances interpretability compared to other state-of-the-art machine learning models by exhibiting dependences, independences and causal relations between domain variables. It also allows the incorporation of prior knowledge during model learning in order to select a better model or improve the estimation of its data-driven parameters. In addition, the BN performs feature selection as part of model construction and permits the inclusion of hidden nodes which enhances model representability and predictability. Recently, the traditional use of BNs in general probabilistic inference has been extended to pattern classification. This extension is demonstrated in the application of BN classifiers in diversity of domains. It is also manifested in increasing numbers of methods developed for learning the BN classifier.

In its first part, this self-contained tutorial will provide a graduate-level introduction to the field of Bayesian networks and graphical models. It will demonstrate applications of BNs in machine learning and pattern classification and focus on the main challenges in the field. Then, the tutorial will present different approaches for learning a BN model and provide example methods for each approach as well as experimental comparison of these methods on synthetic and real-world databases. In the second part, the tutorial will concentrate on BN classifiers and describe recent progress in the field. Novel structure learning algorithms for BN classifiers will be introduced and compared to common algorithms with respect to structural correctness, complexity and prediction accuracy. Additionally, novel BN multinet classifiers will be presented and compared to other multinet and BN classifiers.

Outline

The following topics will be covered:

Part 1

Introduction to graphical models and Bayesian networks
Applications of BNs in machine learning
Learning the BN parameters
Learning the BN structure using search-and-score or constraint-based methods
Comparison of methods for learning BNs using different case studies
Probabilistic inference in the BN

Part 2

BN classifiers and challenges in learning a BN classifier
Extending methods for learning a BN to learning a BN classifier
Novel methodologies for learning a BN classifier
Multinet BN classifiers

Biography

Boaz Lerner is a Senior Lecturer at the Department of Electrical & Computer Engineering at Ben-Gurion University, Israel. Before joining Ben-Gurion University in 2000 he spent two years (1996-1998) at the Neural Computing Research Group at Aston University, Birmingham, UK, and another two years (1998-2000) at the Computer Laboratory of the University of Cambridge, UK. Dr. Lerner’s fields of interest lie in machine learning, learning Bayesian networks, neural networks and statistical pattern recognition. In addition, he has been consulting to the Hi-Tech industry in numerous R&D projects applying machine learning to text, signal and image processing.