Professor Xiaowen Chu and His Team Win the Best Paper Award in IEEE INFOCOM 2021

May 21, 2021
Prof. Xiaowen Chu (right), Dr. Shaohuai Shi (left) and Prof. Bo Li from HKUST, received the Best Paper Award IEEE INFOCOM 2021.

Prof. Xiaowen Chu, Professor, Dr. Shaohuai Shi, PhD Graduate of Department of Computer Science and Prof. Bo Li, HKUST, received the Best Paper Award for their co-authored paper “Exploiting Simultaneous Communications to Accelerate Data Parallel Distributed Deep Learning“ at the IEEE International Conference on Computer Communications (IEEE INFOCOM 2021).

The award-winning paper proposes a novel algorithm to reduce the training time of large AI models when running on GPU clusters. Through theoretical analysis and experiments, it demonstrates that simultaneous All-Reduce communications can effectively improve the communication efficiency of small tensors. To exploit both tensor fusion and simultaneous communications, the team formulated an optimization problem and developed an efficient solution named ASC-WFBP. The team also ran real-world experiments on an 8-node GPU cluster with 32 GPUs and 10Gbps Ethernet. Tested on four modern AI models, their experimental findings indicated that ASC-WFBP could achieve speedups of around 1.09x - 2.48x over the baseline without tensor fusion, and 1.15x - 1.35x over the state-of-the-art tensor fusion solution.

The 4-day Conference on Computer Communications (INFOCOM) is a top-ranked conference on networking in the research community. It is for researchers to present and exchange significant and innovative contributions and ideas in the field of networking and closely related areas. Out of 1266 submissions, only one-fifth were accepted after a rigorous double-blind review process, and finally three papers were selected to receive the Best Paper Award.