Efficient Training and Inference of Transformer Models (Longkai Huang et al.)

Efficient Training and Inference of Transformer Models (Longkai Huang et al.)

This project aims to make transformer models more efficient in training and deployment. Most AI systems today are built on transformer architectures, and these models are often extremely large and require substantial computational resources. This creates a major barrier for both research and real-world applications.

This project aims to addresse the challenge from three directions: efficient pre-training, efficient fine-tuning, and efficient inference.

In efficient pre-training, we study how to train the best possible models under limited computational budgets. This involves three closely related questions: how to reduce computational cost, how to maintain strong model performance, and how to allocate limited resources in the most effective way. Reducing cost alone is not enough. The more fundamental yet harder problem is how to keep compressed or structured models performing close to full models. To address this, we combine theoretical analysis with the design of model structures and optimization algorithms, so that efficient models can better approximate the learning behavior of their full counterparts. We also study the relationship between model performance and resource factors such as model size, batch size, and other factors, with the goal of improving overall training efficiency under practical constraints.

In efficient fine-tuning, we focus on how models can acquire new capabilities from limited data or limited supervision, and how they can continue learning over time without repeated full retraining. Our work in this area includes model editing and continual learning methods. We are also exploring meta-learning paradigms that may enable models to learn new tasks autonomously and more efficiently.

In efficient inference, we study how to significantly improve the speed of large models while preserving performance, for example through more efficient KV-cache usage and improved decoding strategies.

Overall, this project aims to make advanced AI models more efficient, more affordable, and more widely accessible.

For further information on this research topic, please contact Prof. Longkai Huang.

Research Interest of Prof. Longkai Huang

His research lies at the intersection of machine learning theory and applications. He is particularly interested in understanding how ML models learn, store, and transfer knowledge, and in using these insights to develop more effective and efficient learning algorithms. His work focuses on continual learning, meta-learning, and efficient training and inference of transformer models. He is also interested in applying AI to scientific discovery, especially in life science domains such as genomics, protein design, and drug discovery.