Large Vision-Language Models : Pre-Training, Prompting and Applications

The rapid progress in the field of large multimodal foundation models, especially vision-language models, has dramatically transformed the landscape of machine learning, computer vision, and natural language processing. These powerful models, trained on vast amounts of multimodal data mixed with images and text, have demonstrated remarkable capabilities in tasks ranging from image classification and object detection to visual content generation and question answering. This book provides a comprehensive and up-to-date exploration of large vision-language models, covering the key aspects of their pre-training, prompting techniques, and diverse real-world computer vision applications. It is an essential resource for researchers, practitioners, and students in the fields of computer vision, natural language processing, and artificial intelligence.

Kaiyang Zhou is an Assistant Professor at the Department of Computer Science, Hong Kong Baptist University, working on computer vision and machine learning. He has published more than 30 technical papers in top-tier journals and conferences in relevant fields, including CVPR, ICCV, ECCV, NeurlPS, ICLR, ICML, AAAI, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), and International Journal of Computer Vision (IJCV), with over 10,000 citations received in total. He is an Associate Editor of IJCV, the flagship journal in computer vision, and regularly serves as area chair and senior program committee for top-tier computer vision and machine learning conferences, such as NeurIPS, CVPR, ECCV, and AAAI.

Ziwei Liu is an Associate Professor at Nanyang Technological University, Singapore. His research interests include computer vision, machine learning, and computer graphics. He has published extensively with top-tier conferences and journals in relevant fields, including CVPR, ICCV, ECCV, NeurlPS, ICLR, ICML, IEEE Transactions on Pattern Analysis and Machine Intelligence, ACM Transactions on Graphics and Nature - Machine Intelligence. He is the recipient of ICCV Young Researcher Award, HKSTP Best Paper Award, CVPR Best Paper Award Candidate, ICBS Frontiers of Science Award and MIT Technology Review Innovators under 35 Asia Pacific. He serves as an area chair of CVPR, ICCV, ECCV, NeurlPS and ICLR, as well as an associate editor of International Journal of Computer Vision. 

Peng Gao is a research scientist at Shanghai Artificial Intelligence Laboratory, working on large language models and vision-language models. His research interests include vision-language models, large language models and diffusion models for contents creation. He has published more than 40 papers in top-tier journals and conferences, including International Journal of Computer Vision (IJCV), ICML, ICLR, NeurIPS, CVPR, ICCV and ECCV, receiving more than 10,000 citations. He has led several influential open-source projects including LLaMa-Adapter and the Lumina series, receiving more than 7000 and 2000 stars, respectively.