Everything You Need To Know About Transformers In Computer Vision! Part1/5(Summary, Issues And Outlook)

Transformer 22/01/2021

3 main points
✔️Explain the applications of Transformer in computer vision
✔️Introduce models and methods related to various tasks
✔️Explain the challenges and future prospects of Transformer in vision

Transformers in Vision: A Survey
written by Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah
(Submitted on 4 Jan 2021)
Comments: 24 pages
Subjects: Computer Vision and Pattern Recognition (cs.CV)

first of all

Transformer has shown its high performance not only in natural language processing but also in many other areas. Among them, the application research of the Transformer in the field of computer vision, which deals with visual information, has become very popular.

In light of this demand, we will provide a comprehensive explanation of the current status and future prospects of Transformers in computer vision.

This article provides a general description of transformers in computer vision, including a general classification of methods proposed in the past, current issues, and future prospects.

In Parts 2 to 5, we will provide individual and specific explanations of each method related to Transformer in computer vision that has been proposed in the past.

The total number of Transformer models to be explained is a whopping37 pieces in total!

The most recent of the methods introduced was published in December 2020, and covers very recent research. We hope that it will help those who are interested in the application of Transformer to image and video processing, etc., and those who want to acquire detailed knowledge of specific models.

Overall Structure (Table of Contents)

1. about Transformer in Computer Vision
・Broad classification of various methods

2. concrete examples of Transformer in computer vision (details are explained in Part2-5)
・Transformers for Image Recognition(Part2)
・Transformers for Object Detection(Part2)
・Transformers for Segmentation(Part3)
・Transformers for Image Generation (Part3)
・Transformers for Low-level Vision(Part3)
・Transformers for Multi-modal Tasks(Part4)
・Video Understanding(Part5)
・Transformers in Low-shot Learning(Part5)
・Transformers for Clustering(Part5)
・Transformers for 3D Analysis(Part5)

3. challenges and future prospects of transformer in computer vision
・High Computational Cost
・High Data Cost
・Need for Novel Designs
・Interpretability of Transformers
・Hardware Efficient Designs
・Is self-supervision the answer?