The Versatile And Practical DeepMind Unsupervised Learning Method (Representation Learning Of Images Summer 2020 Feature 5)

3 main points
✔️ Super simple, high performance, differentiated method BYOL 
✔️ Can be used in various modalities Generic. method CPC 
✔️ Fewer resources, more robust, practical, and versatile

Data-Efficient Image Recognition with Contrastive Predictive Coding (CPC v2)
written by Olivier J. Hénaff, Aravind Srinivas, Jeffrey De Fauw, Ali Razavi, Carl Doersch, S. M. Ali Eslami, Aaron van den Oord
(Submitted on 22 May 2019 (v1), last revised 1 Jul 2020 (this version, v3))
Comments: Published by arXiv
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Paper Official Code COMM Code

Bootstrap your own latent: A new approach to self-supervised Learning (BYOL)
written by Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko
(Submitted on 13 Jun 2020 (v1), last revised 10 Sep 2020 (this version, v3))
Comments: Published by arXiv
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
Paper Official Code COMM Code

The writer's special project entitled "Learning to Express Images Summer 2020" introduces various methods of unsupervised learning.

Part 1. Image GPT for domain knowledge-free and unsupervised learning, and image generation is amazing!
Part 2. Contrastive Learning's Two Leading Methods SimCLR And MoCo, And The Evolution Of Each
Part 3. SOTA With Contrastive Learning And Clustering!
Part 4. Questions For Contrastive Learning : "What Makes?"
Part 5. The Versatile And Practical DeepMind Unsupervised Learning Method

In this feature. Results. We've been focusing on mainstream Contrastive Learning, but this time we're going to take the "DeepMind research" angle.

When you think of DeepMind, you might think of AlphaGo or AlphaZero.
It's an organization that is working on research that is far more difficult to get to than general deep learning... Doesn't that give you the impression that it's someone else's business?

Surprisingly, however, this may not be the case. In particular, the BYOL method described in the second half of this article is very practical and has the potential to be used in many different ways in the future, so check it out.

daisukelab