Is Metric Learning Really Taking A "big Leap Forward"? A Paper Re-evaluating Recent Research!
3 main points
✔️ Deep distance learning has claimed significant accuracy gains over the past four years, but this is questionable due to errors in the experimental setup.
✔️ Three problems: the unfairness of the comparison method, the use of feedback from test data for training, and the weakness of the metric for measuring accuracy.
✔️ Proposing a new method for evaluating distance learning algorithms
A comparison of the new evaluation method with traditional algorithms shows that the "great leap forward" in recent years has actually been only a modest one.
A Metric Learning Reality Check
written by Kevin Musgrave, Serge Belongie, Ser-Nam Lim
(Submitted on 18 Mar 2020)
Comments: Published by arXiv
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Introduction
At ECCV2020, it was pointed out by researchers at Facebook that the distance learning that has been done over the last four years has not evolved from basic contrastive/triplet loss. There are many reasons for this, and the evaluation is too sloppy. In addition, we have come up with some research that may have set up the hyperparameters in testing.
This research will be the content of the ECCV held August 24-27, 2020, and we covered it in April; it was a hot topic of discussion at the ECCV and we hope that you will catch up with it again under re-writing. This is especially important starting with the section on unfair performance comparisons.
What is distance learning?
Metric learning is a generic term for the task of training functions that embed data so that similar data move closer together and dissimilar data move away from each other and map to an embedding space.
The reasons for doing so are as follows. It seems easy to distinguish between A and C or B and C, but difficult to distinguish between A and B when classifying data like the following.
This is because A and B are close to each other in terms of the distance between the data belonging to each class. If we can embed this data in a separate space with some function so that the same class is close to each other and different classes are farther apart, as shown in the figure below, it should be easier to distinguish between A and B.
Deep Metric Learning refers to all methods that use deep neural networks (DNNs) for functions that embed input data. The aim of deep distance learning is to achieve higher accuracy than conventional methods by using DNNs, which have a high generalization performance in various tasks, including image recognition tasks, as the embedding function.
To read more,
Please register with AI-SCHOLAR.
ORCategories related to this article
