Contrastive Learning with Hard Negative Samples

Speaker(s): Kevin Wang
Topic: Contrastive Learning with Hard Negative Samples

Kevin talked about the paper Contrastive Learning with Hard Negative Samples, by Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie Jegelka. In the past two years, contrastive learning has emerged as a powerful unsupervised computer vision technique for learning effective representations of data for downstream tasks. This theory-focused paper proposes a technique for sampling “hard” negative examples in contrastive learning. The authors note improved performance on downstream tasks compared to SimCLR and faster training.

Supplemental Resources

Lilian Weng’s blog post on contrastive representation learning
Ekin Tiu’s post on contrastive learning
Google SimCLR