[iasmath-seminars] Today's Seminar on TML delayed to 1:30pm

[Kristina Phillips]

Seminar on Theoretical Machine Learning

Topic:                    A Theoretical Analysis of Contrastive Unsupervised
Representation Learning

Speaker:              Orestis Plevrakis, Princeton University

Time/Room:       1:30pm - 2:45pm/White Levy Room

Abstract Link:     http://www.math.ias.edu/seminars/abstract?event=139490

 

Recent empirical works have successfully used unlabeled data to learn
feature representations that are broadly useful in downstream classification
tasks. Several of these methods are reminiscent of the well-known word2vec
embedding algorithm: leveraging availability of pairs of semantically
"similar" data points and "negative samples," the learner forces the inner
product of representations of similar pairs with each other to be higher on
average than with negative samples. The current paper uses the term
contrastive learning for such algorithms and presents a theoretical
framework for analyzing them by introducing latent classes and hypothesizing
that semantically similar points are sampled from the same latent class.
This framework allows us to show provable guarantees on the performance of
the learned representations on the average classification task that is
comprised of a subset of the same set of latent classes. Our generalization
bound also shows that learned representations can reduce the (labeled)
sample complexity on downstream tasks. We conduct controlled experiments in
both the text and image domains to support the theory.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imap.math.ias.edu/pipermail/iasmathsemo/attachments/20190311/49bb0009/attachment.html>