NeurIPS 2021 Workshop: Self-Supervised Learning - Theory and Practice


Online Workshop, 07:00 AM – 04:30 PM (PST), Dec 14, 2021

Workshop webpage: https://neurips.cc/Conferences/2021/Schedule?showEvent=21853
Conference (NeurIPS registered users): https://neurips.cc/virtual/2021/workshop/21853

For poster sessions: posters can be viewed in the Accepted Papers page. If interested in a poster, you can click on the corresponding Zoom link. The authors will be presenting their posters in their Zoom.

Self-supervised learning (SSL) is an unsupervised approach for representation learning without relying on human-provided labels. It creates auxiliary tasks on unlabeled input data and learns representations by solving these tasks. SSL has demonstrated great success on images (e.g., MoCo, PIRL, SimCLR) and texts (e.g., BERT) and has shown promising results in graphs, time-series, audio, etc. On a wide variety of tasks, SSL without using human-provided labels achieves performance that is close to fully supervised approaches.

The existing SSL research mostly focuses on improving the empirical performance without a theoretical foundation. While the proposed SSL approaches are empirically effective, theoretically why they perform well is not clear. For example, why certain auxiliary tasks in SSL perform better than others? How many unlabeled data examples are needed by SSL to learn a good representation? How is the performance of SSL affected by neural architectures?

In this workshop, we aim to bridge this gap between theory and practice. We bring together SSL-interested researchers from various domains to discuss the theoretical foundations of empirically well-performing SSL approaches and how the theoretical insights can further improve SSL’s empirical performance. Different from previous SSL-related workshops which focus on empirical effectiveness of SSL approaches without considering their theoretical foundations, our workshop focuses on establishing the theoretical foundation of SSL and providing theoretical insights for developing new SSL approaches.

We invite submissions of both theoretical works and empirical works, and the intersection of the two. The topics include but are not limited to:

  • Theoretical foundations of SSL
  • Sample complexity of SSL methods
  • Theory-driven design of auxiliary tasks in SSL
  • Comparative analysis of different auxiliary tasks
  • Comparative analysis of SSL and supervised approaches
  • Information theory and SSL
  • SSL for computer vision, natural language processing, robotics, speech processing, time-series analysis, graph analytics, etc.
  • SSL for healthcare, social media, neuroscience, biology, social science, etc.
  • Cognitive foundations of SSL

This year, this workshop will have 9 invited talks by leading researchers from diverse backgrounds including theoretical ML, CV, NLP, robotics, etc. The workshop will also have 5 contributed talks, 2 poster sessions, and 1 panel discussion session to share perspectives on establishing foundational understanding of existing SSL approaches and theoretically-principled ways of developing new SSL methods. We accept short (max 4 pages) submissions. Each will be peer-reviewed by at least three reviewers. The accepted papers are allowed to be submitted to other conference venues.