[1] Zagoruyko, Sergey, and Nikos Komodakis. “Diracnets: Training very deep neural networks without skip-connections.” arXiv preprint arXiv:1706.00388 (2017).
[2] Ding, Xiaohan, et al. “Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks.” Proceedings of the IEEE International Conference on Computer Vision. 2019.
[3] “RepVGG: Making VGG-style ConvNets Great Again”, arXiv, 2021.
[1] Gao, Hang, et al. “Low-shot learning via covariance-preserving adversarial augmentation networks.” NIPS, 2018.
[2] Sun, Baochen, and Kate Saenko. “Deep coral: Correlation alignment for deep domain adaptation.” ECCV, 2016.
[3] Kumar, Abhishek, Prasanna Sattigeri, and Avinash Balakrishnan. “Variational inference of disentangled latent concepts from unlabeled observations.” ICLR, 2018.
parametric transform (affine transformation, thin-plate translation, etc): STN [2], hierarchical STN [5], deformable style transfer [10]
learn conv offset: Deformable CNN v1[3], v2[4], deformable kernel [9]
optical flow: [8]
swap disentangled geometry-relevant feature
move keypoints: transGAGA [11]
[1] Recasens, Adria, et al. “Learning to zoom: a saliency-based sampling layer for neural networks.” ECCV, 2018.
[2] Jaderberg, Max, Karen Simonyan, and Andrew Zisserman. “Spatial transformer networks.” NIPS, 2015.
[3] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei:
Deformable Convolutional Networks. ICCV 2017.
[4] Xizhou Zhu, Han Hu, Stephen Lin, Jifeng Dai: Deformable ConvNets v2: More Deformable, Better Results. CoRR abs/1811.11168 (2018)
[5] Shu, Chang, et al. “Hierarchical Spatial Transformer Network.” arXiv preprint arXiv:1801.09467 (2018).
[6] Zheng, Heliang, et al. “Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition.” CVPR, 2019.
[7] Marin, Dmitrii, et al. “Efficient segmentation: Learning downsampling near semantic boundaries.” ICCV, 2019.
[8] Ren, Yurui, et al. “Deep Image Spatial Transformation for Person Image Generation.”, CVPR, 2020.
[9] Gao, Hang, et al. “Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation.” arXiv preprint arXiv:1910.02940 (2019).
[10] Kim, Sunnie SY, et al. “Deformable Style Transfer.” arXiv preprint arXiv:2003.11038 (2020).
[11] Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy: TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation. CVPR 2019
An Overview of Deep Semi-Supervised Learning
mixmatch [4]: gracefully unify data augmentation, sharpening (low entropy), mixup.
[1] Deep Co-Training for Semi-Supervised Image Recognition
[2] Tri-net for Semi-Supervised Deep Learning
[3] Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition
[4] Berthelot, David, et al. “Mixmatch: A holistic approach to semi-supervised learning.” arXiv preprint arXiv:1905.02249 (2019).
[5] Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, Quoc V. Le, “Unsupervised Data Augmentation for Consistency Training.” arXiv preprint arXiv:1904.12848 (2019).
Design a proxy task using unlabeled or weakly-labeled data to help the original task. Essentially, self-supervised learning is multi-task learning with the proxy task not relying on heavy human annotation. The problem is which proxy task without human annotation is the most effective one.
Please refer to the tutorial slides [1] [2], the survey, and the paper list.
image-to-image
video-to-image
video-to-video: all video-to-image methods can be used for video-to-video by averaging frame features.
Muti-task self-supervised learning: integrate multiple proxy tasks [1] [2]
Combined with other frameworks: self-supervised GAN [1]
A recent paper [1*] claims that the best self-supervised learning method is still the earliest image inpainting model. The design of network architecture has a significant impact on the performance of self-supevivsed learning methods.
SimCLR [2*] is a SOTA self-supervised learning method with performance approaching supervised learning.
[1*] Alexander Kolesnikov, Xiaohua Zhai, Lucas Beyer: Revisiting Self-Supervised Visual Representation Learning. CVPR 2019.
[2*] Chen, Ting, et al. “A simple framework for contrastive learning of visual representations.” arXiv preprint arXiv:2002.05709 (2020).
detect repeated patterns [1]
inpaint corrupted images with repeated patterns [2]: use frequency convolution
[1] Louis Lettry, Michal Perdoch, Kenneth Vanhoey, and Luc Van Gool. Repeated pattern detection using cnn activations. In WACV, 2017
[2] Suvorov, Roman, et al. “Resolution-robust Large Mask Inpainting with Fourier Convolutions.” arXiv preprint arXiv:2109.07161 (2021).
Real receptive field is smaller than theorical receptive field, and shrinks by $\frac{1}{\sqrt{n}}$ with $n$ being the number of layers.
Advanced networks (e.g., ResNet) have larger receptive field than old networks (e.g., AlexNet). In latest networks, the receptive field of each pixel in the last layer is as large as the whole image. Generally, larger receptive field leads to higher accuracy, but is not the only factor that influences the accuracy.
Fomoro: a website to calculate receptive field.
Distill: mathematical derivations and open-source library to compute receptive field.