Spatial Transformation

  • parametric transform (affine transformation, thin-plate translation, etc): STN [2], hierarchical STN [5], deformable style transfer [10]

  • learn distortion grid [1] [6] [7]

  • learn conv offset: Deformable CNN v1[3], v2[4], deformable kernel [9]

  • optical flow: [8]

  • swap disentangled geometry-relevant feature

  • move keypoints: transGAGA [11]

Reference

[1] Recasens, Adria, et al. “Learning to zoom: a saliency-based sampling layer for neural networks.” ECCV, 2018.

[2] Jaderberg, Max, Karen Simonyan, and Andrew Zisserman. “Spatial transformer networks.” NIPS, 2015.

[3] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei:
Deformable Convolutional Networks. ICCV 2017.

[4] Xizhou Zhu, Han Hu, Stephen Lin, Jifeng Dai: Deformable ConvNets v2: More Deformable, Better Results. CoRR abs/1811.11168 (2018)

[5] Shu, Chang, et al. “Hierarchical Spatial Transformer Network.” arXiv preprint arXiv:1801.09467 (2018).

[6] Zheng, Heliang, et al. “Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition.” CVPR, 2019.

[7] Marin, Dmitrii, et al. “Efficient segmentation: Learning downsampling near semantic boundaries.” ICCV, 2019.

[8] Ren, Yurui, et al. “Deep Image Spatial Transformation for Person Image Generation.”, CVPR, 2020.

[9] Gao, Hang, et al. “Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation.” arXiv preprint arXiv:1910.02940 (2019).

[10] Kim, Sunnie SY, et al. “Deformable Style Transfer.” arXiv preprint arXiv:2003.11038 (2020).

[11] Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy: TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation. CVPR 2019