Artistic and Photorealistic Style Transfer

Posted on | In

Transfer strategy

  • Feature transfer: AdaIN [1], WCT [2], SANet [3].

  • Color transfer: Learn color transformation (explicit function or implicit function (e.g., look-up table) conditioned on color values, location, semantic information, or other guidance.

Compare Different Backbones

[6] [7]

Losses:

  • paired supervision: L2 loss
  • unpaired supervision: adversarial loss
  • smooth loss: variation loss, Poisson loss
  • content loss: perception loss
  • style loss: Gram loss, AdaIn loss

Multi-scale stylization

  • parallel: [4]

  • sequential: [5]

Reference

[1] Huang, Xun, and Serge Belongie. “Arbitrary style transfer in real-time with adaptive instance normalization.” ICCV, 2017.

[2] Li, Yijun, et al. “Universal style transfer via feature transforms.” NeurIPS, 2017.

[3] Park, Dae Young, and Kwang Hee Lee. “Arbitrary style transfer with style-attentional networks.” CVPR, 2019.

[4] Liu, Songhua, et al. “Adaattn: Revisit attention mechanism in arbitrary neural style transfer.” ICCV, 2021.

[5] Xia, Xide, et al. “Joint bilateral learning for real-time universal photorealistic style transfer.” ECCV, 2020.

[6] Wang, Pei, Yijun Li, and Nuno Vasconcelos. “Rethinking and improving the robustness of image style transfer.” CVPR, 2021.

[7] Wei, Hua-Peng, et al. “A Comparative Study of CNN-and Transformer-Based Visual Style Transfer.” Journal of Computer Science and Technology 37.3 (2022): 601-614.