VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder

Yuchao Gu1,2, Xintao Wang2, Liangbin Xie2,5, Chao Dong4,5, Gen Li3, Ying Shan2, Ming-Ming Cheng1
1Nankai University, 2ARC Lab, Tencent PCG 3Platform Technologies, Tencent Online Video 4Shanghai AI Laboratory 5Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Interpolate start reference image.

Compared to GFP-GAN, VQFR exploits the Vector-Quantized (VQ) dictionary and parallel decoder to restore high-quality facial details on various facial regions.


Although generative facial prior and geometric prior have recently demonstrated high-quality results for blind face restoration, producing fine-grained facial details faithful to inputs remains a challenging problem.

Motivated by the classical dictionary-based methods and the recent vector quantization (VQ) technique, we propose a VQ-based face restoration method - VQFR. VQFR takes advantage of high-quality low-level feature banks extracted from high-quality faces and can thus help recover realistic facial details. However, the simple application of the VQ codebook cannot achieve good results with faithful details and identity preservation.

Therefore, we further introduce two special network designs. 1). We first investigate the compression patch size in the VQ codebook and find that the VQ codebook designed with a proper compression patch size is crucial to balance the quality and fidelity. 2). To further fuse low-level features from inputs while not “contaminating” the realistic details generated from the VQ codebook, we proposed a parallel decoder consisting of a texture decoder and a main decoder. Those two decoders then interact with a texture warping module with deformable convolution. Equipped with the VQ codebook as a facial detail dictionary and the parallel decoder design, the proposed VQFR can largely enhance the restored quality of facial details while keeping the fidelity to previous methods.

Method Overview

Interpolate start reference image.

Overview of VQFR framework. It consists of an encoder to map degraded face into latent and a parallel decoder to exploit the HQ code and input feature. The encoder and decoder are bridged by vector quantization model and a pretrained HQ codebook to replace the encoded latent to HQ code.

Visual Comparison

Interpolate start reference image.

Comparisons with state-of-the-art face restoration methods: HiFaceGAN, DFDNet, PSFRGAN, PULSE and GFP-GAN on the real-world low-quality images.


title={VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder},
author={Gu, Yuchao and Wang, Xintao and Xie, Liangbin and Dong, Chao and Li, Gen and Shan, Ying and Cheng, Ming-Ming},


If you have any question, please contact Yuchao Gu at yuchaogu9710@gmail.com.