Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network

Wanze CHEN; Liqing HUANG; Jiazhen CHEN; Feng YE; Tianqiang HUANG; Haifeng LUO

doi:10.11959/j.issn.2096-109x.2023046

您当前的位置：

首页 >

文章列表页 >

Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network

Papers | 更新时间：2024-06-03

- Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network
- Chinese Journal of Network and Information Security Vol. 9, Issue 3, Pages: 150-160(2023)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2023046
  CLC： TP37；TP391
- Online First：2023-06，
  
  Published：25 June 2023
- 稿件说明：
移动端阅览
Wanze CHEN, Liqing HUANG, Jiazhen CHEN, et al. Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 150-160.
DOI：

Wanze CHEN, Liqing HUANG, Jiazhen CHEN, et al. Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 150-160. DOI： 10.11959/j.issn.2096-109x.2023046.

摘要

面部属性编辑领域的主流方法由于数据以及模型架构的限制，存在如下两方面的问题：自编码器模型在编码以及解码阶段的瓶颈结构会丢失特征信息，以及在解码过程中逐阶段对源域特征进行连续的样式注入，使得生成图像中目标域信息占比过大，导致生成图像丢失了源域的身份信息以及部分细节；在人脸图像中，图像人物的性别、种族或年龄等属性的差异会使得不同图像的频域信息组成差别较大，在无监督的训练前提下，当前主流网络框架并不能在样式注入阶段调整源域信息以及目标域信息之间的占比，导致生成图像仍然存在伪影。针对上述问题，基于生成对抗网络和图像到图像的翻译提出一种面部性别伪造模型，即融合快捷小波连接生成对抗网络（WscGAN）。通过对自编码器结构添加快捷连接，将不同编码阶段的输出通过小波变换进行特征级别的分解，再引入通道注意力机制将其进行逐个处理，以此动态改变在解码过程中不同频率的源域特征信息的比例，最终实现面部图像在性别属性上的伪造。为了验证所提模型的有效性，WscGAN 分别在 CelebA-HQ 数据集和 FFHQ 数据集上进行了实验。实验结果表明，WscGAN 在 CelebA-HQ 以及 FFHQ 数据集上均领先现有先进模型，其中弗雷歇起始距离分别提升5.4%和19.8%，特征度量相似度分别提升1.8%和4.1%。此外，定性的视觉对比结果充分体现了WscGAN可有效改善面部图像性别属性伪造的效果。

Abstract

The mainstream methods in the field of facial attribute manipulation had the following two defects due to data and model architecture limitations.First

the bottleneck structure of the autoencoder model results in the loss of feature information

and the traditional method of continuously injected styles to the source domain features during the decoding process makes the generated image too referential to the target domain while losing the identity information and fine-grained details.Second

differences in facial attributes composition between images

such as gender

ethnicity

or age can cause variations in frequency domain information.And the current unsupervised training methods do not automatically adjust the proportion of source and target domain information in the style injection stage

resulting in artifacts in generated images.A facial gender forgery model based on generative adversarial networks and image-to-image translation techniques

namely fused wavelet shortcut connection generative adversarial network (WscGAN)

was proposed to address the these issues.Shortcut connections were added to the autoencoder structure

and the outputs of different encoding stages were decomposed at the feature level by wavelet transform.Attention mechanism was employed to process them one by one

to dynamically change the proportion of source domain features at different frequencies in the decoding process.This model could complete forgery of facial images in terms of gender attributes.To verify the effectiveness of the model

it was conducted on the CelebA-HQ dataset and the FFHQ dataset.Compared with the existing optimal models

the method improves the FID and LPIPS indices by 5.4% and 11.2%

and by 1.8% and 6.7%

respectively.Furthermore

the effectiveness of the proposed method in improving the gender attribute conversion of facial images is fully demonstrated by the results based on qualitative visual comparisons.

关键词

Keywords

references

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C ] // Proceedings of the 27th International Conference on Neural Information Processing Systems . 2014 : 2672 - 2680 .

MIRZA M , OSINDERO S . Conditional generative adversarial nets [J ] . arXiv Preprint arXiv:1411.1784 , 2014 .

TANG H , XU D , YAN Y , et al . Local class-specific and global image-level generative adversarial networks for semantic-guided scene generation [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 7867 - 7876 .

CHEN B J , TAN W J , WANG Y T , et al . Distinguishing between natural and GAN-generated face images by combining global and local features [J ] . Chinese Journal of Electronics , 2022 , 31 ( 1 ): 59 - 67 .

GUO X , WANG Z , YANG Q , et al . Gan-based virtual-to-real image translation for urban scene semantic segmentation [J ] . Neurocomputing , 2020 , 394 : 127 - 135 .

KIM T , CHA M , KIM H , et al . Learning to discover cross-domain relations with generative adversarial networks [J ] . arXiv Preprint arXiv:1703.05192 , 2017 .

王旭东 , 卫红权 , 高超 , 等 . 身份保持约束下的人脸图像补全 [J ] . 网络与信息安全学报 , 2018 , 4 ( 8 ): 71 - 76 .

WANG X D , WEI H Q , GAO C , et al . Identity preserving face completion with generative adversarial networks [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 8 ): 71 - 76 .

TUNG H Y F , HARLEY A W , SETO W , et al . Adversarial inverse graphics networks:learning 2D-to-3D lifting and image-to-image translation from unpaired supervision [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 4364 - 4372 .

LI X Y , ZHANG S C , HU J , et al . Image-to-image translation via hierarchical style disentanglement [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 8635 - 8644 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 2242 - 2251 .

TOMEI M , CORNIA M , BARALDI L , et al . Art2Real:unfolding the reality of artworks via semantically-aware image-to-image translation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 5842 - 5852 .

ISOLA P , ZHU J Y , ZHOU T H , et al . Image-to-image translation with conditional adversarial networks [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 5967 - 5976 .

KINGMA D P , WELLING M . Auto-encoding variational bayes [J ] . arXiv Preprint arXiv:1312.6114 , 2013 .

MAKHZANI A , SHLENS J , JAITLY N , et al . Adversarial autoencoders" [J ] . arXiv Preprint arXiv:1511.05644 , 2015 .

KARRAS T , AILA , LAINE S , et al . Progressive growing of GANs for improved quality,stability,and variation [J ] . arXiv preprint arXiv:1710.10196 , 2017 .

KARRAS T , LAINE S , AILA T M . A style-based generator architecture for generative adversarial networks [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 4396 - 4405 .

DENTON E L , CHINTALA S , FERGUS R . Deep generative image models using a laplacian pyramid of adversarial networks [J ] . arXiv preprint arXiv:1506.05751 , 2015 .

MAO X D , LI Q , XIE H R , et al . Least Squares generative adversarial networks [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 2813 - 2821 .

ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein GAN [J ] . arXiv Preprint arXiv:1701.07875 , 2017 .

MIYATO T , KATAOKA T , KOYAMA M , et al . Spectral normalization for generative adversarial networks [J ] . arXiv Preprint arXiv:1802.05957 , 2018 .

RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks [J ] . arXiv Preprint arXiv:1511.06434 , 2015 .

CHEN X , DUAN Y , HOUTHOOFT R , et al . InfoGAN:interpretable representation learning by information maximizing generative adversarial nets [C ] // Proceedings of Neural Information Processing Systems . 2016 : 2172 - 2180 .

BROCK A , DONAHUE J , SIMONYAN K . Large scale GAN training for high fidelity natural image synthesis [J ] . arXiv Preprint arXiv:1809.11096 , 2018 .

TANG H , XU D , SEBE N , et al . Multi-channel attention selection GAN with cascaded semantic guidance for cross-view image translation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 2412 - 2421 .

PARK T , LIU M Y , WANG T C , et al . Semantic image synthesis with spatially-adaptive normalization [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 2332 - 2341 .

HUANG X , LIU M Y , BELONGIE S , et al . Multimodal unsupervised image-to-image translation [C ] // Proceedings of European Conference on Computer Vision . 2018 : 179 - 196 .

LEE H Y , TSENG H Y , MAO Q , et al . DRIT++:diverse image-to-image translation via disentangled representations [J ] . arXiv Preprint arXiv:1905.01270 , 2019 .

CHOI Y , CHOI M , KIM M , et al . StarGAN:unified generative adversarial networks for multi-domain image-to-image translation [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2018 : 8789 - 8797 .

GRASSUCCI E , SIGILLO L , UNCINI A , et al . Hyper complex image-to-image translation [J ] . arXiv Preprint arXiv:2205.02087 , 2022 .

DEMIREL H , ANBARJAFARI G . Discrete wavelet transform-based satellite image resolution enhancement [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2011 , 49 ( 6 ): 1997 - 2004 .

CHEN T S , LIN L , ZUO W M , et al . Learning a wavelet-like auto-encoder to accelerate deep neural networks [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2018 , 32 ( 1 ).

HUANG H B , HE R , SUN Z N , et al . Wavelet-SRNet:a wavelet-based CNN for multi-scale face super resolution [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 1698 - 1706 .

FUJIEDA S , TAKAYAMA K , HACHISUKA T . Wavelet convolutional neural networks for texture classification" [J ] . arXiv Preprint arXiv:1707.07394 , 2017 .

WILLIAMS T , LI R . Wavelet pooling for convolutional neural networks [C ] // Proceedings of International Conference on Learning Representations . 2018 .

RONNEBERGER O , FISCHER P , BROX T . U-Net:convolutional networks for biomedical image segmentation [C ] // Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention . 2015 : 234 - 241 .

HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 1510 - 1519 .

YOO J , UH Y , CHUN S , et al . Photorealistic style transfer via wavelet transforms [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . 2020 : 9035 - 9044 .

MAO Q , LEE H Y , TSENG H Y , et al . Mode seeking generative adversarial networks for diverse image synthesis [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 1429 - 1437 .

CHOI Y , UH Y , YOO J , et al . StarGAN v2:diverse image synthesis for multiple domains [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 8185 - 8194 .

HEUSEL M , RAMSAUER H , UNTERTHINER T , et al . GANs trained by a two time-scale update rule converge to a local nash equilibrium [C ] // Proceedings of Neural Information Processing Systems (NIPS) . 2017 : 6626 - 6637 .

ZHANG R , ISOLA P , EFROS A A , et al . The unreasonable effectiveness of deep features as a perceptual metric [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2018 : 586 - 595 .

Views

131

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

ISEM：image steganography enhancement model based on generative multi-adversarial network

Privacy-enhanced federated learning scheme based on generative adversarial networks

Survey of generative adversarial network

Differentially private sequence generative adversarial networks for data privacy masking

Performance evaluation for JPEG compression on model-based source camera identification

Related Author

ZHOU Saixing

YE Miaoxin

LUO Weiqi

Feng YU

Qingxin LIN

Hui LIN

Xiaoding WANG

Zhenglong WANG

Related Institution

School of Computer Science and Engineering, SunYat-sen University

Guangdong Key Laboratory of Information Security Technology

College of Computer and Cyber Security, Fujian Normal University

Engineering Research Center of Cyber Security and Education Informatization, Fujian Province University

Zhicheng College, Fuzhou University

⁰