融合小波快捷连接生成对抗网络的面部性别伪造

陈万泽; 黄丽清; 陈家祯; 叶锋; 黄添强; 罗海峰

doi:10.11959/j.issn.2096-109x.2023046

您当前的位置：

首页 >

文章列表页 >

融合小波快捷连接生成对抗网络的面部性别伪造

学术论文 | 更新时间：2024-06-03

- 融合小波快捷连接生成对抗网络的面部性别伪造
- Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network
- 网络与信息安全学报 2023年9卷第3期页码：150-160
- 作者机构：
- 作者简介：
  
  [ "陈万泽（1995- ），男，甘肃兰州人，福建师范大学硕士生，主要研究方向为深度学习、生成对抗网络、图像生成" ]
  [ "黄丽清（1991- ），女，福建莆田人，福建师范大学讲师，主要研究方向为视频图像超分辨率、多媒体内容安全" ]
  [ "陈家祯（1971- ），女，福建福州人，福建师范大学副教授，主要研究方向为信息安全" ]
  [ "叶锋（1978- ），男，福建福州人，福建师范大学教授，主要研究方向为视频通信、视频图像处理" ]
  [ "黄添强（1971- ），男，福建仙游人，福建师范大学教授、博士生导师，主要研究方向为人工智能、大数据分析、数字媒体内容安全理论和技术" ]
  [ "罗海峰（1990- ），男，福建龙岩人，福建师范大学讲师，主要研究方向为二维/三维计算机视觉" ]
- 基金信息：
  
  国家自然科学基金;The National Natural Science Foundation of China(62072106);福建省自然科学基金;Fujian Province Natural Science Foundation(2020J01168);福建省教育厅（A类）;Fujian Provincial Department of Education (Rank A)(JAT210053)
- DOI：10.11959/j.issn.2096-109x.2023046
  中图分类号： TP37；TP391
- 网络首发：2023-06，
  
  纸质出版：2023-06-25
- 稿件说明：
移动端阅览
陈万泽, 黄丽清, 陈家祯, 等. 融合小波快捷连接生成对抗网络的面部性别伪造[J]. 网络与信息安全学报, 2023,9(3):150-160.

Wanze CHEN, Liqing HUANG, Jiazhen CHEN, et al. Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 150-160.
陈万泽, 黄丽清, 陈家祯, 等. 融合小波快捷连接生成对抗网络的面部性别伪造[J]. 网络与信息安全学报, 2023,9(3):150-160. DOI： 10.11959/j.issn.2096-109x.2023046.

Wanze CHEN, Liqing HUANG, Jiazhen CHEN, et al. Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 150-160. DOI： 10.11959/j.issn.2096-109x.2023046.

摘要

面部属性编辑领域的主流方法由于数据以及模型架构的限制，存在如下两方面的问题：自编码器模型在编码以及解码阶段的瓶颈结构会丢失特征信息，以及在解码过程中逐阶段对源域特征进行连续的样式注入，使得生成图像中目标域信息占比过大，导致生成图像丢失了源域的身份信息以及部分细节；在人脸图像中，图像人物的性别、种族或年龄等属性的差异会使得不同图像的频域信息组成差别较大，在无监督的训练前提下，当前主流网络框架并不能在样式注入阶段调整源域信息以及目标域信息之间的占比，导致生成图像仍然存在伪影。针对上述问题，基于生成对抗网络和图像到图像的翻译提出一种面部性别伪造模型，即融合快捷小波连接生成对抗网络（WscGAN）。通过对自编码器结构添加快捷连接，将不同编码阶段的输出通过小波变换进行特征级别的分解，再引入通道注意力机制将其进行逐个处理，以此动态改变在解码过程中不同频率的源域特征信息的比例，最终实现面部图像在性别属性上的伪造。为了验证所提模型的有效性，WscGAN 分别在 CelebA-HQ 数据集和 FFHQ 数据集上进行了实验。实验结果表明，WscGAN 在 CelebA-HQ 以及 FFHQ 数据集上均领先现有先进模型，其中弗雷歇起始距离分别提升5.4%和19.8%，特征度量相似度分别提升1.8%和4.1%。此外，定性的视觉对比结果充分体现了WscGAN可有效改善面部图像性别属性伪造的效果。

Abstract

The mainstream methods in the field of facial attribute manipulation had the following two defects due to data and model architecture limitations.First

the bottleneck structure of the autoencoder model results in the loss of feature information

and the traditional method of continuously injected styles to the source domain features during the decoding process makes the generated image too referential to the target domain while losing the identity information and fine-grained details.Second

differences in facial attributes composition between images

such as gender

ethnicity

or age can cause variations in frequency domain information.And the current unsupervised training methods do not automatically adjust the proportion of source and target domain information in the style injection stage

resulting in artifacts in generated images.A facial gender forgery model based on generative adversarial networks and image-to-image translation techniques

namely fused wavelet shortcut connection generative adversarial network (WscGAN)

was proposed to address the these issues.Shortcut connections were added to the autoencoder structure

and the outputs of different encoding stages were decomposed at the feature level by wavelet transform.Attention mechanism was employed to process them one by one

to dynamically change the proportion of source domain features at different frequencies in the decoding process.This model could complete forgery of facial images in terms of gender attributes.To verify the effectiveness of the model

it was conducted on the CelebA-HQ dataset and the FFHQ dataset.Compared with the existing optimal models

the method improves the FID and LPIPS indices by 5.4% and 11.2%

and by 1.8% and 6.7%

respectively.Furthermore

the effectiveness of the proposed method in improving the gender attribute conversion of facial images is fully demonstrated by the results based on qualitative visual comparisons.

关键词

Keywords

references

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C ] // Proceedings of the 27th International Conference on Neural Information Processing Systems . 2014 : 2672 - 2680 .

MIRZA M , OSINDERO S . Conditional generative adversarial nets [J ] . arXiv Preprint arXiv:1411.1784 , 2014 .

TANG H , XU D , YAN Y , et al . Local class-specific and global image-level generative adversarial networks for semantic-guided scene generation [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 7867 - 7876 .

CHEN B J , TAN W J , WANG Y T , et al . Distinguishing between natural and GAN-generated face images by combining global and local features [J ] . Chinese Journal of Electronics , 2022 , 31 ( 1 ): 59 - 67 .

GUO X , WANG Z , YANG Q , et al . Gan-based virtual-to-real image translation for urban scene semantic segmentation [J ] . Neurocomputing , 2020 , 394 : 127 - 135 .

KIM T , CHA M , KIM H , et al . Learning to discover cross-domain relations with generative adversarial networks [J ] . arXiv Preprint arXiv:1703.05192 , 2017 .

王旭东 , 卫红权 , 高超 , 等 . 身份保持约束下的人脸图像补全 [J ] . 网络与信息安全学报 , 2018 , 4 ( 8 ): 71 - 76 .

WANG X D , WEI H Q , GAO C , et al . Identity preserving face completion with generative adversarial networks [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 8 ): 71 - 76 .

TUNG H Y F , HARLEY A W , SETO W , et al . Adversarial inverse graphics networks:learning 2D-to-3D lifting and image-to-image translation from unpaired supervision [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 4364 - 4372 .

LI X Y , ZHANG S C , HU J , et al . Image-to-image translation via hierarchical style disentanglement [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2021 : 8635 - 8644 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 2242 - 2251 .

TOMEI M , CORNIA M , BARALDI L , et al . Art2Real:unfolding the reality of artworks via semantically-aware image-to-image translation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 5842 - 5852 .

ISOLA P , ZHU J Y , ZHOU T H , et al . Image-to-image translation with conditional adversarial networks [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 5967 - 5976 .

KINGMA D P , WELLING M . Auto-encoding variational bayes [J ] . arXiv Preprint arXiv:1312.6114 , 2013 .

MAKHZANI A , SHLENS J , JAITLY N , et al . Adversarial autoencoders" [J ] . arXiv Preprint arXiv:1511.05644 , 2015 .

KARRAS T , AILA , LAINE S , et al . Progressive growing of GANs for improved quality,stability,and variation [J ] . arXiv preprint arXiv:1710.10196 , 2017 .

KARRAS T , LAINE S , AILA T M . A style-based generator architecture for generative adversarial networks [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 4396 - 4405 .

DENTON E L , CHINTALA S , FERGUS R . Deep generative image models using a laplacian pyramid of adversarial networks [J ] . arXiv preprint arXiv:1506.05751 , 2015 .

MAO X D , LI Q , XIE H R , et al . Least Squares generative adversarial networks [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 2813 - 2821 .

ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein GAN [J ] . arXiv Preprint arXiv:1701.07875 , 2017 .

MIYATO T , KATAOKA T , KOYAMA M , et al . Spectral normalization for generative adversarial networks [J ] . arXiv Preprint arXiv:1802.05957 , 2018 .

RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks [J ] . arXiv Preprint arXiv:1511.06434 , 2015 .

CHEN X , DUAN Y , HOUTHOOFT R , et al . InfoGAN:interpretable representation learning by information maximizing generative adversarial nets [C ] // Proceedings of Neural Information Processing Systems . 2016 : 2172 - 2180 .

BROCK A , DONAHUE J , SIMONYAN K . Large scale GAN training for high fidelity natural image synthesis [J ] . arXiv Preprint arXiv:1809.11096 , 2018 .

TANG H , XU D , SEBE N , et al . Multi-channel attention selection GAN with cascaded semantic guidance for cross-view image translation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 2412 - 2421 .

PARK T , LIU M Y , WANG T C , et al . Semantic image synthesis with spatially-adaptive normalization [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . 2020 : 2332 - 2341 .

HUANG X , LIU M Y , BELONGIE S , et al . Multimodal unsupervised image-to-image translation [C ] // Proceedings of European Conference on Computer Vision . 2018 : 179 - 196 .

LEE H Y , TSENG H Y , MAO Q , et al . DRIT++:diverse image-to-image translation via disentangled representations [J ] . arXiv Preprint arXiv:1905.01270 , 2019 .

CHOI Y , CHOI M , KIM M , et al . StarGAN:unified generative adversarial networks for multi-domain image-to-image translation [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2018 : 8789 - 8797 .

GRASSUCCI E , SIGILLO L , UNCINI A , et al . Hyper complex image-to-image translation [J ] . arXiv Preprint arXiv:2205.02087 , 2022 .

DEMIREL H , ANBARJAFARI G . Discrete wavelet transform-based satellite image resolution enhancement [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2011 , 49 ( 6 ): 1997 - 2004 .

CHEN T S , LIN L , ZUO W M , et al . Learning a wavelet-like auto-encoder to accelerate deep neural networks [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2018 , 32 ( 1 ).

HUANG H B , HE R , SUN Z N , et al . Wavelet-SRNet:a wavelet-based CNN for multi-scale face super resolution [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 1698 - 1706 .

FUJIEDA S , TAKAYAMA K , HACHISUKA T . Wavelet convolutional neural networks for texture classification" [J ] . arXiv Preprint arXiv:1707.07394 , 2017 .

WILLIAMS T , LI R . Wavelet pooling for convolutional neural networks [C ] // Proceedings of International Conference on Learning Representations . 2018 .

RONNEBERGER O , FISCHER P , BROX T . U-Net:convolutional networks for biomedical image segmentation [C ] // Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention . 2015 : 234 - 241 .

HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . 2017 : 1510 - 1519 .

YOO J , UH Y , CHUN S , et al . Photorealistic style transfer via wavelet transforms [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . 2020 : 9035 - 9044 .

MAO Q , LEE H Y , TSENG H Y , et al . Mode seeking generative adversarial networks for diverse image synthesis [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 1429 - 1437 .

CHOI Y , UH Y , YOO J , et al . StarGAN v2:diverse image synthesis for multiple domains [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 2020 : 8185 - 8194 .

HEUSEL M , RAMSAUER H , UNTERTHINER T , et al . GANs trained by a two time-scale update rule converge to a local nash equilibrium [C ] // Proceedings of Neural Information Processing Systems (NIPS) . 2017 : 6626 - 6637 .

ZHANG R , ISOLA P , EFROS A A , et al . The unreasonable effectiveness of deep features as a perceptual metric [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2018 : 586 - 595 .

浏览量

131

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

ISEM：基于生成多对抗网络的图像隐写增强模型

基于生成对抗网络的隐私增强联邦学习方案

生成对抗网络研究综述

基于生成对抗网络的文本序列数据集脱敏

JPEG压缩对相机型号来源取证的影响分析