Survey of generative adversarial network

Zhenglong WANG; Baowen ZHANG

doi:10.11959/j.issn.2096-109x.2021080

您当前的位置：

首页 >

文章列表页 >

Survey of generative adversarial network

TopicⅠ: Network Security: Attack and Defense | 更新时间：2024-06-03

- Survey of generative adversarial network
- Chinese Journal of Network and Information Security Vol. 7, Issue 4, Pages: 68-85(2021)
- 作者机构：
  
  1. 上海交通大学网络安全技术研究院，上海 200240
  2. 上海市信息安全综合管理技术研究重点实验室，上海 200240
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2021080
  CLC： TP183
- Online First：2021-08，
  
  Published：15 August 2021
- 稿件说明：
移动端阅览
Zhenglong WANG, Baowen ZHANG. Survey of generative adversarial network[J]. Chinese Journal of Network and Information Security, 2021, 7(4): 68-85.
DOI：

Zhenglong WANG, Baowen ZHANG. Survey of generative adversarial network[J]. Chinese Journal of Network and Information Security, 2021, 7(4): 68-85. DOI： 10.11959/j.issn.2096-109x.2021080.

摘要

首先介绍了生成对抗网络基本理论、应用场景和研究现状，并列举了其亟待改进的问题。围绕针对提升模型训练效率、提升生成样本质量和降低模式崩溃现象发生可能性3类问题的解决，从模型结构和训练过程两大改进方向和7个细分维度，对近年来生成对抗网络的主要研究工作、改进机理和特点进行了归纳和总结，并结合3方面对其未来的研究方向进行了探讨。

Abstract

Firstly

the basic theory

application scenarios and current state of research of GAN (generative adversarial network) were introduced

and the problems need to be improved were listed.Then

recent research

improvement mechanism and model features in 2 categories and 7 subcategories revolved around 3 points (improving model training efficiency

improving the quality of generated samples

and reducing the possibility of model collapse) were generalized and summarized.Finally

3 future research directions were discussed.

关键词

Keywords

references

SMOLENSKY P . Information processing in dynamical systems:Foundations of harmony theory [R ] . 1986 .

HINTON G , OSINDERO S , TEH Y W . A fast learning algorithm for deep belief nets [J ] . Neural Computation , 2006 18 ( 7 ): 1527 - 1554 .

SALAKHUTDINOV R , HINTON G . Deep boltzmann machines [C ] // Artificial Intelligence and Statistics . 2009 : 448 - 455 .

KINGMA D P , WELLING M . Auto-encoding variational bayes [J ] . arXiv preprint arXiv:1312.6114 , 2013 .

GOODFELLOW I , POUGET-ABADIE J ,, MIRZA M , et al . Generative adversarial nets [C ] // Advances in Neural Information Processing Systems . 2014 : 2672 - 2680 .

KOO S , . Automatic colorization with deep convolutional generative adversarial networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 2017 : 212 - 217 .

RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks [J ] . arXiv preprint arXiv:1511.06434 , 2015 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 2223 - 2232 .

BAO J , CHEN D , WEN F , et al . CVAE-GAN:fine-grained image generation through asymmetric training [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 2745 - 2754 .

SCHLEGL T , SEEBÖCK P ,, WALDSTEIN S M , et al . Unsupervised anomaly detection with generative adversarial networks to guide marker discovery [C ] // International Conference on Information Processing in Medical Imaging . 2017 : 146 - 157 .

ARJOVSKY M , CHINTALA S , BOTTOU L . WassersteinGAN [J ] . arXiv preprint arXiv:1701.07875 , 2017 .

WOLTERINK J M , LEINER T , ISGUM I . Blood vessel geometry synthesis using generative adversarial networks [J ] . arXiv preprint arXiv:1804.04381 , 2018 .

刘西蒙 , 谢乐辉 , 王耀鹏 , 等 . 深度学习中的对抗攻击与防御 [J ] . 网络与信息安全学报 , 2020 ( 5 ): 36 - 53 .

LIU X M , XIE L H , WANGY P , et al . Adversarial attacks and defenses in deep learning [J ] . Chinese Journal of Network and Information Security , 2020 , 6 ( 5 ): 36 - 53 .

张煜 , 吕锡香 , 邹宇聪 , 等 . 基于生成对抗网络的文本序列数据集脱敏 [J ] . 网络与信息安全学报 , 2020 ( 4 ): 109 - 119 .

ZHANG Y , LYU X X , ZOU Y C , et al . Differentially private sequence generative adversarial networks for data privacy masking [J ] . Chinese Journal of Network and Information Security , 2020 , 6 ( 4 ): 109 - 119 .

BEAULIEU-JONES B K , WU Z S , WILLIAMS C , et al . Privacy-preserving generative deep neural networks support clinical data sharing [J ] . Circulation:Cardiovascular Quality and Outcomes , 2019 , 12 ( 7 ): e005122 .

郭鹏 , 钟尚平 , 陈开志 , 等 . 差分隐私 GAN 梯度裁剪阈值的自适应选取方法 [J ] . 网络与信息安全学报 , 2018 , 4 ( 5 ): 10 - 20 .

GUO P , ZHONG S P , CHEN K J , et al . Adaptive selection method of differential privacy GAN gradient clipping thresholds [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 5 ): 10 - 20 .

王旭东 , 卫红权 , 高超 , 等 . 身份保持约束下的人脸图像补全 [J ] . 网络与信息安全学报 , 2018 , 4 ( 8 ): 71 - 76 .

WANG X D , WEI H Q , GAO C , et al . Identity preserving face completion with generative adversarial networks [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 8 ): 71 - 76 .

MIYATO T , KATAOKA T , KOYAMA M , et al . Spectral normalization for generative adversarial networks [J ] . arXiv preprint arXiv:1802.05957 , 2018 .

GOODFELLOW I . NIPS 2016 tutorial:generative adversarial networks [J ] . arXiv preprint arXiv:1701.00160 , 2016 .

MIRZA M , OSINDERO S . Conditional generative adversarial nets [J ] . arXiv preprint arXiv:1411.1784 , 2014 .

JAISWAL A , ABDALMAGEED W , WU Y , et al . Bidirectional conditional generative adversarial networks [C ] // Asian Conference on Computer Vision . 2018 : 216 - 232 .

PERARNAU G , VAN DE WEIJER J , RADUCANU B , et al . Invertible conditional GANs for image editing [J ] . arXiv preprint ar Xiv:1611.06355 , 2016 .

CHEN X , DUAN Y , HOUTHOOFT R , et al . Infogan:Interpretable representation learning by information maximizing generative adversarial nets [C ] // Advances in Neural Information Processing Systems . 2016 : 2172 - 2180 .

GURUMURTHY S , KIRAN SARVADEVABHATLA R , VENKATESH BABU R . DeLiGAN:Generative adversarial networks for diverse and limited data [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017 : 166 - 174 .

MISHRA D , JAYENDRAN A , SRIVASTAVA V , et al . Mode matching in GANs through latent space learning and inversion [J ] . arXiv preprint arXiv:1811.03692 , 2018 .

KWAK H , ZHANG B T . Ways of conditioning generative adversarial networks [J ] . arXiv preprint arXiv:1611.01455 , 2016 .

ODENA A . Semi-supervised learning with generative adversarial networks [J ] . arXiv preprint arXiv:1606.01583 , 2016 .

ODENA A , OLAH C , SHLENS J . Conditional image synthesis with auxiliary classifier GANs [C ] // International Conference on Machine Learning . 2017 : 2642 - 2651 .

TOLSTIKHIN I O , GELLY S , BOUSQUET O , et al . AdaGAN:boosting generative models [C ] // Advances in Neural Information Processing Systems . 2017 : 5424 - 5433 .

GHOSH A , KULHARIA V , NAMBOODIRI V P , et al . Multi-agent diverse generative adversarial networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 8513 - 8521 .

HOANG Q , NGUYEN T D , LE T , et al . MGAN:training generative adversarial nets with multiple generators [C ] // International Conference on Learning Representations . 2018 .

GHOSH A , KULHARIA V , NAMBOODIRI V . Message passing multi-agent GANs [J ] . arXiv preprint arXiv:1612.01294 , 2016 .

LIN Z , KHETAN A , FANTI G , et al . PacGAN:The power of two samples in generative adversarial networks [J ] . IEEE Journal on Selected Areas in Information Theory , 2020 , 1 ( 1 ): 324 - 335 .

DURUGKAR I , GEMP I , MAHADEVAN S . Generative multi-adversarial networks [J ] . arXiv preprint arXiv:1611.01673 , 2016 .

MORDIDO G , YANG H , MEINEL C . Dropout-GAN:learning from a dynamic ensemble of discriminators [J ] . arXiv preprint arXiv:1807.11346 , 2018 .

NGUYEN T , LE T , VU H , et al . Dual discriminator generative adversarial nets [C ] // Advances in Neural Information Processing Systems . 2017 : 2670 - 2680 .

NEYSHABUR B , BHOJANAPALLI S , CHAKRABARTI A . Stabilizing GAN training with multiple random projections [J ] . arXiv preprint arXiv:1705.07831 , 2017 .

ZHAO J , MATHIEU M , LECUN Y . Energy-based generative adversarial network [J ] . arXiv preprint arXiv:1609.03126 , 2016 .

BERTHELOT D , SCHUMM T , METZ L . BeGAN:boundary equilibrium generative adversarial networks [J ] . arXiv preprint arXiv:1703.10717 , 2017 .

IM D J , KIM C D , JIANG H , et al . Generating images with recurrent adversarial networks [J ] . arXiv preprint arXiv:1602.05110 , 2016 .

ZHANG H , XU T , LI H , et al . StackGAN:text to photo-realistic image synthesis with stacked generative adversarial networks [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 5907 - 5915 .

KARRAS T , AILA T , LAINE S , et al . Progressive growing of gans for improved quality,stability,and variation [J ] . arXiv preprint arXiv:1710.10196 , 2017 .

LI C , XU T , ZHU J , et al . Triple generative adversarial nets [J ] . Advances in Neural Information Processing Systems , 2017 , 30 : 4088 - 4098 .

LEE M , SEOK J . Controllable generative adversarial network [J ] . Ieee Access , 2019 , 7 : 28158 - 28169 .

CHAVDAROVA T , FLEURET F . SGAN:An alternative training of generative adversarial networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 9407 - 9415 .

KIM Y , KIM M , KIM G . Memorization precedes generation:Learning unsupervised gans with memory networks [J ] . arXiv preprint arXiv:1803.01500 , 2018 .

JAISWAL A , ABDALMAGEED W , WU Y , et al . CapsuleGAN:Generative adversarial capsule network [C ] // Proceedings of the European Conference on Computer Vision (ECCV) . 2018 .

LARSEN A B L , SØNDERBY S K ,, LAROCHELLE H , et al . Autoencoding beyond pixels using a learned similarity metric [C ] // International Conference on Machine Learning . 2016 : 1558 - 1566 .

ZHONG G , GAO W , LIU Y , et al . Generative adversarial networks with decoder-encoder output noises [J ] . Neural Networks , 2020 .

MAKHZANI A , SHLENS J , JAITLY N , et al . Adversarial autoencoders [J ] . arXiv preprint arXiv:1511.05644 , 2015 .

DONAHUE J , KRÄHENBÜHL P ,, DARRELL T . Adversarial feature learning [J ] . arXiv preprint arXiv:1605.09782 , 2016 .

DUMOULIN V , BELGHAZI I , POOLE B , et al . Adversarially learned inference [J ] . arXiv preprint arXiv:1606.00704 , 2016 .

MÁTTYUS G , URTASUN R . Matching adversarial networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 8024 - 8032 .

ZHANG H , GOODFELLOW I , METAXAS D , et al . Self-attention generative adversarial networks [C ] // International Conference on Machine Learning . 2019 : 7354 - 7363 .

WANG X , ZHANG R , SUN Y , et al . KdGAN:Knowledge distillation with generative adversarial networks [C ] // Advances in Neural Information Processing Systems . 2018 : 775 - 786 .

WANG J , YU L , ZHANG W , et al . IrGAN:a minimax game for unifying generative and discriminative information retrieval models [C ] // Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval . 2017 : 515 - 524 .

DENTON E L , CHINTALA S , FERGUS R . Deep generative image models using a laplacian pyramid of adversarial networks [C ] // Advances in Neural Information Processing Systems . 2015 : 1486 - 1494 .

LLOYD S , WEEDBROOK C . Quantum generative adversarial learning [J ] . Physical Review Letters , 2018 , 121 ( 4 ).

SAATCI Y , WILSON A G . Bayesian GAN [C ] // Advances in Neural Information Processing Systems . 2017 : 3622 - 3631 .

NOWOZIN S , CSEKE B , TOMIOKA R . f-GAN:training generative neural samplers using variational divergence minimization [C ] // Advances in Neural Information Processing Systems . 2016 : 271 - 279 .

MAO X , LI Q , XIE H , et al . Least squares generative adversarial networks [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2017 : 2794 - 2802 .

GULRAJANI I , AHMED F , ARJOVSKY M , et al . Improved training of WassersteinGANs [C ] // Advances in Neural Information Processing Systems . 2017 : 5767 - 5777 .

PETZKA H , FISCHER A , LUKOVNICOV D . On the regularization of WassersteinGANs [J ] . arXiv preprint arXiv:1709.08894 , 2017 .

ADLER J , LUNZ S . Banach WassersteinGAN [C ] // Advances in Neural Information Processing Systems . 2018 : 6754 - 6763 .

WEI X , GONG B , LIU Z , et al . Improving the improved training of WassersteinGANs:a consistency term and its dual effect [J ] . arXiv preprint arXiv:1803.01541 , 2018 .

GUO X , HONG J , LIN T , et al . Relaxed wasserstein with applications to GANs [J ] . arXiv preprint arXiv:1705.07164 , 2017 .

BELLEMARE M G , DANIHELKA I , DABNEY W , et al . The cramer distance as a solution to biased wasserstein gradients [J ] . arXiv preprint arXiv:1705.10743 , 2017 .

WU J , HUANG Z , THOMA J , et al . Wasserstein divergence for GANs [C ] // Proceedings of the European Conference on Computer Vision (ECCV) . 2018 : 653 - 668 .

MROUEH Y , SERCU T , GOEL V . McGAN:Mean and covariance feature matching GAN [J ] . arXiv preprint arXiv:1702.08398 , 2017 .

LI Y , SWERSKY K , ZEMEL R . Generative moment matching networks [C ] // International Conference on Machine Learning . 2015 : 1718 - 1727 .

LI C L , CHANG W C , CHENG Y , et al . MmdGAN:towards deeper understanding of moment matching network [C ] // Advances in Neural Information Processing Systems . 2017 : 2203 - 2213

MROUEH Y , SERCU T . Fisher GAN [C ] // Advances in Neural Information Processing Systems . 2017 : 2513 - 2523 .

SALIMANS T , GOODFELLOW I , ZAREMBA W , et al . Improved techniques for training GANS [C ] // Advances in Neural Information Processing Systems . 2016 : 2234 - 2242 .

ARORA S , GE R , LIANG Y , et al . Generalization and equilibrium in generative adversarial nets (GANs) [J ] . arXiv preprint arXiv:1703.00573 , 2017 .

SALIMANS T , ZHANG H , RADFORD A , et al . Improving GANs using optimal transport [J ] . arXiv preprint arXiv:1803.05573 , 2018 .

PARK N , ANAND A , MONIZ J R A , et al . MMGAN:manifold-Matching generative adversarial networks [C ] // 2018 24th International Conference on Pattern Recognition (ICPR) . 2018 : 1343 - 1348 .

WANG R , CULLY A , CHANG H J , et al . MaGAN:margin adaptation for generative adversarial networks [J ] . arXiv preprint arXiv:1704.03817 , 2017 .

LIN M . SoftmaxGAN [J ] . arXiv preprint arXiv:1704.06191 , 2017 .

JOLICOEUR-MARTINEAU A . The relativistic discriminator:a key element missing from standard GAN [J ] . arXiv preprint ar Xiv:1807.00734 , 2018 .

MÜLLER A . Integral probability metrics and their generating classes of functions [J ] . Advances in Applied Probability , 1997 : 429 - 443 .

EVANS L C . Partial differential equations and monge-kantorovich mass transfer [J ] . Current Developments in Mathematics , 1997 , 1997 ( 1 ): 65 - 126 .

BORJI A . Pros and cons of GAN evaluation measures [J ] . Computer Vision and Image Understanding , 2019 , 179 : 41 - 65 .

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2818 - 2826 .

CHE T , LI Y , JACOB A P , et al . Mode regularized generative adversarial networks [J ] . arXiv preprint arXiv:1612.02136 , 2016 .

ZHOU Z , CAI H , RONG S , et al . Activation maximization generative adversarial nets [J ] . arXiv preprint arXiv:1703.02000 , 2017 .

BARRATT S , SHARMA R . A note on the inception score [J ] . arXiv preprint arXiv:1801.01973 , 2018 .

HEUSEL M , RAMSAUER H , UNTERTHINER T , et al . GANs trained by a two time-scale update rule converge to a local nash equilibrium [C ] // Advances in Neural Information Processing Systems . 2017 : 6626 - 6637 .

LOPEZ-PAZ D , OQUAB M . Revisiting classifier two-sample tests [J ] . arXiv preprint arXiv:1610.06545 , 2016 .

SHMELKOV K , SCHMID C , ALAHARI K . How good is my GAN [C ] // Proceedings of the European Conference on Computer Vision (ECCV) . 2018 : 213 - 229 .

ZHANG Z , SONG Y , QI H . Decoupled learning for conditional adversarial networks [C ] // 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . 2018 : 700 - 708 .

QI G J , ZHANG L , HU H , et al . Global versus localized generative adversarial nets [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 : 1517 - 1525 .

LEI N , GUO Y , AN D , et al . Mode collapse and regularity of optimal transportation maps [J ] . arXiv preprint arXiv:1902.02934 , 2019 .

LIU K , TANG W , ZHOU F , et al . Spectral regularization for combating mode collapse in GANs [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2019 : 6382 - 6390 .

GUAN S , LOEW M , KO H . Data separability for neural network classifiers and the development of a separability index [J ] . arXiv preprint arXiv:2005.13120 , 2020 .

MO S , CHO M , SHIN J . Freeze discriminator:a simple baseline for fine-tuning GANs [J ] . arXiv preprint arXiv:2002.10964 , 2020 .

ROBB E , CHU W S , KUMAR A , et al . Few-shot adaptation of generative adversarial networks [J ] . arXiv preprint arXiv:2010.11943 , 2020 .

ZHANG R , CHE T , GHAHRAMANI Z , et al . MetaGAN:an adversarial approach to few-shot learning [C ] // Advances in Neural Information Processing Systems . 2018 : 2365 - 2374 .

NGUYEN K , TODOROVIC S . A self-supervised GAN for unsupervised few-shot object recognition [J ] . arXiv preprint arXiv:2008.06982 , 2020 .

KARRAS T , AITTALA M , HELLSTEN J , et al . Training generative adversarial networks with limited data [J ] . Advances in Neural Information Processing Systems , 2020 ,33.

KANEKO T , USHIKU Y , HARADA T . Label-noise robust generative adversarial networks [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2019 : 2467 - 2476 .

KANEKO T , HARADA T . Noise robust generative adversarial networks [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2020 : 8404 - 8414 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [C ] // 2nd International Conference on Learning Representations,ICLR 2014 . 2014 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [J ] . Stat,2015 , 1050 :20.

BAI T , LUO J , ZHAO J . Recent advances in understanding adversarial robustness of deep neural networks [J ] . arXiv preprint arXiv:2011.01539 , 2020 .

BASTANI O , IOANNOU Y , LAMPROPOULOS L , et al . Measuring neural net robustness with constraints [C ] // Advances in Neural Information Processing Systems . 2016 : 2613 - 2621 .

WANG S , PEI K , WHITEHOUSE J , et al . Efficient formal safety analysis of neural networks [C ] // Advances in Neural Information Processing Systems . 2018 : 6367 - 6377 .

Views

994

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Phishing website detection based on two-way feature extraction and adaptive fusion

Anomaly detection in DNS: an empirical study

ISEM：image steganography enhancement model based on generative multi-adversarial network

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

Multi-task adversarial attribution method based on hierarchical structure

Related Author

Guo Kai

Wang Heng

Ma Ziqiang

HOU Ronghao

HE Ming

KONG Kaichuan

CHEN Yong

ZUO Peng

Related Institution

School of Information Engineering, Ningxia University

China Internet Network Information Center

School of Cyber Science and Engineering, Jinan University

Guangdong Key Laboratory of Information Security Technology

School of Computer Science and Engineering, SunYat-sen University

⁰