Survey of artificial intelligence data security and privacy protection

Kui REN; Quanrun MENG; Shoukun YAN; Zhan QIN

doi:10.11959/j.issn.2096-109x.2021001

您当前的位置：

首页 >

文章列表页 >

Survey of artificial intelligence data security and privacy protection

Comprehensive Review | 更新时间：2024-06-03

- Survey of artificial intelligence data security and privacy protection
- Chinese Journal of Network and Information Security Vol. 7, Issue 1, Pages: 1-10(2021)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2021001
  CLC： TP393
- Online First：2021-02，
  
  Published：15 February 2021
- 稿件说明：
移动端阅览
Kui REN, Quanrun MENG, Shoukun YAN, et al. Survey of artificial intelligence data security and privacy protection[J]. Chinese Journal of Network and Information Security, 2021, 7(1): 1-10.
DOI：

Kui REN, Quanrun MENG, Shoukun YAN, et al. Survey of artificial intelligence data security and privacy protection[J]. Chinese Journal of Network and Information Security, 2021, 7(1): 1-10. DOI： 10.11959/j.issn.2096-109x.2021001.

摘要

人工智能和深度学习算法正在高速发展，这些新兴技术在音视频识别、自然语言处理等领域已经得到了广泛应用。然而，近年来研究者发现，当前主流的人工智能模型中存在着诸多安全隐患，并且这些隐患会限制人工智能技术的进一步发展。因此，研究了人工智能模型中的数据安全与隐私保护问题。对于数据与隐私泄露问题，主要研究了基于模型输出的数据泄露问题和基于模型更新的数据泄露问题。在基于模型输出的数据泄露问题中，主要探讨了模型窃取攻击、模型逆向攻击、成员推断攻击的原理和研究现状；在基于模型更新的数据泄露问题中，探讨了在分布式训练过程中，攻击者如何窃取隐私数据的相关研究。对于数据与隐私保护问题，主要研究了常用的3类防御方法，即模型结构防御，信息混淆防御，查询控制防御。综上，围绕人工智能深度学习模型的数据安全与隐私保护领域中最前沿的研究成果，探讨了人工智能深度学习模型的数据窃取和防御技术的理论基础、重要成果以及相关应用。

Abstract

Artificial intelligence and deep learning algorithms are developing rapidly.These emerging techniques have been widely used in audio and video recognition

natural language processing and other fields.However

in recent years

researchers have found that there are many security risks in the current mainstream artificial intelligence model

and these problems will limit the development of AI.Therefore

the data security and privacy protection was studied in AI.For data and privacy leakage

the model output based and model update based problem of data leakage were studied.In the model output based problem of data leakage

the principles and research status of model extraction attack

model inversion attack and membership inference attack were discussed.In the model update based problem of data leakage

how attackers steal private data in the process of distributed training was discussed.For data and privacy protection

three kinds of defense methods

namely model structure defense

information confusion defense and query control defense were studied.In summarize

the theoretical foundations

classic algorithms of data inference attack techniques were introduced.A few research efforts on the defense techniques were described in order to provoke further research efforts in this critical area.

关键词

Keywords

references

ATENIESE G , MANCINI L V , SPOGNARDI A , et al . Hacking smart machines with smarter ones:how to extract meaningful data from machine learning classifiers [J ] . International Journal of Security and Networks , 2015 , 10 ( 3 ): 137 - 150 .

JUUTI M , SZYLLER S , MARCHAL S , et al . PRADA:protecting against DNN model stealing attacks [C ] // In IEEE European Symposium on Security and Privacy . 2019 : 512 - 527 .

YANG Q , LIU Y , CHEN T , et al . Federated machine learning:concept and applications [J ] . ACM Transactions on Intelligent Systems and Technology (TIST) , 2019 , 10 ( 2 ): 1 - 19 .

PAPERNOT N , MCDANIEL P D , GOODFELLOW I J , et al . Practical black-box attacks against machine learning [C ] // In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security . 2017 : 506 - 519 .

TRAMÈR F , ZHANG F , JUELS A , et al . Stealing machine learning models via prediction APIs [C ] // In 25th USENIX Security Symposium,USENIX Security 16 . 2016 : 601 - 618 .

WANG B H , GONG N Z . Stealing hyperparameters in machine learning [C ] // In 2018 IEEE Symposium on Security and Privacy . 2018 : 36 - 52 .

OH S J , SCHIELE B , FRITZ M . Towards reverse-engineering black-box neural networks [J ] . arXiv:1711.01768 , 2019 .

SATHISH K , RAMASUBBAREDDY S , GOVINDA K . Detection and localization of multiple objects using VGGNet and single shot detection [M ] // Emerging Research in Data Engineering Systems and Computer Communications.Singapore:Springer . 2020 : 427 - 439 .

TARG S , ALMEIDA D , LYMAN K . Resnet in resnet:generalizing residual architectures [J ] . arXiv preprint arXiv:1603.08029 , 2016 .

CORREIA-SILVA J R , BERRIEL R F , BADUE C , et al . Copycat CNN:stealing knowledge by persuading confession with random non-labeled data [C ] // In 2018 International Joint Conference on Neural Networks . 2018 : 1 - 8 .

BATINA L,BHASINS , JAP D , et al . CSI NN:reverse engineering of neural network architectures through electromagnetic side channel [C ] // In 28th USENIX Security Symposium,USENIX Security 2019 . 2019 : 515 - 532 .

YU H G , YANG K C , ZHANG T , et al . Cloudleak:large-scale deep learning models stealing through adversarial examples [C ] // Network and Distributed System Security Symposium . 2020 .

FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures [C ] // In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security . 2015 : 1322 - 1333 .

JANG E , GU S , POOLE B . Categorical reparameterization with gumbel-softmax [J ] . arXiv preprint arXiv:1611.01144 , 2016 .

SHOKRI R , STRONATI M , SONG C Z , et al . Membership inference attacks against machine learning models [C ] // In 2017 IEEE Symposium on Security and Privacy . 2017 : 3 - 18 .

YEOM S , GIACOMELLI I , FREDRIKSON M , et al . Privacy risk in machine learning:analyzing the connection to overfitting [C ] // In 31st IEEE Computer Security Foundations Symposium . 2018 : 268 - 282 .

SALEM A , ZHANG Y , HUMBERT M , et al . Ml-leaks:model and data independent membership inference attacks and defenses on machine learning models [C ] // In 26th Annual Network and Distributed System Security Symposium . 2019 : 24 - 27 .

LONG Y H , BINDSCHAEDLER V , GUNTER C A . Towards measuring membership privacy [J ] . CoRR,abs/1712.09136 , 2017 .

LONG Y H , BINDSCHAEDLER V , WANG L , et al . Understanding membership inferences on well-generalized learning models [J ] . CoRR,abs/1802.04889 , 2018 .

YEOM S , FREDRIKSON M , JHA S . The unintended consequences of overfitting:Training data inference attacks [J ] . CoRR,abs/1709.01604 , 2017 .

SAM D B , SURYA S , BABU R V . Switching convolutional neural network for crowd counting [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 2017 : 4031 - 4039 .

KUHA J , MILLS C . On group comparisons with logistic regression models [J ] . Sociological Methods ＆ Research , 2020 , 49 ( 2 ): 498 - 525 .

PAL M . Random forest classifier for remote sensing classification [J ] . International journal of remote sensing , 2005 , 26 ( 1 ): 217 - 222 .

SONG L , SHOKRI R , MITTAL P . Privacy risks of securing machine learning models against adversarial examples [C ] // Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security . 2019 : 241 - 257 .

SALEM A , BHATTACHARYA A , BACKES M , et al . Updates-leak:data set inference and reconstruction attacks in online learning [J ] . arXiv preprint arXiv:1904.01067 , 2019 .

HAYES J , MELIS L , DANEZIS G , et al . LOGAN:membership inference attacks against generative models [J ] . PoPETs , 2019 ( 1 ): 133 - 152 .

NASR M , SHOKRI R , HOUMANSADR A . Comprehensive privacy analysis of deep learning:passive and active white-box inference attacks against centralized and federated learning [C ] // In 2019 IEEE Symposium on Security and Privacy . 2019 : 739 - 753 .

LEINO K , FREDRIKSON M . Stolen memories:leveraging model memorization for calibrated white-box membership inference [J ] . arXiv preprint arXiv:1906.11798 , 2019 .

MELIS L , SONG C Z , CRISTOFARO E D , et al . Exploiting unintended feature leakage in collaborative learning [C ] // In 2019 IEEE Symposium on Security and Privacy . 2019 : 691 - 06 .

WANG Z B,SONG M K , Zhang Z F , Yet al . Beyond inferring class representatives:user-level privacy leakage from federated learning [C ] // In 2019 IEEE conference on Computer Communications . 2019 : 2512 - 2520 .

HITAJ B , ATENIESE G,PÉREZ-CRUZ F . Deep models under the GAN:information leakage from collaborative deep learning [C ] // In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 603 - 618 .

ZHU L G , LIU Z J , HAN S . Deep leakage from gradients [C ] // In Advances in Neural Information Processing Systems Annual Conference on Neural Information Processing Systems 2019 . 2019 : 14747 - 14756 .

NASR M , SHOKRI R , HOUMANSADR A . Machine learning with membership privacy using adversarial regularization [C ] // In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security . 2018 : 634 - 646 .

WANG C , LIU G Y , HUANG H J , et al . MIASec:enabling data indistinguishability against membership inference attacks in MLaaS [J ] . IEEE Transactions on Sustainable Computing , 2020 , 5 ( 3 ): 365 - 376 .

WU N , FAROKHI F , SMITH D , et al . The Value of collaboration in convex machine learning with differential privacy [J ] . IEEE Symposium on Security and Privacy , 2020 : 304 - 317 .

PATRA A , SURESH A . BLAZE:blazing fast privacy-preserving machine learning [J ] . arXiv preprint arXiv:2005.09042 , 2020 .

JIA J Y , SALEM A , BACKES M , et al . MemGuard:defending against black-box membership inference attacks via adversarial examples [C ] // In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security,CCS 2019 . 2019 : 259 - 274 .

HE Y Z , MENG G Z , CHEN K , et al . Towards privacy and security of deep learning systems:a survey [J ] . arXiv:1911.12562 , 2019 .

KESARWANI M , MUKHOTY B , ARYA V , et al . Model extraction warning in MLaaS paradigm [C ] // In Proceedings of the 34th Annual Computer Security Applications Conference,ACSAC 2018 . 2018 : 371 - 380 .

OH S J , SCHIELE B , FRITZ M . Towards reverse-engineering black-box neural networks [M ] // Explainable AI:Interpreting,Explaining and Visualizing Deep Learning . Springer,Cham , 2019 : 121 - 144 .

OREKONDY T , SCHIELE B , FRITZ M . Knockoff nets:Stealing functionality of black-box models [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2019 : 4954 - 4963 .

CHANDRASEKARAN V , CHAUDHURI K , GIACOMELLI I , et al . Exploring connections between active learning and model extraction [J ] . arXiv preprint arXiv:1811.02054 , 2018 .

PENGCHENG L , YI J , ZHANG L . Query-efficient black-box attack by active learning [C ] // 2018 IEEE International Conference on Data Mining (ICDM) . 2018 : 1200 - 1205 .

ILYAS A , ENGSTROM L , ATHALYE A , et al . Black-box adversarial attacks with limited queries and information [J ] . arXiv preprint arXiv:1804.08598 , 2018 .

Views

2989

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Blockchain-based cross-data center anonymous and verifiable identity authentication scheme

AI-based risk assessment and early warning method for futures trading

Bidirectional privacy authorization for large models based on private set intersection

Research on searchable encryption scheme based on enhanced PIR

Research progress on autonomous driving security technology for vehicle-road-cloud collaboration

Related Author

XU Shujiang

ZHANG Chaoyang

WANG Lianhai

ZHANG Shuhui

SHAO Wei

Chang Fei

Wang Junjie

Hu Yi

Related Institution

Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science

Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)

Key Laboratory of Knowledge Engineering with Big Data, Hefei University of Technology

Huishang Futures Co., Ltd.

School of Computer Science and Information Engineering, Hefei University of Technology

⁰