Machine learning security and privacy:a survey

Lei SONG; Chunguang MA; Guanghan DUAN

doi:10.11959/j.issn.2096-109x.2018067

您当前的位置：

首页 >

文章列表页 >

Machine learning security and privacy:a survey

Comprehensive Review | 更新时间：2024-06-03

- Machine learning security and privacy:a survey
- Chinese Journal of Network and Information Security Vol. 4, Issue 8, Pages: 1-11(2018)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2018067
  CLC： TP309.2
- Online First：2018-08，
  
  Published：15 August 2018
- 稿件说明：
移动端阅览
Lei SONG, Chunguang MA, Guanghan DUAN. Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018, 4(8): 1-11.
DOI：

Lei SONG, Chunguang MA, Guanghan DUAN. Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018, 4(8): 1-11. DOI： 10.11959/j.issn.2096-109x.2018067.

摘要

机器学习作为实现人工智能的一种重要方法，在数据挖掘、计算机视觉、自然语言处理等领域得到广泛应用。随着机器学习应用的普及发展，其安全与隐私问题受到越来越多的关注。首先结合机器学习的一般过程，对敌手模型进行了描述。然后总结了机器学习常见的安全威胁，如投毒攻击、对抗攻击、询问攻击等，以及应对的防御方法，如正则化、对抗训练、防御精馏等。接着对机器学习常见的隐私威胁，如训练数据窃取、逆向攻击、成员推理攻击等进行了总结，并给出了相应的隐私保护技术，如同态加密、差分隐私。最后给出了亟待解决的问题和发展方向。

Abstract

As an important method to implement artificial intelligence

machine learning technology is widely used in data mining

computer vision

natural language processing and other fields.With the development of machine learning

it brings amount of security and privacy issues which are getting more and more attention.Firstly

the adversary model was described according to machine learning.Secondly

the common security threats in machine learning was summarized

such as poisoning attacks

adversarial attacks

oracle attacks

and major defense methods such as regularization

adversarial training

and defense distillation.Then

privacy issues such were summarized as stealing training data

reverse attacks

and membership tests

as well as privacy protection technologies such as differential privacy and homomorphic encryption.Finally

the urgent problems and development direction were given in this field.

关键词

Keywords

references

GHORBEL A , GHORBEL M , JMAIEL M . Privacy in cloud computing environments:a survey and research challenges [J ] . Journal of Supercomputing , 2017 , 73 ( 6 ): 2763 - 2800 .

SILVER D , HUANG A , MADDISON C J , et al . Mastering the game of go with deep neural networks and tree search [J ] . Nature , 2016 , 529 ( 7587 ): 484 - 489 .

BARRENO M , NELSON B , SEARS R , et al . Can machine learning be secure? [C ] // ACM Symposium on Information,Computer and Communications Security . 2006 : 16 - 25 .

KEARNS M , LI M . Learning in the presence of malicious errors [J ] . SIAM Journal on Computing , 1993 , 22 ( 4 ): 807 - 837 .

BIGGIO B , NELSON B , LASKOV P . Support vector machines under adversarial label noise [J ] . Journal of Machine Learning Research , 2011 , 20 ( 3 ): 97 - 112 .

BIGGIO B , NELSON B , LASKOV P . Poisoning attacks against support vector machines [C ] // International Coference on International Conference on Machine Learning . 2012 : 1467 - 1474 .

MEI S , ZHU X . Using machine teaching to identify optimal training-set attacks on machine learners [C ] // AAAI . 2015 : 2871 - 2877 .

BIGGIO B , DIDACI L , FUMERA G , et al . Poisoning attacks to compromise face templates [C ] // International Conference on Biometrics . 2013 : 1 - 7 .

KLOFT M , LASKOV P . Security analysis of online anomaly detection [J ] . Journal of Machine Learning Research , 2010 , 13 ( 1 ): 3681 - 3724 .

C SZEGEDY , W ZAREMBA , I SUTSKEVER , , et al . Intriguing properties of neural networks [C ] // 2014 International Conference on Learning Representations.Computational and Biological Learning Society . 2014 .

PAPERNOT N , MC D P , SINHA A , et al . Towards the science of security and privacy in machine learning [J ] . arXiv preprint arXiv:1611.03814 , 2016 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // International Conference on Learning Representations . 2015 .

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale [J ] . arXiv preprint arXiv:1611.01236 , 2017 .

DONG Y P , LIAO F Z , PANG T Y , et al . Boosting adversarial attacks with momentum [J ] . arXiv preprint arXiv:1710.06081 , 2017 .

MIYATO T , MAEDA S , KOYAMA M , et al . Virtual adversarial training:a regularization method for supervised and semi-supervised learning [J ] . arXiv preprint 1704.03976 , 2017 .

MOOSAVI-DEZFOOLI S , FAWZI A , FROSSARD P . DeepFool:a simple and accurate method to fool deep neural networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2574 - 2582 .

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // IEEE European Symposium on Security and Privacy . 2016 : 372 - 387 .

SU J , VARGAS D V , KOUICHI S . One pixel attack for fooling deep neural networks [J ] . arXiv preprint arXiv:1710.08864 , 2017 .

LOWD D , MEEK C . Adversarial learning [C ] // The eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining . 2005 : 641 - 647 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Universal adversarial perturbations [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2017 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // 2017 ACM on Asia Conf on Computer and Communications Security . 2017 : 506 - 519 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples [J ] . arXiv preprint arXiv:1605.07277 , 2016 .

GU S X , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples [J ] . arXiv preprint arXiv:1412.5068 , 2014 .

LYU C , HUANG K , LIANG H N . A unified gradient regularization family for adversarial examples [C ] // IEEE International Conference on Data Mining . 2016 : 301 - 309 .

ZHAO Q Y , GRIFFIN L D . Suppressing the unusual:towards robust cnns using symmetric activation functions [J ] . arXiv preprint arXiv:1603.05145 , 2016 .

ROZSA A , GUNTHER M , BOULT T E . Towards robust deep neural networks with BANG [J ] . arXiv preprint arXiv:1612.00138 , 2016 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // International Conference on Learning Representations . Computational and Biological Learning Society , 2015 .

HUANG R , XU B , SCHUURMANS D , et al . Learning with a strong adversary [J ] . arXiv preprint arXiv:1511.03034 , 2015 .

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . ensemble adversarial training:attacks and defenses [J ] . arXiv preprint arXiv:1705.07204 , 2017 .

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // IEEE Symp on Security and Privacy . 2016 : 582 - 597 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [J ] . arXiv preprint arXiv:1503.02531 , 2015 .

PAPERNOT N , MCDANIEL P . Extending defensive distillation [J ] . arXiv preprint arXiv:1705.05264 , 2017 .

BULÒS R , BIGGIO B , PILLAI I , et al . Randomized prediction games for adversarial machine learning [J ] . IEEE transactions on neural networks and learning systems , 2017 , 28 ( 11 ): 2466 - 2478 .

HARDT M , MEGIDDO N , PAPADIMITRIOU C , et al . Strategic classification [C ] // 2016 ACM conference on innovations in theoretical computer science . 2016 : 111 - 122 .

BRÜCKNER M , KANZOW C , SCHEFFER T . Static prediction games for adversarial learning problems [J ] . Journal of Machine Learning Research , 2012 , 13 ( Sep ): 2617 - 2654 .

METZEN J H , GENEWEIN T , FISCHER V , et al . On detecting adversarial perturbations [J ] . arXiv preprint arXiv:1702.04267 , 2017 .

LU JIAJUN , ISSARANON T , FORSYTH D . SAFETYNET:Detecting and rejecting adversarial examples robustly [J ] . arXiv preprint arXiv:1704.00103 , 2017 .

HITAJ B , ATENIESE G ， PEREZ-CRUZ F . Deep models under the GAN:information leakage from collaborative deep learning [C ] // ACM Sigsac Conference . 2017 : 603 - 618 .

FREDRIKSON M , LANTZ E , JHA S , et al . Privacy in pharmacogenetics:an end-to-end case study of personalized warfarin dosing [C ] // The 23rd Usenix Security Symposium . 2014 : 17 - 32 .

FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures [C ] // The 22nd ACM Sigsac Conference on Computer and Communications Security . 2015 : 1322 - 1333 .

ATENIESE G , MANCINI L V , SPOGNARDI A , et al . Hacking smart machines with smarter ones:How to extract meaningful data from machine learning classifiers [J ] . International Journal of Security and Networks , 2015 , 10 ( 3 ): 137 - 150 .

SHOKRI R , STRONATI M , SONG C , et al . Membership inference attacks against machine learning models [J ] . arXiv preprint arXiv:1610.05820 , 2016 .

TRAMER F , ZHANG F , JUELS A , et al . Stealing machine learning models via prediction apis [J ] . arXiv preprint arXiv:1609.02943 , 2016 .

GENTRY , CRAIG , Fully homomorphic encryption using ideal lattices [J ] . Stoc , 2009 , 9 ( 4 ): 169 - 178 .

DOWLIN N , RAN G B , LAINE K , et al . CryptoNets:applying neural networks to encrypted data with high throughput and accuracy [C ] // Radio and Wireless Symposium . 2016 : 76 - 78 .

HESAMIFARD E , TAKABI H , GHASEMI M , et al . Privacy-preserving machine learning in cloud [C ] // The 2017 on Cloud Computing Security Workshop . 2017 : 39 - 43 .

BARYALAI M , JANG-JACCARD J , LIU D . Towards privacy-preserving classification in neural networks [C ] // IEEE Privacy,Security and Trust . 2017 : 392 - 399 .

XIE P , BILENKO M , FINLEY T , et al . Crypto-nets:neural networks over encrypted data [J ] . Computer Science , 2014 .

STONE M H . The generalized weierstrass approximation theorem [J ] . Mathematics Magazine , 1948 , 21 ( 4 ): 167 - 184 .

ZHANG Q , YANG L , CHEN Z . Privacy preserving deep computation model on cloud for big data feature learning [J ] . IEEE Transactions on Computers , 2016 , 65 ( 5 ): 1351 - 1362 .

DWORK C , MCSHERRY F , NISSIM K , et al . Calibrating noise to sensitivity in private data analysis [C ] // The Third conference on Theory of Cryptography . 2006 : 265 - 284 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // 2016 ACM Sigsac Conference on Computer and Communications Security . 2016 : 308 - 318 .

PAPERNOT N , ABADI M , ERLINGSSON U , et al . Semi- supervised knowledge transfer for deep learning from private training data [J ] . arXiv preprint arXiv:1610.05755 , 2016 .

BEAULIEUJONES B K , WU Z S , WILLIAMS C J , et al . Privacy-preserving generative deep neural networks support clinical data sharing [J ] . bioRxiv , 2017 .

郭鹏 , 钟尚平 , 陈开志 , 等 . 差分隐私 GAN 梯度裁剪阈值的自适应选取方法 [J ] . 网络与信息安全学报 , 2018 , 4 ( 5 ): 10 - 20 .

GUO P , ZHONG S P , CHEN K Z , et al . Adaptive selection method of differential privacy [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 5 ): 10 - 20 .

SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning [C ] // The 22nd ACM SIGSAC Conference on Computer and Communications Security . 2015 : 1310 - 1321 .

LIU M , JIANG H , CHEN J , et al . A collaborative privacy-preserving deep learning system in distributed mobile environment [C ] // International Conference on Computational Science and Computational Intelligence . 2017 : 192 - 197 .

LE T P , AONO Y , HAYASHI T , et al . Privacy-preserving deep learning via additively homomorphic encryption [J ] . IEEE Transactions on Information Forensics ＆ Security , 2018 , 13 ( 5 ): 1333 - 1345 .

MCMAHAN B , RAMAGE D . Federated learning:collaborative machine learning without centralized training data [J ] . Google Research Blog , 2017 .

BONAWITZ K , IVANOV V , KREUTER B , et al . Practical secure aggregation for privacy-preserving machine learning [C ] // 2017 ACM Sigsac Conference on Computer and Communications Security . 2017 : 1175 - 1191 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Federated learning of deep networks using model averaging [J ] . arXiv preprint arXiv:1502.01710v5 , 2016 .

OSSIA S A , SHAMSABADI A S , TAHERI A , et al . A hybrid deep learning architecture for privacy-preserving mobile analytics [J ] . arXiv preprint arXiv:1703.02952 , 2017 .

Views

10399

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Anomaly detection in DNS: an empirical study

Research progress on detection of software vulnerabilities based on code similarity

method for machine learning backdoor attacks based on defense method based on diffusion models

Research progress on autonomous driving security technology for vehicle-road-cloud collaboration

Personal data privacy protection method based onvertical partitioning

Related Author

ZUO Peng

CHEN Yong

KONG Kaichuan

HE Ming

HOU Ronghao

DONG Weiyu

TANG Yonghe

LIU Chunling

Related Institution

China Internet Network Information Center

School of Cyber Science and Engineering, Jinan University

Information Engineering University

School of Medical Information, Wannan Medical College

School of Physics and Electronic Information, Anhui Normal University

⁰