机器学习安全及隐私保护研究进展

宋蕾; 马春光; 段广晗

doi:10.11959/j.issn.2096-109x.2018067

您当前的位置：

首页 >

文章列表页 >

机器学习安全及隐私保护研究进展

综述 | 更新时间：2024-06-03

- 机器学习安全及隐私保护研究进展
- Machine learning security and privacy:a survey
- 网络与信息安全学报 2018年4卷第8期页码：1-11
- 作者机构：
- 作者简介：
  
  [ "宋蕾（1989-），女，黑龙江牡丹江人，哈尔滨工程大学博士生，主要研究方向为机器学习安全与隐私保护、云计算、网络安全。" ]
  [ "马春光（1974-），男，黑龙江双城人，哈尔滨工程大学教授、博士生导师，主要研究方向为分布式密码算法与协议、云计算安全与隐私、格密码、机器学习安全与隐私保护。" ]
  [ "段广晗（1994-），男，黑龙江海伦人，哈尔滨工程大学博士生，主要研究方向为深度学习、对抗样本、机器学习。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61472097)
- DOI：10.11959/j.issn.2096-109x.2018067
  中图分类号： TP309.2
- 网络首发：2018-08，
  
  纸质出版：2018-08-15
- 稿件说明：
移动端阅览
宋蕾, 马春光, 段广晗. 机器学习安全及隐私保护研究进展[J]. 网络与信息安全学报, 2018,4(8):1-11.

Lei SONG, Chunguang MA, Guanghan DUAN. Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018, 4(8): 1-11.
宋蕾, 马春光, 段广晗. 机器学习安全及隐私保护研究进展[J]. 网络与信息安全学报, 2018,4(8):1-11. DOI： 10.11959/j.issn.2096-109x.2018067.

Lei SONG, Chunguang MA, Guanghan DUAN. Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018, 4(8): 1-11. DOI： 10.11959/j.issn.2096-109x.2018067.

摘要

机器学习作为实现人工智能的一种重要方法，在数据挖掘、计算机视觉、自然语言处理等领域得到广泛应用。随着机器学习应用的普及发展，其安全与隐私问题受到越来越多的关注。首先结合机器学习的一般过程，对敌手模型进行了描述。然后总结了机器学习常见的安全威胁，如投毒攻击、对抗攻击、询问攻击等，以及应对的防御方法，如正则化、对抗训练、防御精馏等。接着对机器学习常见的隐私威胁，如训练数据窃取、逆向攻击、成员推理攻击等进行了总结，并给出了相应的隐私保护技术，如同态加密、差分隐私。最后给出了亟待解决的问题和发展方向。

Abstract

As an important method to implement artificial intelligence

machine learning technology is widely used in data mining

computer vision

natural language processing and other fields.With the development of machine learning

it brings amount of security and privacy issues which are getting more and more attention.Firstly

the adversary model was described according to machine learning.Secondly

the common security threats in machine learning was summarized

such as poisoning attacks

adversarial attacks

oracle attacks

and major defense methods such as regularization

adversarial training

and defense distillation.Then

privacy issues such were summarized as stealing training data

reverse attacks

and membership tests

as well as privacy protection technologies such as differential privacy and homomorphic encryption.Finally

the urgent problems and development direction were given in this field.

关键词

Keywords

references

GHORBEL A , GHORBEL M , JMAIEL M . Privacy in cloud computing environments:a survey and research challenges [J ] . Journal of Supercomputing , 2017 , 73 ( 6 ): 2763 - 2800 .

SILVER D , HUANG A , MADDISON C J , et al . Mastering the game of go with deep neural networks and tree search [J ] . Nature , 2016 , 529 ( 7587 ): 484 - 489 .

BARRENO M , NELSON B , SEARS R , et al . Can machine learning be secure? [C ] // ACM Symposium on Information,Computer and Communications Security . 2006 : 16 - 25 .

KEARNS M , LI M . Learning in the presence of malicious errors [J ] . SIAM Journal on Computing , 1993 , 22 ( 4 ): 807 - 837 .

BIGGIO B , NELSON B , LASKOV P . Support vector machines under adversarial label noise [J ] . Journal of Machine Learning Research , 2011 , 20 ( 3 ): 97 - 112 .

BIGGIO B , NELSON B , LASKOV P . Poisoning attacks against support vector machines [C ] // International Coference on International Conference on Machine Learning . 2012 : 1467 - 1474 .

MEI S , ZHU X . Using machine teaching to identify optimal training-set attacks on machine learners [C ] // AAAI . 2015 : 2871 - 2877 .

BIGGIO B , DIDACI L , FUMERA G , et al . Poisoning attacks to compromise face templates [C ] // International Conference on Biometrics . 2013 : 1 - 7 .

KLOFT M , LASKOV P . Security analysis of online anomaly detection [J ] . Journal of Machine Learning Research , 2010 , 13 ( 1 ): 3681 - 3724 .

C SZEGEDY , W ZAREMBA , I SUTSKEVER , , et al . Intriguing properties of neural networks [C ] // 2014 International Conference on Learning Representations.Computational and Biological Learning Society . 2014 .

PAPERNOT N , MC D P , SINHA A , et al . Towards the science of security and privacy in machine learning [J ] . arXiv preprint arXiv:1611.03814 , 2016 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // International Conference on Learning Representations . 2015 .

KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale [J ] . arXiv preprint arXiv:1611.01236 , 2017 .

DONG Y P , LIAO F Z , PANG T Y , et al . Boosting adversarial attacks with momentum [J ] . arXiv preprint arXiv:1710.06081 , 2017 .

MIYATO T , MAEDA S , KOYAMA M , et al . Virtual adversarial training:a regularization method for supervised and semi-supervised learning [J ] . arXiv preprint 1704.03976 , 2017 .

MOOSAVI-DEZFOOLI S , FAWZI A , FROSSARD P . DeepFool:a simple and accurate method to fool deep neural networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 2574 - 2582 .

PAPERNOT N , MCDANIEL P , JHA S , et al . The limitations of deep learning in adversarial settings [C ] // IEEE European Symposium on Security and Privacy . 2016 : 372 - 387 .

SU J , VARGAS D V , KOUICHI S . One pixel attack for fooling deep neural networks [J ] . arXiv preprint arXiv:1710.08864 , 2017 .

LOWD D , MEEK C . Adversarial learning [C ] // The eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining . 2005 : 641 - 647 .

MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O , et al . Universal adversarial perturbations [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2017 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I , et al . Practical black-box attacks against machine learning [C ] // 2017 ACM on Asia Conf on Computer and Communications Security . 2017 : 506 - 519 .

PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples [J ] . arXiv preprint arXiv:1605.07277 , 2016 .

GU S X , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples [J ] . arXiv preprint arXiv:1412.5068 , 2014 .

LYU C , HUANG K , LIANG H N . A unified gradient regularization family for adversarial examples [C ] // IEEE International Conference on Data Mining . 2016 : 301 - 309 .

ZHAO Q Y , GRIFFIN L D . Suppressing the unusual:towards robust cnns using symmetric activation functions [J ] . arXiv preprint arXiv:1603.05145 , 2016 .

ROZSA A , GUNTHER M , BOULT T E . Towards robust deep neural networks with BANG [J ] . arXiv preprint arXiv:1612.00138 , 2016 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C ] // International Conference on Learning Representations . Computational and Biological Learning Society , 2015 .

HUANG R , XU B , SCHUURMANS D , et al . Learning with a strong adversary [J ] . arXiv preprint arXiv:1511.03034 , 2015 .

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . ensemble adversarial training:attacks and defenses [J ] . arXiv preprint arXiv:1705.07204 , 2017 .

PAPERNOT N , MCDANIEL P , WU X , et al . Distillation as a defense to adversarial perturbations against deep neural networks [C ] // IEEE Symp on Security and Privacy . 2016 : 582 - 597 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [J ] . arXiv preprint arXiv:1503.02531 , 2015 .

PAPERNOT N , MCDANIEL P . Extending defensive distillation [J ] . arXiv preprint arXiv:1705.05264 , 2017 .

BULÒS R , BIGGIO B , PILLAI I , et al . Randomized prediction games for adversarial machine learning [J ] . IEEE transactions on neural networks and learning systems , 2017 , 28 ( 11 ): 2466 - 2478 .

HARDT M , MEGIDDO N , PAPADIMITRIOU C , et al . Strategic classification [C ] // 2016 ACM conference on innovations in theoretical computer science . 2016 : 111 - 122 .

BRÜCKNER M , KANZOW C , SCHEFFER T . Static prediction games for adversarial learning problems [J ] . Journal of Machine Learning Research , 2012 , 13 ( Sep ): 2617 - 2654 .

METZEN J H , GENEWEIN T , FISCHER V , et al . On detecting adversarial perturbations [J ] . arXiv preprint arXiv:1702.04267 , 2017 .

LU JIAJUN , ISSARANON T , FORSYTH D . SAFETYNET:Detecting and rejecting adversarial examples robustly [J ] . arXiv preprint arXiv:1704.00103 , 2017 .

HITAJ B , ATENIESE G ， PEREZ-CRUZ F . Deep models under the GAN:information leakage from collaborative deep learning [C ] // ACM Sigsac Conference . 2017 : 603 - 618 .

FREDRIKSON M , LANTZ E , JHA S , et al . Privacy in pharmacogenetics:an end-to-end case study of personalized warfarin dosing [C ] // The 23rd Usenix Security Symposium . 2014 : 17 - 32 .

FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures [C ] // The 22nd ACM Sigsac Conference on Computer and Communications Security . 2015 : 1322 - 1333 .

ATENIESE G , MANCINI L V , SPOGNARDI A , et al . Hacking smart machines with smarter ones:How to extract meaningful data from machine learning classifiers [J ] . International Journal of Security and Networks , 2015 , 10 ( 3 ): 137 - 150 .

SHOKRI R , STRONATI M , SONG C , et al . Membership inference attacks against machine learning models [J ] . arXiv preprint arXiv:1610.05820 , 2016 .

TRAMER F , ZHANG F , JUELS A , et al . Stealing machine learning models via prediction apis [J ] . arXiv preprint arXiv:1609.02943 , 2016 .

GENTRY , CRAIG , Fully homomorphic encryption using ideal lattices [J ] . Stoc , 2009 , 9 ( 4 ): 169 - 178 .

DOWLIN N , RAN G B , LAINE K , et al . CryptoNets:applying neural networks to encrypted data with high throughput and accuracy [C ] // Radio and Wireless Symposium . 2016 : 76 - 78 .

HESAMIFARD E , TAKABI H , GHASEMI M , et al . Privacy-preserving machine learning in cloud [C ] // The 2017 on Cloud Computing Security Workshop . 2017 : 39 - 43 .

BARYALAI M , JANG-JACCARD J , LIU D . Towards privacy-preserving classification in neural networks [C ] // IEEE Privacy,Security and Trust . 2017 : 392 - 399 .

XIE P , BILENKO M , FINLEY T , et al . Crypto-nets:neural networks over encrypted data [J ] . Computer Science , 2014 .

STONE M H . The generalized weierstrass approximation theorem [J ] . Mathematics Magazine , 1948 , 21 ( 4 ): 167 - 184 .

ZHANG Q , YANG L , CHEN Z . Privacy preserving deep computation model on cloud for big data feature learning [J ] . IEEE Transactions on Computers , 2016 , 65 ( 5 ): 1351 - 1362 .

DWORK C , MCSHERRY F , NISSIM K , et al . Calibrating noise to sensitivity in private data analysis [C ] // The Third conference on Theory of Cryptography . 2006 : 265 - 284 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // 2016 ACM Sigsac Conference on Computer and Communications Security . 2016 : 308 - 318 .

PAPERNOT N , ABADI M , ERLINGSSON U , et al . Semi- supervised knowledge transfer for deep learning from private training data [J ] . arXiv preprint arXiv:1610.05755 , 2016 .

BEAULIEUJONES B K , WU Z S , WILLIAMS C J , et al . Privacy-preserving generative deep neural networks support clinical data sharing [J ] . bioRxiv , 2017 .

郭鹏 , 钟尚平 , 陈开志 , 等 . 差分隐私 GAN 梯度裁剪阈值的自适应选取方法 [J ] . 网络与信息安全学报 , 2018 , 4 ( 5 ): 10 - 20 .

GUO P , ZHONG S P , CHEN K Z , et al . Adaptive selection method of differential privacy [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 5 ): 10 - 20 .

SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning [C ] // The 22nd ACM SIGSAC Conference on Computer and Communications Security . 2015 : 1310 - 1321 .

LIU M , JIANG H , CHEN J , et al . A collaborative privacy-preserving deep learning system in distributed mobile environment [C ] // International Conference on Computational Science and Computational Intelligence . 2017 : 192 - 197 .

LE T P , AONO Y , HAYASHI T , et al . Privacy-preserving deep learning via additively homomorphic encryption [J ] . IEEE Transactions on Information Forensics ＆ Security , 2018 , 13 ( 5 ): 1333 - 1345 .

MCMAHAN B , RAMAGE D . Federated learning:collaborative machine learning without centralized training data [J ] . Google Research Blog , 2017 .

BONAWITZ K , IVANOV V , KREUTER B , et al . Practical secure aggregation for privacy-preserving machine learning [C ] // 2017 ACM Sigsac Conference on Computer and Communications Security . 2017 : 1175 - 1191 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Federated learning of deep networks using model averaging [J ] . arXiv preprint arXiv:1502.01710v5 , 2016 .

OSSIA S A , SHAMSABADI A S , TAHERI A , et al . A hybrid deep learning architecture for privacy-preserving mobile analytics [J ] . arXiv preprint arXiv:1703.02952 , 2017 .

浏览量

10399

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

DNS异常检测：大规模实验分析

基于代码相似性的软件漏洞检测研究进展

基于扩散模型的机器学习后门攻击防御方法

车-路-云协同自动驾驶安全技术研究进展

基于垂直分割的个人数据隐私保护方法