基于黑盒水印的NLP神经网络版权保护

代龙; 张静; 樊雪峰; 周晓谊

doi:10.11959/j.issn.2096-109x.2023009

您当前的位置：

首页 >

文章列表页 >

基于黑盒水印的NLP神经网络版权保护

学术论文 | 更新时间：2024-06-03

- 基于黑盒水印的NLP神经网络版权保护
- NLP neural network copyright protection based on black box watermark
- 网络与信息安全学报 2023年9卷第1期页码：140-149
- 作者机构：
- 作者简介：
  
  [ "代龙（1996- ），男，安徽合肥人，海南大学硕士生，主要研究方向为深度学习、人工智能安全" ]
  [ "张静（1998- ），女，河南平顶山人，海南大学硕士生，主要研究方向为深度学习、人工智能安全" ]
  [ "樊雪峰（1998- ），男，河南汝州人，海南大学硕士生，主要研究方向为人工智能安全、神经网络水印" ]
  [ "周晓谊（1979- ），女，海南海口人，博士，海南大学副教授，主要研究方向为多媒体信息隐藏和加密、神经网络模型版权保护" ]
- 基金信息：
  
  海南省重点研发计划;The Key R＆D Project of Hainan(ZDYF2022GXTS224)
- DOI：10.11959/j.issn.2096-109x.2023009
  中图分类号： TP391
- 网络首发：2023-02，
  
  纸质出版：2023-02-25
- 稿件说明：
移动端阅览
代龙, 张静, 樊雪峰, 等. 基于黑盒水印的NLP神经网络版权保护[J]. 网络与信息安全学报, 2023,9(1):140-149.

Long DAI, Jing ZHANG, Xuefeng FAN, et al. NLP neural network copyright protection based on black box watermark[J]. Chinese Journal of Network and Information Security, 2023, 9(1): 140-149.
代龙, 张静, 樊雪峰, 等. 基于黑盒水印的NLP神经网络版权保护[J]. 网络与信息安全学报, 2023,9(1):140-149. DOI： 10.11959/j.issn.2096-109x.2023009.

Long DAI, Jing ZHANG, Xuefeng FAN, et al. NLP neural network copyright protection based on black box watermark[J]. Chinese Journal of Network and Information Security, 2023, 9(1): 140-149. DOI： 10.11959/j.issn.2096-109x.2023009.

摘要

随着自然语言处理（NLP，natural language processing）技术的快速发展，语言模型在文本分类和情感分析中的应用不断增加。然而，语言模型容易遭到盗版再分发，对模型所有者的知识产权造成严重威胁。因此，研究者着手设计保护机制来识别语言模型的版权信息。现有的适用于文本分类任务的语言模型水印无法与所有者身份相关联，且鲁棒性不足以及无法再生成触发集。为了解决这些问题，提出一种新的适用于文本分类任务模型的黑盒水印方案，可以远程快速验证模型所有权。将模型所有者的版权消息和密钥通过密钥相关的哈希运算消息认证码（HMAC，hash-based message authentication code）得到版权消息摘要，由HMAC得到的消息摘要可以防止被伪造，具有很强的安全性。从原始训练集各个类别中随机挑选一定的文本数据，将摘要与文本数据结合构建触发集，并在训练过程中对语言模型嵌入水印。为了评估水印的性能，在IMDB电影评论、CNEWS中文新闻文本分类数据集上对3种常见的语言模型嵌入水印。实验结果表明，在不影响原始模型测试精度的情况下，所提出的水印验证方案的准确率可以达到 100%。即使在模型微调和剪枝等常见攻击下，也能表现出较强的鲁棒性，并且具有抗伪造攻击的能力。同时，水印的嵌入不会影响模型的收敛时间，具有较高的嵌入效率。

Abstract

With the rapid development of natural language processing techniques

the use of language models in text classification and sentiment analysis has been increasing.However

language models are susceptible to piracy and redistribution by adversaries

posing a serious threat to the intellectual property of model owners.Therefore

researchers have been working on designing protection mechanisms to identify the copyright information of language models.However

existing watermarking of language models for text classification tasks cannot be associated with the owner’s identity

and they are not robust enough and cannot regenerate trigger sets.To solve these problems

a new model

namely black-box watermarking scheme for text classification tasks

was proposed.It was a scheme that can remotely and quickly verify model ownership.The copyright message and the key of the model owner were obtained through the Hash-based Message Authentication Code (HMAC)

and the message digest obtained by HMAC can prevent forgery and had high security.A certain amount of text data was randomly selected from each category of the original training set and the digest was combined with the text data to construct the trigger set

then the watermark was embedded on the language model during the training process.To evaluate the performance of the proposed scheme

watermarks were embedded on three common language models on the IMDB’s movie reviews and CNews text classification datasets.The experimental results show that the accuracy of the proposed watermarking verification scheme can reach 100% without affecting the original model.Even under common attacks such as model fine-tuning and pruning

the proposed watermarking scheme shows strong robustness and resistance to forgery attacks.Meanwhile

the embedding of the watermark does not affect the convergence time of the model and has high embedding efficiency.

关键词

Keywords

references

MOHANARATHINAM A , KAMALRAJ S , PRASANNA-VENKATESAN G K D , , et al . Digital watermarking techniques for image security:A review [J ] . Journal of Ambient Intelligence and Humanized Computing , 2020 , 11 ( 8 ): 3221 - 3229 .

QASIM A F , MEZIANE F , ASPIN R . Digital watermarking:applicability for developing trust in medical imaging workflows state of the art review [J ] . Computer Science Review , 2018 , 27 : 45 - 60 .

樊雪峰 , 周晓谊 , 朱冰冰 , 等 . 深度神经网络模型版权保护方案综述 [J ] . 计算机研究与发展 , 2022 , 59 ( 5 ): 953 - 977 .

FAN X F , ZHOU X Y , ZHU B B , et al . Survey of copyright protection schemes based on DNN model [J ] . Journal of Computer Research and Development , 2022 , 59 ( 5 ): 953 - 977 .

UCHIDA Y , NAGAI Y , SAKAZAWA S , et al . Embedding watermarks into deep neural networks [C ] // Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval . 2017 : 269 - 277 .

ADI Y , BAUM C , CISSE M , et al . Turning your weakness into a strength:Watermarking deep neural networks by backdooring [C ] // 27th USENIX Security Symposium (USENIX Security 18) . 2018 : 1615 - 1631 .

ZHANG J , GU Z , JANG J , et al . Protecting intellectual property of deep neural networks with watermarking [C ] // Proceedings of the 2018 on Asia Conference on Computer and Communications Security . 2018 :159172.

LI M , ZHONG Q , ZHANG L Y , et al . Protecting the intellectual property of deep neural networks with watermarking:The frequency domain approach [C ] // 2020 IEEE 19th International Conference on Trust,Security and Privacy in Computing and Communications (TrustCom) . 2020 : 402 - 409 .

ZHONG Q , ZHANG L Y , ZHANG J , et al . Protecting IP of deep neural networks with watermarking:A new label helps [C ] // PacificAsia Conference on Knowledge Discovery and Data Mining . 2020 :462474.

ABDELNABI S , FRITZ M . Adversarial watermarking transformer:Towards tracing text provenance with data hiding [C ] // 2021 IEEE Symposium on Security and Privacy (SP) . 2021 : 121 - 140 .

YADOLLAHI M M , SHOELEH F , DADKHAH S , et al . Robust black-box watermarking for deep neural network using inverse document frequency [EB ] .

CAMBRIA E , WHITE B . Jumping NLP curves:a review of natural language processing research [J ] . IEEE Computational Intelligence Magazine , 2014 , 9 ( 2 ): 48 - 57 .

ALSHEMALI B , KALITA J . Improving the reliability of deep neural networks in NLP:A review [J ] . Knowledge-Based Systems , 2020 ,191:105210.

DOS SANTOS C , ZADROZNY B . Learning character level representations for part of speech tagging [C ] // International Conference on Machine Learning . 2014 : 1818 - 1826 .

XU D , TIAN Z , LAI R , et al . Deep learning based emotion analysis of microblog texts [J ] . Information Fusion , 2020 , 64 : 1 - 11 .

DHAR A , MUKHERJEE H , DASH N S , et al . Text categorization:past and present [J ] . Artificial Intelligence Review , 2021 , 54 ( 4 ): 3007 - 3054 .

DWIVEDI S K , SINGH V . Research and reviews in question answering system [J ] . Procedia Technology , 2013 , 10 : 417 - 424 .

HUANG H , XU H , WANG X , et al . Maximum F1-score discriminative training criterion for automatic mispronunciation detection [J ] . IEEE/ACM Transactions on Audio,Speech,and Language Processing , 2015 , 23 ( 4 ): 787 - 797 .

YAO Y , LI H , ZHENG H , et al . Latent backdoor attacks on deep neural networks [C ] // Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security . 2019 : 2041 - 2055 .

SUN C , QIU X , XU Y , et al . How to fine-tune bert for text classifycation [C ] // China National Conference on Chinese Computational linguistics . 2019 : 194 - 206 .

OLCHANOV P , MALLYA A , TYREE S , et al . Importance estimation for neural network pruning [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 : 11264 - 11272 .

浏览量

452

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于LLM的多粒度口令分析研究

基于词向量和图卷积的攻击模式与技术实体关联方法

基于DSR和BGRU模型的聊天文本证据分类方法

网络钓鱼识别研究综述

基于深度学习的文本分类研究进展