Generation-based linguistic steganography with controllable security

Jiameng MEI; Yanzhen REN; Lina WANG

doi:10.11959/j.issn.2096-109x.2022039

您当前的位置：

首页 >

文章列表页 >

Generation-based linguistic steganography with controllable security

Topic: Multimedia Content Security | 更新时间：2024-06-03

- Generation-based linguistic steganography with controllable security
- Chinese Journal of Network and Information Security Vol. 8, Issue 3, Pages: 53-65(2022)
- 作者机构：
  
  1. 空天信息安全与可信计算教育部重点实验室，湖北武汉 430072
  2. 武汉大学国家网络安全学院，湖北武汉 430072
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2022039
  CLC： TP37
- Online First：2022-06，
  
  Published：15 June 2022
- 稿件说明：
移动端阅览
Jiameng MEI, Yanzhen REN, Lina WANG. Generation-based linguistic steganography with controllable security[J]. Chinese Journal of Network and Information Security, 2022, 8(3): 53-65.
DOI：

Jiameng MEI, Yanzhen REN, Lina WANG. Generation-based linguistic steganography with controllable security[J]. Chinese Journal of Network and Information Security, 2022, 8(3): 53-65. DOI： 10.11959/j.issn.2096-109x.2022039.

摘要

生成式文本隐写算法通过对候选池中的单词进行控制性选择映射来隐藏秘密信息，通常包含3个模块：文本生成模型、候选池概率分布截断和隐写嵌入算法。由于不同时刻文本生成模型输出的概率分布差异巨大，现有算法通常采用top-k或top-p对候选池单词的概率分布进行截断，以减少低概率的生成词，提高生成文本的安全性。当文本生成模型输出的候选池概率分布过于集中（over-concentrated）或过于平坦（over-flat）时，原有的top-k或top-p截断方式不足以应对概率分布的变化，容易产生概率较低的词或忽略概率较高的词，导致生成文本的安全性指标出现异常。针对此类问题，提出了安全性可控的生成式文本隐写算法，在候选池中根据秘密信息控制性选择生成词时，所提算法基于困惑度和KL散度的参数约束，动态进行候选池概率分布的截断，使候选池中所有单词都满足参数约束，提高了生成文本的安全性。实验结果表明，所提算法生成的隐写文本困惑度和KL散度可控；在相同KL散度情况下，生成文本的困惑度较现有算法下降最高达20%～30%；可以同时控制困惑度和KL散度，在指标合理的情况下，使生成的文本同时满足困惑度和KL散度两个指标。在使用3种文本隐写分析算法检测生成的隐写文本时，检测准确率均在50%左右，表现出很好的统计安全性。

Abstract

Generation-based linguistic steganography hides secret information through controllable modification and mapping of words in the candidate pool.It usually consists of three parts: text generation model

candidate pool probability distribution truncation and steganographic embedding algorithm.Due to the huge difference in the probability distribution of the text generation model outputs at different times

existing algorithms usually use top-k or top-p methods to truncate the probability distribution of words in the candidate pool to reduce the low-probability generated words and improve the security of the generated text.When the probability distribution of the candidate pool output by the text generation model is over-concentrated or over-flat

the original top-k or top-p truncation method will be not enough to cope with the change of the probability distribution

and it is easy to generate low-probability words or ignore high-probability words.This will lead to abnormal security metrics of the generated text.To address these problems

a generation-based linguistic steganography with controllable security was proposed.When selecting generated words with controllability in the candidate pool according to secret information

the proposed algorithm was based on the parameter constraints of perplexity and KL divergence.The truncation of the candidate pool probability distribution made all words satisfy the parameter constraints

which improved the security of the generated text.Experiment results showed that the perplexity and KL divergence of the steganographic text generated by the proposed algorithm are controllable.Under the same KL divergence

the perplexity of the text generated by the proposed algorithm is reduced by up to 20%～30% compared with the existing algorithm.This algorithm could control the perplexity and KL divergence at the same time

and make the generated text satisfy both perplexity and KL divergence when the indicators are reasonable.When using the three text steganalysis algorithms to detect the generated steganographic text

the detection accuracy is about 50%

showing excellent statistical security.

关键词

Keywords

references

BENNETT K . Linguistic steganography:survey,analysis,and robustness concerns for hiding information in text [R ] . 2004 .

XIANG L , WANG X , YANG C , et al . A novel linguistic steganography based on synonym run-length encoding [J ] . IEICE Transactions on Information and Systems , 2017 , 100 ( 2 ): 313 - 322 .

KHOSRAVI B , KHOSRAVI B , KHOSRAVI B , et al . A new method for pdf steganography in justified texts [J ] . Journal of Information Security and Applications , 2019 , 45 ( APR. ): 61 - 70 .

ALATTAR A M , MEMON N D , HEITZENRATER C D , et al . Linguistic steganography on Twitter:hierarchical language modeling with manual interaction [C ] // Media Watermarking,Security,＆Forensics.International Society for Optics and Photonics . 2014 :902803.

DAI W , YU Y , DAI Y , et al . Text steganography system using markov chain source model and DES algorithm [J ] . Journal of Software , 2010 , 5 ( 7 ): 785 - 792 .

MORALDO H H . An Approach for text steganography based on Markov chains [J ] . arXiv preprint arXiv:1409.0915 , 2014 .

LUO Y , HUANG Y , LI F , et al . Text selenography based on cia-poetry generation using markov chain model [J ] . Ksii Transactions on Internet ＆ Information Systems , 2016 , 10 ( 9 ).

YANG Z , JIN S , HUANG Y , et al . Automatically generate steganographic text based on Markov model and Huffman coding [J ] . arXiv preprint arXiv:1811.04720 , 2018 .

ZAREMBA W , SUTSKEVER I , VINYALS O . Recurrent neural network regularization [J ] . arXiv preprint arXiv:1409.2329 , 2014 .

YANG Z L , ZHANG S Y , HU Y T , et al . VAE-Stega:linguistic steganography based on variational auto-encoder [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 16 : 880 - 895 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Advances In Neural Information Processing Systems . 2017 : 5998 - 6008 .

DEVLIN J , CHANG M W , LEE K , et al . Bert:Pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint arXiv:1810.04805 , 2018 .

RADFORD A , WU J , CHILD R , et al . Language models are unsupervised multitask learners [J ] . OpenAI blog , 2019 , 1 ( 8 ): 9 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [J ] . Advances in Neural Information Processing Systems , 2014 ,27.

FANG T , JAGGI M , ARGYRAKI K . Generating steganographic text with LSTMs [J ] . arXiv preprint arXiv:1705.10742 , 2017 .

YANG Z , ZHANG P , JIANG M , et al . Rits:real-time interactive text steganography based on automatic dialogue model [C ] // International Conference on Cloud Computing and Security . 2018 : 253 - 264 .

YANG Z L , GUO X Q , CHEN Z M , et al . RNN-stega:linguistic steganography based on recurrent neural networks [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 14 ( 5 ): 1280 - 1295 .

HUFFMAN D A . A method for the construction of minimum-redundancy codes [J ] . Proceedings of the IRE , 1952 , 40 ( 9 ): 1098 - 1101 .

DAI F Z , CAI Z . Towards near-imperceptible steganographic text [J ] . arXiv preprint arXiv:1907.06679 , 2019 .

SUTSKEVER I , VINYALS O , LE Q V . Sequence to sequence learning with neural networks [C ] // Advances in Neural Information Processing Systems . 2014 : 3104 - 3112 .

ZIEGLER Z M , DENG Y , RUSH A M . Neural linguistic steganography [J ] . arXiv preprint arXiv:1909.01496 , 2019 .

SHEN J , JI H , HAN J . Near-imperceptible neural linguistic steganography via self-adjusting arithmetic coding [J ] . arXiv preprint arXiv:2010.00677 , 2020 .

WITTEN I H , NEAL R M , CLEARY J G . Arithmetic coding for data compression [J ] . Communications of the ACM , 1987 , 30 ( 6 ): 520 - 540 .

FAN A , LEWIS M , DAUPHIN Y . Hierarchical neural story generation [C ] // Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics . 2018 : 889 - 898 .

HOLTZMAN A , BUYS J , FORBES M , et al . Learning to write with cooperative discriminators [C ] // Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics . 2018 : 1638 - 1649 .

HOLTZMAN A , BUYS J , DU L , et al . The curious case of neural text degeneration [C ] // International Conference on Learning Representations . 2019 .

HERMANN K M , KOCISKY T , GREFENSTETTE E , et al . Teaching machines to read and comprehend [J ] . Advances in Neural Information Processing Systems , 2015 , 28 : 1693 - 1701 .

NALLAPATI R , ZHOU B , GULCEHRE C , et al . Abstractive text summarization using sequence-to-sequence rnns and beyond [J ] . arXiv preprint arXiv:1602.06023 , 2016 .

JOULIN A , GRAVE E , BOJANOWSKI P , et al . Bag of tricks for efficient text classification [J ] . arXiv preprint arXiv:1607.01759 , 2016 .

YANG Z , WEI N , SHENG J , et al . TS-CNN:Text steganalysis from semantic space based on convolutional neural network [J ] . arXiv preprint arXiv:1810.08136 , 2018 .

YANG Z , WANG K , LI J , et al . TS-RNN:text steganalysis based on recurrent neural networks [J ] . IEEE Signal Processing Letters , 2019 , 26 ( 12 ): 1743 - 1747 .

WANG S I , MANNING C D . Baselines and bigrams:Simple,good sentiment and topic classification [C ] // Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers) . 2012 : 90 - 94 .

KINGMA D P , BA J . Adam:A method for stochastic optimization [J ] . arXiv preprint arXiv:1412.6980 , 2014 .

PRECHELT L . Early stopping-but when [M ] . Neural Networks : Tricks of the trade.Springer,Berlin,Heidelberg , 1998 : 55 - 69 .

ZHOU P , SHI W , TIAN J , et al . Attention-based bidirectional long short-term memory networks for relation classification [C ] // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics . 2016 : 207 - 212 .

Views

395

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰