基于改进Transformer的社交媒体谣言检测

郑洪浩; 郝一诺; 于洪涛; 李邵梅; 吴翼腾

doi:10.11959/j.issn.2096-109x.2022042

您当前的位置：

首页 >

文章列表页 >

基于改进Transformer的社交媒体谣言检测

学术论文 | 更新时间：2024-06-03

- 基于改进Transformer的社交媒体谣言检测
- Rumor detection in social media based on eahanced Transformer
- 网络与信息安全学报 2022年8卷第4期页码：168-174
- 作者机构：
- 作者简介：
  
  [ "郑洪浩（1992− ），男，山东济宁人，信息工程大学硕士生，主要研究方向为自然语言处理" ]
  [ "郝一诺（1997− ），女，江苏徐州人，信息工程大学硕士生，主要研究方向为无线物理层安全、自然语言处理" ]
  [ "于洪涛（1970− ），男，辽宁丹东人，博士，信息工程大学研究员、博士生导师，主要研究方向为网络大数据分析与处理" ]
  [ "李邵梅（1982− ），女，湖北钟祥人，博士，信息工程大学副研究员，主要研究方向为计算机视觉" ]
  [ "吴翼腾（1992− ），男，山东乐陵人，信息工程大学博士生，主要研究方向为人工智能安全" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61601513);郑州市协同创新重大专项;Major Collaborative Innovation Projects of Zhengzhou(162/32410218)
- DOI：10.11959/j.issn.2096-109x.2022042
  中图分类号： TP391
- 网络首发：2022-08，
  
  纸质出版：2022-08-15
- 稿件说明：
移动端阅览
郑洪浩, 郝一诺, 于洪涛, 等. 基于改进Transformer的社交媒体谣言检测[J]. 网络与信息安全学报, 2022,8(4):168-174.

Honghao ZHENG, Yinuo HAO, Hongtao YU, et al. Rumor detection in social media based on eahanced Transformer[J]. Chinese Journal of Network and Information Security, 2022, 8(4): 168-174.
郑洪浩, 郝一诺, 于洪涛, 等. 基于改进Transformer的社交媒体谣言检测[J]. 网络与信息安全学报, 2022,8(4):168-174. DOI： 10.11959/j.issn.2096-109x.2022042.

Honghao ZHENG, Yinuo HAO, Hongtao YU, et al. Rumor detection in social media based on eahanced Transformer[J]. Chinese Journal of Network and Information Security, 2022, 8(4): 168-174. DOI： 10.11959/j.issn.2096-109x.2022042.

摘要

随着互联网的快速发展，社交媒体日益广泛而深刻地融入人们日常生活的各个方面。社交媒体逐渐成为人们彼此之间用来分享意见、见解、经验和观点的工具和平台，是人们获取分享信息、表达交流观点的主要途径。社交媒体在互联网的沃土上蓬勃发展，爆发出令人眩目的能量。由于社交媒体的开放性，用户规模庞大且来源复杂众多，容易产生各种各样的谣言虚假信息。社交媒体谣言左右着网民对事件的认识、动摇着社会的稳定。因此，如何准确高效地检测谣言成为当下亟待解决的问题。现有基于 Transformer的社交媒体谣言检测模型忽略了文本位置信息。为有效提取文本位置信息，充分利用文本潜在信息，提出了一种基于改进 Transformer 的社交媒体谣言检测模型。该模型从相对位置和绝对位置两方面对传统Transformer 进行改进：一方面采用可学习的相对位置编码捕捉文本的方向信息和距离信息；另一方面采用绝对位置编码将不同位置词语映射到不同特征空间。实验结果表明，与其他基准模型相比，所提模型在Twitter15、Twitter16和Weibo 3种数据集上的准确率分别提高了0.9%、0.6%和1.4%。实验结果验证了所提的位置编码改进有效，基于位置编码改进的Transformer模型可显著提升社交媒体谣言检测效果。

Abstract

With the rapid development of the Internet

social media is increasingly integrated into all aspects of people’s daily life.Social media has gradually become a tool and even a platform for people to share opinions

insights

experiences and viewpoints.It is the main method for people to obtain and share information as well as express and exchange opinions.Currently

social media mainly includes social networking sites

Weibo

Twitter

blogs

forums

podcasts and so on.Due to the openness of social media

the user scale is large and the sources are complex and numerous

then all kinds of rumors and false information may be generated easily.Rumors on social media influence netizens’ understanding of events and shake the stability of society.Therefore

how to accurately and efficiently detect rumors has become an urgent problem to be solved.Existing Transformer based social media rumor detection models ignored the text location information.To effectively extract text location information and make full use of text potential information

a rumor detection model in social media was proposed and it was based on the enhanced Transformer.This model enhanced the traditional Transformer from two aspects of relative position and absolute position.It captured the direction information and distance information of the text using learnable relative position coding and mapped words from different positions to different feature spaces using absolute position coding.Experimental results show that

compared with the best benchmark model

the accuracy of the proposed model on Twitter15

Twitter16 and Weibo datasets is enhanced by 0.9%

0.6% and 1.4%

respectively.Experimental results verify the effectiveness of the proposed location coding.And the enhanced Transformer based on location coding can significantly improve the effects of social media rumor detection.

关键词

Keywords

references

LIU Z Y , ZHANG L , TU C C , et al . Statistical and semantic analysis of rumors in Chinese social media [J ] . Scientia Sinica , 2015 , 45 ( 12 ): 1536 - 1546 .

MIKOLOV T , SUTSKEVER L , CHEN K , et al . Distributed representations of words and phrases and their compositionality [J ] . Advances in Neural Information Processing Systems , 2013 ,( 26 ): 3111 - 3119 .

RUCHANSKY N , SEO S , LIU Y . CSI:a hybrid deep model for fake news detection [J ] . 2017:arXiv:1703.06959 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . 2017 : 6000 - 6010 .

YUAN C Y , MA Q W , ZHOU W , et al . Jointly embedding the local and global relations of heterogeneous graph for rumor detection [C ] // Proceedings of 2019 IEEE International Conference on Data Mining (ICDM) . 2019 : 796 - 805 .

琚心怡 . 基于深层双向 Transformer 编码器的早期谣言检测 [J ] . 信息通信 , 2020 , 33 ( 5 ): 17 - 22 .

QU X Y . Early rumor detection based on deep two-way Transformer encoder [J ] . Information ＆ Communications , 2020 , 33 ( 5 ): 17 - 22 .

MA J , GAO W , WONG K F . Detect rumors in microblog posts using propagation structure via kernel learning [C ] // Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics . 2017 : 708 - 717 .

MA J , GAO W , MITRA P , et al . Detecting rumors from microblogs with recurrent neural networks [C ] // Proceedings of International Joint Conference on Artificial Intelligence . 2016 .

SHAW P , USZKOREIT J , VASWANI A . Self-attention with relative position representations [J ] . arXiv preprint arXiv:1803.02155 , 2018 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint arXiv:1810.04805 , 2018 .

LIU Y H , OTT M , GOYAL N , et al . RoBERTa:a robustly optimized BERT pretraining approach [J ] . arXiv preprint arXiv:1907.11692 , 2019 .

YANG Z L , DAI Z H , YANG Y M , et al . XLNet:generalized autoregressive pretraining for language understanding [J ] . CoRR,2019,abs/1906.08237 .

YAN H , DENG B C , LI X N , et al . TENER:adapting transformer encoder for name entity recognition [J ] . arXiv preprint arXiv:1911.04474 , 2019 .

DAI Z H , YANG Z L , YANG Y M , et al . Transformer-XL:attentive language models beyond a fixed-length context [J ] . arXiv preprint arXiv:1901.02860 , 2019 .

HE P C , LIU X D , GAO J F , et al . DeBERTa:decoding-enhanced BERT with disentangled attention [J ] . arXiv preprint arXiv:2006.03654 , 2020 .

KE G L , HE D , LIU T Y . Rethinking the positional encoding in language pre-training [J ] . arXiv preprint arXiv:2006.15595 , 2020 .

WANG B Y , ZHAO D H , LIOMA C , et al . Encoding word order in complex embeddings [J ] . arXiv preprint arXiv:2006.15595 , 2020 .

LIU Y , WU Y F . Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks [C ] // Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence . 2018 .

MIKOLOV T , CHEN K , CORRADO G , et al . Efficient estimation of word representations in vector space [J ] . arXiv preprint arXiv:1301.3781 , 2013 .

SMITH L N , . Cyclical learning rates for training neural networks [C ] // Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) . 2017 : 464 - 472 .

CASTILLO C , MENDOZA M , POBLETE B . Information credibility on twitter [C ] // Proceedings of the 20th International Conference on World Wide Web-WWW '11 . 2011 : 675 - 684 .

MA J , GAO W , WEI Z Y , et al . Detect rumors using time series of social context information on microblogging websites [C ] // Proceedings of the 24th ACM International on Conference on Information and Knowledge Management . 2015 : 1751 - 1754 .

ZHAO Z , RESNICK P , MEI Q . Enquiring minds:early detection of rumors in social media from enquiry posts [C ] // Proceedings of International World Wide Web Conferences Steering Committee . 2015 .

MA J , GAO W , WONG K F . Rumor detection on twitter with tree-structured recursive neural networks [C ] // Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics . 2018 : 1980 - 1989 .

浏览量

543

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据