Out-of-context misinformation detection method based on stance analysis

YUAN Xin; GUO Jie; QIU Weidong; HUANG Zheng

doi:10.11959/j.issn.2096-109x.2024017

您当前的位置：

首页 >

文章列表页 >

Out-of-context misinformation detection method based on stance analysis

Papers | 更新时间：2024-08-14

- Out-of-context misinformation detection method based on stance analysis
- Chinese Journal of Network and Information Security Vol. 10, Issue 2, Pages: 133-142(2024)
- 作者机构：
  
  上海交通大学网络空间安全学院，上海 200240
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(61972249)
- DOI：10.11959/j.issn.2096-109x.2024017
  CLC： TP309
- Received：29 May 2023，
  
  Revised：2023-07-24，
  
  Published：15 April 2024
- 稿件说明：
移动端阅览
袁鑫, 郭捷, 邱卫东, 等. 基于立场分析的脱离上下文虚假信息检测方法[J]. 网络与信息安全学报, 2024, 10(02): 133-142.

Format: YUAN X, GUO J, QIU W D, et al. Out-of-context misinformation detection method based on stance analysis[J]. Chinese Journal of Network and Information Security, 2024, 10(02): 133-142.
袁鑫, 郭捷, 邱卫东, 等. 基于立场分析的脱离上下文虚假信息检测方法[J]. 网络与信息安全学报, 2024, 10(02): 133-142. DOI： 10.11959/j.issn.2096-109x.2024017.

Format: YUAN X, GUO J, QIU W D, et al. Out-of-context misinformation detection method based on stance analysis[J]. Chinese Journal of Network and Information Security, 2024, 10(02): 133-142. DOI： 10.11959/j.issn.2096-109x.2024017.

摘要

随着基于人工智能的自动生成技术的发展，虚假信息以更具欺骗性的形式出现，对经济和社会秩序造成了巨大危害。脱离上下文虚假信息是最具欺骗性和最容易实现的虚假信息类型之一，恶意攻击者通过歪曲真实图像中的人物、事件、地点等上下文信息来增强虚假叙事的可信度。针对已有检测算法严重依赖知识库、未能全面考虑待检测信息与互联网证据之间的立场关系的问题，提出了一种基于立场分析的脱离上下文虚假信息检测方法。对于需要检测的图像－标题对以及在互联网上检索到的文本证据与视觉证据，该方法根据标题与文本证据中命名实体的共现关系，为每条文本证据计算一个立场增益分数；使用互相独立的立场分析网络对图像与视觉证据、标题与文本证据分别进行层次聚类，并完成基于多个注意力机制与立场分析模块的语义立场表征抽取；根据语义比较与立场分析的结果对图像－标题对的真实性进行预测。实验结果表明，得益于立场分析的引入，所提方法能够显著提高检测效果，相比其他使用互联网证据进行检测的最佳算法准确率提升了2.3%。

Abstract

As artificial intelligence-based automatic generation technology advances

the emergence of misinformation in more sophisticated guises has become a significant threat to the economy and social order. Among its forms

out-of-context misinformation stands out as particularly deceptive and readily executable. This type of misinformation involves malicious actors enhancing the credibility of false narratives by misrepresenting contextual details such as individuals

events

and locations within real images. To address the shortcomings of current detection algorithms

which heavily depend on knowledge bases and often overlook the stance relationship between the information under scrutiny and online evidence

a stance analysis-based out-of-context misinformation detection method was developed. This method involved several steps for the detection of an image-caption pair along with the corresponding textual and visual evidence retrieved from the Internet. Initially

a stance gain score for each piece of textual evidence was calculated based on the co-occurrence of named entities. Subsequently

independent stance analysis networks were utilized to perform hierarchical clustering on both the image and visual evidence

as well as on the caption and textual evidence. This process involved the extraction of semantic stance representations

facilitated by multiple attention mechanisms and a stance analysis module. The authenticity of the image-caption pair was subsequently predicted based on the outcomes of semantic comparison and stance analysis. Experimental results indicate that the incorporation of stance analysis significantly enhances the method's detection capabilities. Specifically

the accuracy of this method outperforms the state-of-the-art algorithm that employs Internet evidence for detection by 2.3%.

关键词

Keywords

references

THOMAS C , KOVASHKA A . Preserving semantic neighborhoods for robust cross-modal retrieval [C ] // Proceedings of the European Conference on Computer Vision . Cham : Springer , 2020 : 317 - 335 .

JAISWAL A , SABIR E , ABDALMAGEED W , et al . Multimedia semantic integrity assessment using joint embedding of images and text [C ] // Proceedings of the 25th ACM international conference on Multimedia . New York : ACM Press , 2017 : 1465 - 1471 .

SABIR E , ABDALMAGEED W , WU Y , et al . Deep multi-modal image-repurposing detection [C ] // Proceedings of the 26th ACM International Conference on Multimedia . New York : ACM , 2018 : 1337 - 1345 .

JAISWAL A , WU Y , ABDALMAGEED W , et al . AIRD: adversarial learning framework for image repurposing detection [C ] // Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2019 : 11322 - 11331 .

LUO G , DARRELL T , ROHRBACH A . NewsCLIPpings: automatic generation of out-of-context multimodal media [C ] // Proceedings of the Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 6801 - 6817 .

MU M , DAS BHATTACHARJEE S , YUAN J S . Self-supervised distilled learning for multi-modal misinformation identification [C ] // Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE Press , 2023 : 2818 - 2827 .

MÜLLER-BUDACK E , THEINER J , DIERING S , et al . Multimodal analytics for Real-world news using measures of Cross-modal entity consistency [C ] // Proceedings of the 2020 International Conference on Multimedia Retrieval . New York : ACM , 2020 : 16 - 25 .

ABDELNABI S , HASAN R , FRITZ M . Open-domain, content-based, multi-modal fact-checking of out-of-context images via online resources [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 14920 - 14929 .

THORNE J , VLACHOS A , CHRISTODOULOPOULOS C , et al . FEVER: a large-scale dataset for fact extraction and VERification [C ] //Proceedings of the Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) . Stroudsburg : Association for Computational Linguistics , 2018 : 809 - 819 .

IMRAN M , OFLI F , CARAGEA D , et al . Using AI and social media multimodal content for disaster response and management: opportunities, challenges, and future directions [J ] . Information Processing & Management , 2020 , 57 ( 5 ).

WANG Y Q , MA F L , JIN Z W , et al . EANN: event adversarial neural networks for multi-modal fake news detection [C ] // Proceedings of the Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM press , 2018 : 849 - 857 .

ZHOU X Y , WU J D , ZAFARANI R . $$\mathsf{SAFE}$$: similarity-aware multi-modal fake news detection [M ] . Springer International Publishing , 2020 : 354 - 367 .

XUE J X , WANG Y B , TIAN Y C , et al . Detecting fake news by exploring the consistency of multimodal data [J ] . Information Processing & Management , 2021 , 58 ( 5 ).

SUN M Z , ZHANG X , MA J Q , et al . Inconsistency matters: a knowledge-guided dual-inconsistency network for multi-modal rumor detection [C ] // Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2021 . Stroudsburg : Association for Computational Linguistics , 2021 : 1412 - 1423 .

李婧文 , 李雅文 . 深度合成技术应用与风险应对 [J ] . 网络与信息安全学报 , 2023 , 9 ( 2 ): 184 - 190 .

LI J W , LI Y W . Application and risk response of deep synthesis technology [J ] . Chinese Journal of Network and Information Security , 2023 , 9 ( 2 ): 184 - 190 .

TAN R , PLUMMER B , SAENKO K . Detecting cross-modal inconsistency to defend against neural fake news [C ] // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Stroudsburg : Association for Computational Linguistics , 2020 : 2081 - 2106 .

ZELLERS R , HOLTZMAN A , RASHKIN H , et al . Defending against neural fake news [C ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . Red Hook : 2019 : 9054 - 9065 .

BITEN A F , GOMEZ L , RUSIÑOL M , et al . Good news, everyone! context driven entity-aware captioning for news images [C ] // Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2019 : 12458 - 12467 .

FUNG Y , THOMAS C , GANGI REDDY R , et al . InfoSurgeon: cross-media fine-grained information consistency checking for fake news detection [C ] // Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 1683 - 1698 .

LEWIS M , LIU Y H , GOYAL N , et al . BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension [C ] // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 7871 - 7880 .

RADFORD A , KIM J W , HALLACY C , et al . Learning transfer-able visual models from natural language supervi-sion [C ] // Proceedings of the International Conference on Machine Learning . Virtual Event : PMLR , 2021 : 8748 - 8763 .

ALDAYEL A , MAGDY W . Stance detection on social media: state of the art and trends [J ] . Information Processing & Management , 2021 , 58 ( 4 ).

ZUBIAGA A , AKER A , BONTCHEVA K , et al . Detection and resolution of rumours in social media: a survey [J ] . ACM Computing Surveys , 51 ( 2 ): 32 .

YAO B M , SHAH A , SUN L C , et al . End-to-end multimodal fact-checking and explanation generation: a challenging dataset and models [C ] // Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM press , 2023 : 2733 - 2743 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2016 : 770 - 778 .

ZHOU B L , LAPEDRIZA A , KHOSLA A , et al . Places: a 10 million image database for scene recognition [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018 , 40 ( 6 ): 1452 - 1464 .

REIMERS N , GUREVYCH I . Sentence-BERT: sentence embeddings using siamese BERT-Networks [C ] // Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Stroudsburg : Association for Computational Linguistics , 2019 : 3982 - 3992 .

SUKHBAATAR S , SZLAM A , WESTON J , et al . End-to-end memory networks [C ] // Proceedings of the 28th International Conference on Neural Information Processing Systems . New York : ACM , 2015 : 2440 - 2448 .

WANG B , KUO C C J . SBERT-WK: a sentence embedding method by dissecting BERT-based word models [J ] . IEEE/ACM Transactions on Audio , Speech, and Language Processing, 2020 ( 28 ): 2146 - 2157 .

LI L H , YATSKAR M , YIN D , et al . VisualBERT: a simple and performant baseline for vision and language [EB ] . arXiv preprint arXiv: 1908. 03557 . 2019 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰