Cyber security entity recognition method based on residual dilation convolution neural network

Bo XIE; Guowei SHEN; Chun GUO; Yan ZHOU; Miao YU

doi:10.11959/j.issn.2096-109x.2020009

您当前的位置：

首页 >

文章列表页 >

Cyber security entity recognition method based on residual dilation convolution neural network

Papers | 更新时间：2024-06-03

- Cyber security entity recognition method based on residual dilation convolution neural network
- Chinese Journal of Network and Information Security Vol. 6, Issue 5, Pages: 126-138(2020)
- 作者机构：
  
  1. 贵州大学计算机科学与技术学院，贵州贵阳 550025
  2. 贵州省公共大数据重点实验室，贵州贵阳 550025
  3. 中国科学院信息工程研究所，北京 100093
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2020009
  CLC： TP391
- Online First：2020-10，
  
  Published：15 October 2020
- 稿件说明：
移动端阅览
Bo XIE, Guowei SHEN, Chun GUO, et al. Cyber security entity recognition method based on residual dilation convolution neural network[J]. Chinese Journal of Network and Information Security, 2020, 6(5): 126-138.
DOI：

Bo XIE, Guowei SHEN, Chun GUO, et al. Cyber security entity recognition method based on residual dilation convolution neural network[J]. Chinese Journal of Network and Information Security, 2020, 6(5): 126-138. DOI： 10.11959/j.issn.2096-109x.2020009.

摘要

近年来，网络安全威胁日益增多，数据驱动的安全智能分析成为网络安全领域研究的热点。特别是以知识图谱为代表的人工智能技术可为多源异构威胁情报数据中的复杂网络攻击检测和未知网络攻击检测提供支撑。网络安全实体识别是威胁情报知识图谱构建的基础。开放网络文本数据中的安全实体构成非常复杂，导致传统的深度学习方法难以准确识别。在BERT（pre-training of deep bidirectional transformers）预训练语言模型的基础上，提出一种基于残差空洞卷积神经网络和条件随机场的网络安全实体识别模型 BERT-RDCNN-CRF。通过BERT模型训练字符级特征向量表示，结合残差卷积与空洞神经网络模型有效提取安全实体的重要特征，最后通过CRF获得每一个字符的BIO标注。在所构建的大规模网络安全实体标注数据集上的实验表明，所提方法取得了比LSTM-CRF模型、BiLSTM-CRF模型和传统的实体识别模型更好的效果。

Abstract

In recent years

cybersecurity threats have increased

and data-driven security intelligence analysis has become a hot research topic in the field of cybersecurity.In particular

the artificial intelligence technology represented by the knowledge graph can provide support for complex cyberattack detection and unknown cyberattack detection in multi-source heterogeneous threat intelligence data.Cybersecurity entity recognition is the basis for the construction of threat intelligence knowledge graphs.The composition of security entities in open network text data is very complex

which makes traditional deep learning methods difficult to identify accurately.Based on the pre-training language model of BERT (pre-training of deep bidirectional transformers)

a cybersecurity entity recognition model BERT-RDCNN-CRF based on residual dilation convolutional neural network and conditional random field was proposed.The BERT model was used to train the character-level feature vector representation.Combining the residual convolution and the dilation neural network model to effectively extract the important features of the security entity

and finally obtain the BIO annotation of each character through CRF.Experiments on the large-scale cybersecurity entity annotation dataset constructed show that the proposed method achieves better results than the LSTM-CRF model

the BiLSTM-CRF model and the traditional entity recognition model.

关键词

Keywords

references

SHU X , ARAUJO F , SCHALES D L , et al . Threat intelligence computing [C ] // ACM SIGSAC Conference on Computer and Communications Security . 2018 : 1883 - 1898 .

王通 , 艾中良 , 张先国 . 基于深度学习的威胁情报知识图谱构建技术 [J ] . 计算机与现代化 , 2018 ( 12 ): 21 - 26 .

WANG T , AI Z L , ZHANG X G . Knowledge graph construction of threat intelligence based on deep learning [J ] . Computer and Modernization , 2018 ( 12 ): 21 - 26 .

刘浏 , 王东波 . 命名实体识别研究综述 [J ] . 情报学报 , 2018 ( 3 ): 329 - 340 .

LIU L.WANG D B . A review on named entity recognition [J ] . Journal of the China Society for Scientific and Technical Information , 2018 ( 3 ): 329 - 340 .

张晓斌 , 陈福才 , 黄瑞阳 . 基于CNN和双向LSTM融合的实体关系抽取 [J ] . 网络与信息安全学报 , 2018 , 4 ( 9 ): 44 - 51 .

ZHANG X B , CHEN F C , HUANG R Y . Relation extraction based on CNN and Bi-LSTM [J ] . Chinese Journal of Network and Information Security , 2018 , 4 ( 9 ): 44 - 51 .

GEORGESCU T M , IANCU B , ZURINI M . Named-entityrecognition-based automated system for diagnosing cybersecurity situations in IoT networks [J ] . Sensors , 2019 , 19 ( 15 ):3380.

HAMMERTON J , . Named entity recognition with long short-term memory [C ] // The 7th Conference on Natural Language Learning at Hltnaacl . 2003 : 172 - 175 .

COLLOBERT R , WESTON J , BOTTOU L , et al . Natural language processing (almost) from scratch [J ] . Journal of Machine Learning Research , 2011 , 12 ( 8 ): 2493 - 2537 .

SANTOS C N , GUIMARAES V . Boosting named entity recognition with neural character embeddings [J ] . arXiv preprint arXiv:1505.05008 , 2015

CHIU J P C , NICHOLS E . Named entity recognition with bidirectional LSTM-CNNs [J ] . Transactions of the Association for Computational Linguistics , 2016 , 4 : 357 - 370 .

STRUBELL E , VERGA P , BELANGER D , et al . Fast and accurate entity recognition with iterated dilated convolutions [J ] . arXiv preprint arXiv:1702.02098 , 2017

HE J , WANG H . Chinese named entity recognition and word segmentation based on character [C ] // The Sixth SIGHAN Workshop on Chinese Language Processing . 2008 .

LIU Z , ZHU C , ZHAO T . Chinese named entity recognition with a sequence labeling approach:based on characters,or based on words [C ] // International Conference on Intelligent Computing . 2010 : 634 - 640 .

LI H , HAGIWARA M , LI Q , et al . Comparison of the impact of word segmentation on name tagging for Chinese and Japanese [C ] // LREC . 2014 : 2532 - 2536 .

秦娅 , 申国伟 , 赵文波 , 等 . 基于深度神经网络的网络安全实体识别方法 [J ] . 南京大学学报(自然科学) , 2019 , 55 ( 1 ): 29 - 40 .

QIN Y , SHEN G W , ZHAO W B , et al . Research on the method of network security entity recognition based on deep neural network [J ] . Journal of Nanjing University (Natural Sciences) , 2019 , 55 ( 1 ): 29 - 40 .

XU Y , WANG Y , LIU T , et al . Joint segmentation and named entity recognition using dual decomposition in Chinese discharge summaries [J ] . Journal of the American Medical Informatics Association , 2013 , 21 ( e1 ):e84-e92.

ZHANG Y , YANG J . Chinese NER using lattice LSTM [J ] . arXiv preprint arXiv:1805.02023 , 2018

MNIH V , HEESS N , GRAVES A . Recurrent models of visual attention [C ] // Advances in Neural Information Processing Systems 27 (NIPS 2014) . 2014 : 2204 - 2212 .

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [J ] . arXiv preprint arXiv:1409.0473 , 2014

YIN W , SCHÜTZE H , XIANG B , et al . AbCNN:attention-based convolutional neural network for modeling sentence pairs [J ] . Transactions of the Association for Computational Linguistics , 2016 , 4 : 259 - 272 .

WANG L , CAO Z , DE-MELO G , et al . Relation classification via multi-level attention CNNs [C ] // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics . 2016 : 1298 - 1307 .

DEVLIN J , CHANG M W , LEE K , et al . Bert:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint arXiv:1810.04805 , 2018

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 770 - 778 .

LAFFERTY J , MCCALLUM0 -A , PEREIRA F C N . Conditional random fields:probabilistic models for segmenting and labeling sequence data [C ] // Proceedings of the Eighteenth International Conference on Machine Learning . 2001 : 282 - 289 .

HOCHREITER S , SCHMIDHUBER J . Long short-term memory [J ] . Neural Computation , 1997 , 9 ( 8 ): 1735 - 1780 .

PENG N , DREDZE M . Named entity recognition for chinese social media with jointly trained embeddings [C ] // 2015 Conference on Empirical Methods in Natural Language Processing . 2015 : 548 - 554 .

LAMPLE G , BALLESTEROS M , SUBRAMANIAN S , et al . Neural architectures for named entity recognition [J ] . arXiv preprint arXiv:1603.01360 , 2016

MA X , HOVY E . End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF [J ] . arXiv preprint arXiv:1603.01354 , 2016

ZEILER M D . ADADELTA:an adaptive learning rate method [J ] . arXiv preprint arXiv:1212.5701 , 2012

Views

752

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰