中文文本分类方法综述

于游; 付钰; 吴晓平

doi:10.11959/j.issn.2096-109x.2019045

您当前的位置：

首页 >

文章列表页 >

中文文本分类方法综述

综述 | 更新时间：2024-06-03

- 中文文本分类方法综述
- Summary of text classification methods
- 网络与信息安全学报 2019年5卷第5期页码：1-8
- 作者机构：
- 作者简介：
  
  [ "于游（1995- ），女，山东威海人，海军工程大学硕士生，主要研究方向为信息安全。" ]
  [ "付钰（1982- ），女，湖北武汉人，博士，海军工程大学副教授，主要研究方向为信息安全风险评估。" ]
  [ "吴晓平（1961- ），男，山西新绛人，博士，海军工程大学教授、博士生导师，主要研究方向为系统分析与决策。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61672531)
- DOI：10.11959/j.issn.2096-109x.2019045
  中图分类号： TP391
- 网络首发：2019-10，
  
  纸质出版：2019-10-15
- 稿件说明：
移动端阅览
于游, 付钰, 吴晓平. 中文文本分类方法综述[J]. 网络与信息安全学报, 2019,5(5):1-8.

You YU, Yu FU, Xiaoping WU. Summary of text classification methods[J]. Chinese Journal of Network and Information Security, 2019, 5(5): 1-8.
于游, 付钰, 吴晓平. 中文文本分类方法综述[J]. 网络与信息安全学报, 2019,5(5):1-8. DOI： 10.11959/j.issn.2096-109x.2019045.

You YU, Yu FU, Xiaoping WU. Summary of text classification methods[J]. Chinese Journal of Network and Information Security, 2019, 5(5): 1-8. DOI： 10.11959/j.issn.2096-109x.2019045.

摘要

如何高效地文本分类是当前研究的一个热点。首先对文本分类概念及流程中的分词、特征提取和文本分类方法等相关技术及研究现状进行了介绍和阐述，然后分析了现有文本分类相关技术面临的挑战，最后对文本分类的发展趋势进行了总结。

Abstract

How to effectively classify text has become a hot topic.Firstly

the concept of text classification

word segmentation

feature extraction and text classification methods were introduced

and the research actuality was summarized.And then the challenges of text classification related technologies were analyzed.Finally

the development trend of text classification was summarized.

关键词

Keywords

references

郑捷 . NLP 汉语自然语言处理原理与实践 [M ] . 北京 : 电子工业出版社 , 2017 .

ZHENG J . NLP chinese natural language processing principle and practice [M ] . Beijing : Publishing House of Electronics IndustryPress , 2017 .

常建秋 , 沈炜 . 基于字符串匹配的中文分词算法的研究 [J ] . 工业控制计算机 , 2016 , 29 ( 2 ): 115 - 116+119 .

CHANG J Q , SHEN W . Research on chinese word segmentation algorithm based on string matching [J ] . Industrial control computer , 2016 , 29 ( 2 ): 115 - 116+119 .

邹佳伦 , 文汉云 , 王同喜 . 基于统计的中文分词算法研究 [J ] . 电脑知识与技术 , 2019 , 15 ( 4 ): 149 - 150+1５３ .

ZOU J L , WEN H Y , WANG T X . Research on chinese word segmentation algorithm based on statistics [J ] . Computer Knowledge and Technology , 2019 , 15 ( 4 ): 149 - 150＋153 .

刘群 , 张华平 , 俞鸿魁 , 等 . 基于层叠隐马模型的汉语词法分析 [J ] . 计算机研究与发展 , 2004 ( 8 ): 1421 - 1429 .

LIU Q , ZHANG H P , YU H K , et al . Chinese lexical analysis model using cascading hidden horse model [J ] . Journal of Computer Research and Development , 2004 ( 8 ): 1421 - 1429 .

孙茂松 , 陈新雄 , 张开旭 , 等 . THULAC:一个高效的中文词法分析工具包 [R ] . 2016 .

SUN M S , CHEN X X , ZHANG K X et al . THULAC:an efficient Chinese lexical analysis toolkit [R ] . 2016 .

熊回香 , 叶佳鑫 . 基于同义词词林的社会化标签等级结构构建研究 [J ] . 情报杂志 , 2018 , 37 ( 1 ): 126 - 131 .

XIONG H X , YE J X . Research on t Structure construction of social tags TongYiCi CiLin [J ] . Journal of Information , 2018 , 37 ( 1 ): 126 - 131 .

XIE R , YUAN X , LIU Z , et al . Lexical sememe prediction via word embeddings and Matrix factorization [C ] // The 26th International Joint Conference on Artificial Intelligence . 2017 : 4200 - 4206 .

唐籍涛 , 李飞 , 郭昌松 . 网络舆情监控中新词识别问题的研究 [J ] . 计算机技术与发展 , 2012 , 22 ( 1 ): 119 - 121+125 .

TANG J T , LI F , GUO C S . Research on new word pattern recognition in network monitoring public opinion [J ] . Computer Technology and Development , 2012 , 22 ( 01 ): 119 - 121+125 .

刘泽文 , 丁冬 , 李春文 . 基于条件随机场的中文短文本分词方法 [J ] . 清华大学学报(自然科学版) , 2015 , 55 ( 08 ): 906 - 910+915 .

LIU Z W , DING D , LI C W . A chinese short text word segmentation method based on conditional random fields [J ] . Journal of Tsinghua University(Science and Technology) , 2015 , 55 ( 8 ): 906 - 910+915 .

杜丽萍 , 李晓戈 , 于根 , 等 . 基于互信息改进算法的新词发现对中文分词系统改进 [J ] . 北京大学学报(自然科学版) , 2016 , 52 ( 01 ): 35 - 40 .

DU L P , LI X G , YU G , et al . New word detection based on an improved PMI algorithm for enhancing segmentation system [J ] . Journal of Peking University(Natural Science) , 2016 , 52 ( 01 ): 35 - 40 .

胡婕 , 张俊驰 . 双向循环网络中文分词模型 [J ] . 小型微型计算机系统 , 2017 , 38 ( 3 ): 522 - 526 .

HU J , ZHANG J C . Bidirectional recurrent network for chinese word segmentation [J ] . Journal of Chinese Computer Systems , 2017 , 38 ( 3 ): 522 - 526 .

ZHANG Y N , XU J N , MIAO G Y , et al . Improving neural chinese word segmentation using unlabeled data [J ] . IOP Conference Series:Materials Science and Engineering , 2018 , 435 ( 1 ).

杨光豹 , 杨丰赫 , 毛贵军 . 基于分组 hash 与变长匹配的中文分词技术 [J ] . 计算机时代 , 2019 ( 4 ): 52 - 55 .

YANG G B , YANG F H , MAO G J . Chinese word segmentation technology based on group hash and variable length matching [J ] . Computer Age , 2019 ( 4 ): 52 - 55 .

CHEN X C , SHI Z , QIU X P , et al . Adversarial multi-criteria learning for chinese word segmentation [C ] // ACL2017:Computation and Language . 2017 .

YU C H , WANG S P , GUO J J . Learning chinese word segmentation based on bidirectional GRU-CRF and CNN network model [J ] . International Journal of Technology and Human Interaction (IJTHI) , 2019 , 15 ( 3 ).

DARSHAN S L , JAIDHAR C D . Performance evaluation of filter-based feature selection techniques in classifying portable executable files [J ] . Procedia Computer Science , 2018 , 125 : 346 - 356 .

HANCER E . Differential evolution for feature selection:a fuzzy wrapper–filter approach [J ] . Soft Computing , 2019 , 23 ( 13 ): 5233 - 5248 .

MALDONADO S ， LÓPEZ J . Dealing with high-dimensional class-imbalanced datasets:embedded feature selection for SVM classification [J ] . Applied Soft Computing , 2018 ( 67 ): 228 - 246 .

刘赫 , 刘大有 , 裴志利 , 等 . 一种基于特征重要度的文本分类特征加权方法 [J ] . 计算机研究与发展 , 2009 , 46 ( 10 ): 1693 - 1703 .

LIU H , LIU D Y , YAN Z L , et al . A Feature weighting scheme for text categorization based on feature importance [J ] . Journal of Computer Research and Development , 2009 , 46 ( 10 ): 1693 - 1703 .

靖红芳 , 王斌 , 杨雅辉 , 等 . 基于类别分布的特征选择框架 [J ] . 计算机研究与发展 , 2009 , 46 ( 9 ): 1586 - 1593 .

JING H F , WANG B , YANG Y H , et al . Category distribution-based feature selection framework [J ] . Journal of Computer Research and Development , 2009 , 46 ( 9 ): 1586 - 1593 .

单丽莉 , 刘秉权 , 孙承杰 . 文本分类中特征选择方法的比较与改进 [J ] . 哈尔滨工业大学学报 , 2011 , 43 ( S1 ): 319 - 324 .

SHAN L L , LIU B Q , SUN C J . Comparison and improvement of feature selection methods in text categorization [J ] . Journal of Harbin Institute of Technology , 2011 , 43 ( S1 ): 319 - 324 .

王正琦 , 冯晓兵 , 张驰 . 基于两层分类器的恶意网页快速检测系统研究 [J ] . 网络与信息安全学报 , 2017 , 3 ( 8 ): 48 - 64 .

WANG Z Q , FENG X B , ZHANG C . Study of high-speed malicious web pages detection system based on two-type classifier [J ] . Journal of Network and Information Security , 2017 , 3 ( 8 ): 48 - 64 .

蒋胜利 . 高维数据的特征选择与特征提取研究 [D ] . 西安:西安电子科技大学 , 2011 .

JIANG S L . Research on feature selection and feature extraction of high dimensional data [D ] . Xi’an:Xidian University , 2011 .

李炜 , 巢秀琴 . 改进的粒子群算法优化的特征选择方法 [J ] . 计算机科学与探索 , 2019 ,( 6 ): 990 - 1004 .

LI W , CAI X Q . Improved particle swarm optimization method for feature [J ] . Journal of Frontiers of Computer Science and Technology , 2019 ,( 6 ): 990 - 1004 .

路永和 , 曹利朝 . 基于粒子群优化的文本特征选择方法 [J ] . 现代图书情报技术 , 2011 ,( Z1 ): 76 - 81 .

LU Y H , CAO L C . Text feature selection method based on particle swarm optimization [J ] . New Technology of Library and Information Service , 2011 ,( Z1 ): 76 - 81 .

张杰慧 , 何中市 , 王健 , 等 . 基于自适应蚁群算法的组合式特征选择算法 [J ] . 系统仿真学报 , 2009 , 21 ( 6 ): 1605 - 1608+1614 .

ZHANG J H , HE Z S , WANG J , et al . Hybrid feature selection algorithm based on adaptive ant colony algorithm [J ] . Journal of System Simulation , 2009 , 21 ( 6 ): 1605 - 1608+1614 .

张海涛 . 基于文本降维和蚁群算法的文本聚类研究 [D ] . 合肥:安徽大学 , 2016 .

ZHANG H T . Research on text clustering based on text dimension reduction and ant colony algorithm [D ] . Hefei:Anhui University , 2016 .

卢泓宇 , 张敏 , 刘奕群 , 等 . 卷积神经网络特征重要性分析及增强特征选择模型 [J ] . 软件学报 , 2017 , 28 ( 11 ): 2879 - 2890 .

LU Y Y , ZHANG M , LIU Y Q , et al . Convolutional neural networks importance analysis and feature selection enhanced model [J ] . Journal of Software , 2017 , 28 ( 11 ): 2879 - 2890 .

WU Y , WU W , XING C , et al . Sequential matching network:a new architecture for multi-turn response selection in retrieval-based chatbots [C ] // ACL 2017:Computation and Language . 2017 .

ZHAO Z , LIU T , LI S , et al . Ngram2vec:learning improved word representations from ngram co-occurrence statistics [C ] // Conference on Empirical Methods in Natural Language Processing . 2017 : 244 - 253 .

赵芃 . 基于主题模型与深度学习的短文本特征扩展与分类研究 [D ] . 天津:天津工业大学 , 2018 .

ZHAO P . Research on short text feature expansion and classification based on topic model and deep learning [D ] . Tianjin:Tianjin Polytechnic University , 2018 .

孙锐 , 郭晟 , 姬东鸿 . 融入事件知识的主题表示方法 [J ] . 计算机学报 , 2017 , 40 ( 4 ): 791 - 804 .

SUN R , GUO W , JI D H . Topic representation integrated method of integrating with event knowledge [J ] . Chinese Journal of Computers , 2017 , 40 ( 4 ): 791 - 804 .

李森 , 马军 , 赵嫣 , 等 . 对数字化科技论文的自动分类研究 [J ] . 山东大学学报(理学版) , 2006 ( 3 ): 81 - 84 .

LI S , MA J , ZHAO Y , et al . Automatic classification of digital science and technology papers [J ] . Journal of Shandong University(Science Edition) , 2006 ( 3 ): 81 - 84 .

CUI W , . A chinese text classification system based on naive bayes algorithm [C ] // MATEC Web of Conferences . 2016 :1015.

ZHANG M Y , AI X B , HU Y Z . Chinese text classification system on regulatory information based on SVM [C ] // IOP Conference Series:Earth and Environmental Science . 2019 :252.

SAHA D , . Web text classification using a neural network [C ] // 2011 Second International Conference on , 2011 .

雷飞 . 基于神经网络和决策树的文本分类及其应用研究 [D ] . 成都:电子科技大学 , 2018 .

LEI F . Text research on text classification based on neural network and decision tree and its application [D ] . Chengdu:University of Electronic Science and Technology of China , 2018 .

周朴雄 . 基于神经网络集成的Web文档分类研究 [J ] . 图书情报工作 , 2008 ,( 7 ): 110 - 112 .

ZHOU P X . Study on Web document classification based on neural network integration [J ] . Library and Information Service , 2008 ,( 7 ): 110 - 112 .

ZENG D J , LIU K , LAI S W,et.cl . relation classification via convolutional deep neural network [C ] // The 25th international Computational Linguistics : 2014 : 2335 - 2344 .

CHEN Y W , WANG J L , CAI Y Q , et al . A method for chinese text classification based on apparent semantics and latent aspects [J ] . Journal of Ambient Intelligence and Humanized Computing , 2015 , 6 ( 4 ): 473 - 480 .

LI H M , HUANG H N , CAO X , et al . Falcon:a novel chinese short text classification method [J ] . Journal of Computer and Communications , 2018 , 6 : 216 - 226 .

王根生 , 黄学坚 . 基于Word2vec和改进型TF-IDF的卷积神经网络文本分类模型 [J ] . 小型微型计算机系统 , 2019 , 40 ( 5 ): 1120 - 1126 .

WANG G S , HUANG X J . Convolutional neural network text classification model based on Word2vec and improved TF-IDF [J ] . Journal of Chinese Computer Systems , 2019 , 40 ( 5 ): 1120 - 1126 .

王磊 . 基于混合神经网络的中文短文本分类方法研究 [D ] . 杭州:浙江理工大学 , 2019 .

WANG L . Research on chinese short text classification based on hybrid neural network [D ] . Hangzhou:Zhejiang University of Science and Technology , 2019 .

JIN W Z , ZHU H , YANG G C . An efficient character-level and word-level feature fusion method for chinese text classification [C ] // Journal of Physics:Conference Series . 2019 :12057.

WANG P , XU B , XU J M , et al . Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification [J ] . Neurocomputing , 2016 , 174 : 806 - 814 .

龚千健 . 基于循环神经网络模型的文本分类 [D ] . 武汉:华中科技大学 , 2016 .

GONG Q J . Text Classification based on recurrent neural network model [D ] . Wuhan:Huazhong University of Science and Technology , 2016 .

浏览量

3388

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

网络钓鱼识别研究综述

基于LLM的多粒度口令分析研究

基于黑盒水印的NLP神经网络版权保护

新的基于鼠标行为的持续身份认证方法

基于DSR和BGRU模型的聊天文本证据分类方法