Research on system log anomaly detection based on deep learning

Yidong WANG; Peishun LIU; Gbin WAN

doi:10.11959/j.issn.2096-109x.2019055

您当前的位置：

首页 >

文章列表页 >

Research on system log anomaly detection based on deep learning

Papers | 更新时间：2024-06-03

- Research on system log anomaly detection based on deep learning
- Chinese Journal of Network and Information Security Vol. 5, Issue 5, Pages: 105-118(2019)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-109x.2019055
  CLC： TP390
- Online First：2019-10，
  
  Published：15 October 2019
- 稿件说明：
移动端阅览
Yidong WANG, Peishun LIU, Gbin WAN. Research on system log anomaly detection based on deep learning[J]. Chinese Journal of Network and Information Security, 2019, 5(5): 105-118.
DOI：

Yidong WANG, Peishun LIU, Gbin WAN. Research on system log anomaly detection based on deep learning[J]. Chinese Journal of Network and Information Security, 2019, 5(5): 105-118. DOI： 10.11959/j.issn.2096-109x.2019055.

摘要

系统日志反映了系统运行状态，记录着系统中特定事件的活动信息

快速准确地检测出系统异常日志，对维护系统安全稳定具有重要意义。提出了一种基于GRU神经网络的日志异常检测算法，基于log key技术实现日志解析，利用执行路径的异常检测模型和参数值的异常检测模型实现日志异常检测，具有参数少、训练快的优点，在取得较高检测精度的同时提升了运行速度，适用于大型信息系统的日志分析。

Abstract

The system log reflects the running status of the system and records the activity information of specific events in the system.Therefore

the rapid and accurate detection of the system abnormal log is important to the security and stability of the system.A log anomaly detection algorithm based on GRU neural network is proposed.Log parsing is implemented based on log key technology.Log anomaly detection is realized by using anomaly detection model of execution path and anomaly detection model of parameter value.The system has the advantages of less parameters and faster training.It improves the running speed while achieving higher detection accuracy

and is suitable for log analysis of large information systems.

关键词

Keywords

references

XU W , HUANG L , FOX A , et al . Detecting large-scale system problems by mining console logs [C ] // ACM SIGOPS 22nd symposium on Operating systems principles . 2009 : 117 - 132 .

YU X , JOSHI P , XU J , et al . CloudSeer:workflow monitoring of cloud infrastructures via interleaved logs [J ] . ACM Sigarch Computer Architecture News , 2016 , 44 ( 2 ): 489 - 502 .

DU M , LI F , ZHENG G , et al . Deeplog:anomaly detection and diagnosis from system logs through deep learning [C ] // 2017 ACM SIGSAC Conference on Computer and Communications Security . 2017 : 1285 - 1298 .

JURAFSKY D . Speech ＆ language processing [M ] . Pearson Education India , 2000 : 35 - 61 .

DU M , LI F . spell:Streaming parsing of system event logs [C ] // IEEE 16th International Conference on Data Mining (ICDM) . 2016 : 859 - 864 .

ZHU J , HE S , LIU J , et al . Tools and benchmarks for automated log parsing [J ] . arXiv preprint arXiv:1811.03509 , 2018 .

HE P , ZHU J , HE S , et al . An evaluation study on log parsing and its use in log mining [C ] // 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) . 2016 : 654 - 661 .

KINGMA D P , BA J . Adam:a method for stochastic optimization [J ] . arXiv preprint arXiv:1412.6980 , 2014 .

MC MAHAN H B , HOLT G , SCULLEY D , et al . Ad click prediction:a view from the trenches [C ] // The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 2013 : 1222 - 1230 .

HE S , ZHU J , HE P , et al . Experience report:system log analysis for anomaly detection [C ] // IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) . 2016 : 207 - 218 .

XU W , HUANG L , FOX A , et al . Online system problem detection by mining patterns of console logs [C ] // Ninth IEEE International Conference on Data Mining . 2009 : 588 - 597 .

OLINER A , STEARLEY J . What supercomputers say:a study of five system logs [C ] // 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07) . 2007 : 575 - 584 .

Views

1622

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Phishing website detection based on two-way feature extraction and adaptive fusion

Anomaly detection in DNS: an empirical study

FL-IoV: comprehensive intrusion detection method for internet of vehicles based on federated learning

Multi-task adversarial attribution method based on hierarchical structure

Research progress of color image steganalysis

Related Author

Guo Kai

Wang Heng

Ma Ziqiang

HOU Ronghao

HE Ming

KONG Kaichuan

CHEN Yong

ZUO Peng

Related Institution

School of Information Engineering, Ningxia University

China Internet Network Information Center

School of Cyber Science and Engineering, Jinan University

Beijing Unionpay Card Technology Co., Ltd., Shenzhen Branch

School of Computer Science, Nanjing University of Posts and Telecommunications

⁰