Detection of anomalous program behaviors based on hidden Markov models

Xiangliang Zhang*, Wei Wang, Xiaohong Guan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

To improve detection accuracy, a new intrusion detection method with high efficiency was presented. The method is based on hidden Markov model (HMM) to profile normal program behaviors using traces of system calls generated during the normal execution of processes. At the stage of anomaly detection, a testing trace of system calls is divided into short system call sequences by moving along the trace with a sliding window. The output probability of a short system call sequence embedded in the testing trace is calculated based on the normal model. If the output probability of a short system call sequence exceeds a preset threshold, the short system call sequence is identified as a mismatch. If the ratio of the number of mismatch system call sequences to the number of all sequences embedded in the trace exceeds another preset threshold, the trace is then considered as an intrusion. Experimental results show that the proposed method improves the detection accuracy by at most 590% compared to both Forrest's and Lee's methods.

Original languageEnglish (US)
Pages (from-to)1056-1059
Number of pages4
JournalHsi-An Chiao Tung Ta Hsueh/Journal of Xi'an Jiaotong University
Volume39
Issue number10
StatePublished - Oct 1 2005

Keywords

  • Anomaly detection
  • Hidden Markov model
  • Intrusion detection
  • System call

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint

Dive into the research topics of 'Detection of anomalous program behaviors based on hidden Markov models'. Together they form a unique fingerprint.

Cite this