首页    期刊浏览 2024年11月28日 星期四
登录注册

文章基本信息

  • 标题:Unsupervised Spam Detection by Document Probability Estimation with Maximal Overlap Method
  • 本地全文:下载
  • 作者:Takashi Uemura ; Daisuke Ikeda ; Takuya Kida
  • 期刊名称:人工知能学会論文誌
  • 印刷版ISSN:1346-0714
  • 电子版ISSN:1346-8030
  • 出版年度:2011
  • 卷号:26
  • 期号:1
  • 页码:297-306
  • DOI:10.1527/tjsai.26.297
  • 出版社:The Japanese Society for Artificial Intelligence
  • 摘要:In this paper, we study content-based spam detection for spams that are generated by copying a seed document with some random perturbations. We propose an unsupervised detection algorithm based on an entropy-like measure called document complexity, which reflects how many similar documents exist in the input collection of documents. As the document complexity, however, is an ideal measure like Kolmogorov complexity, we substitute an estimated occurrence probability of each document for its complexity. We also present an efficient algorithm that estimates the probabilities of all documents in the collection in linear time to its total length. Experimental results showed that our algorithm especially works well for word salad spams, which are believed to be difficult to detect automatically.
  • 关键词:unsupervised spam detection ; document complexity ; suffix tree ; maximal overlap method ; word salad
国家哲学社会科学文献中心版权所有