首页    期刊浏览 2024年11月25日 星期一
登录注册

文章基本信息

  • 标题:Probability estimate and the optimal text size
  • 本地全文:下载
  • 作者:Kostić Aleksandar ; Ilić Svetlana ; Milin Petar
  • 期刊名称:Psihologija
  • 印刷版ISSN:0048-5705
  • 电子版ISSN:1451-9283
  • 出版年度:2008
  • 卷号:41
  • 期号:1
  • 页码:35-51
  • DOI:10.2298/PSI0801035K
  • 出版社:Društvo psihologa Srbije
  • 摘要:

    Reliable language corpus implies a text sample of size n that provides stable probability distributions of linguistic phenomena. The question is what is the minimal (i.e. the optimal) text size at which probabilities of linguistic phenomena become stable. Specifically, we were interested in probabilities of grammatical forms. We started with an a priori assumption that text size of 1.000.000 words is sufficient to provide stable probability distributions. Text of this size we treated as a "quasi-population". Probability distribution derived from the "quasi-population" was then correlated with probability distribution obtained on a minimal sample size (32 items) for a given linguistic category (e.g. nouns). Correlation coefficient was treated as a measure of similarity between the two probability distributions. The minimal sample was increased by geometrical progression, up to the size where correlation between distribution derived from the quasi-population and the one derived from an increased sample reached its maximum (r=1). Optimal sample size was established for grammatical forms of nouns, adjectives and verbs. General formalism is proposed that allows estimate of an optimal sample size from minimal sample (i.e. 32 items).

  • 关键词:corpus linguistics; reliability of text sample
国家哲学社会科学文献中心版权所有