摘要：Whenever tests are used for making decisions (so-called 'high-stake' tests),different aspects of reliability always become important.If tests are composed of complex items scored by human raters,measurement error increases due to raters' subjectivity.Error variance due to subjectivity of rating process can be estimated through means of generalizability theory which sometimes demand substantial computational strain.Square root of error variance due to subjectivity of rating process is called standard error of rating process (SNOP).The author suggests an alternate computational approach in the special case of two raters for each subject which is faster and easier to compute and should therefore facilitate the use of proposed statistic.Computation of SNOP is shown identical by both approaches on simulated data.The use and interpretation are illustrated by examples of Slovene Matura exams.
其他摘要：Pri preizkusih znanja,ki imajo za kandidate pomembne posledice,je vedno pomembno vprašanje zanesljivosti in njeni različni vidiki.Kadar je preizkus znanja sestavljen iz kompleksnejših nalog,ki jih ocenjujejo ocenjevalci,se napaka merjenja poveča zaradi subjektivnosti ocenjevanja.Varianco napake,ki je povezana s subjektivnostjo ocenjevanja,lahko izračunamo s postopki teorije posplošljivosti,ki pa so v določenih primerih računsko zelo zahtevni.Kvadratni koren variance ocenjevalnega procesa avtor poimenuje standardna napaka ocenjevalnega procesa (SNOP).Alternativni postopek za izračun SNOP,ki ga predlaga avtor v primeru dvojnega ocenjevanja,je hitrejši in enostavnejši in naj bi v praksi spodbudil uporabo predlagane statistike.Izračun SNOP po obeh postopkih je prikazan na simuliranih podatkih,prav tako je prikazana uporaba in interpretacija na primeru maturitetnih izpitov slovenske splošne mature.
关键词：Matura;knowledge tests;reliability;raters' error;generalizability theory
