We wanted to test the progress of medical students at our university in a pharmacology course. The formal teaching was given as lectures to the full class of students. We gave the very same written test of multiple-choice (MC) questions (single best choice) to third-year medical students before and after a one semester course of basic pharmacology. The initial voluntary test (containing 30 MC questions) was taken by 79% of the eligible students ( n = 147), a week before pharmacology lectures had started. Defining a passing grade of 60% of right answers, only 2% of the students passed the test. The range was between 5 and 21 points. The final, now obligatory, written test at the end of the course (one week after the last lecture in pharmacology) was taken by all students in the semester ( n = 179) and was passed by 95%, of students, again defined by the same passing score. Here, the points obtained ranged from 12 to 29. Over the time of the semester, the attendance in the lectures dropped dramatically to less than 10% of the students. Hence, progress tests are useful, but they hardly measure the gain in knowledge through attendance in the pharmacology lecture (the intervention); they also measure other sources of knowledge, such as textbook reading or memorizing only the initial questions and looking up the answers.