Test Fitness

The average item difficulty on the right mark scored (RMS) test was 84%. That is how well the students prepared for the test. 

The test fitness, the average estimate of the Minimum and Maximum number of answers marked for each question (2.2) is how well the test fit student preparation. Test fitness is then 46%. The test design value of one out of four or 25% is lower than the test fitness estimate.

This test functioned close to a true/false test. The test fitness is the average test score when students discard wrong answers they know are wrong and then guess for a right answer from the remaining items. Knowing and guessing, quality and quantity are intermingled .

Right Mark Scoring (RMS)

The average item difficulty on the knowledge and judgment scored (KJS) test was only 73%. Test fitness was 3.1 marks or 32%. The test design value of 1 out of 5 or 20% is again lower than the test fitness estimate. 

Multiple-choice tests are easier than their design values. The KJS tally analysis indicates the complex make up of the average test score. Only 16% of the items were answered by just guessing. Some 35% were recording mastery. Some 43% of the items each split the class into two groups in which one group did significantly better than the other group. What, who and why? There was only one misconception item on the test at the end of the semester.

With KJS knowing and guessing are clearly identified. Quality and quantity are assessed independently. This can be summarized on a single page: Test Performance Profile.

Knowledge and Judgment Scoring (KJS)