|
Difference
Between CAT and PBT
The time limits for CAT should be different from the time limit on a
PBT.
In the PBT, the number of questions are different from that of a CAT.
The candidate can go back and forth between questions. Answer the easier
ones first and keep the tougher ones for the end. He/she also has a better
view of the question as there is no need to scroll with the mouse.
And most people are a lot more comfortable with paper and pencil than
a computer.
All these factors change the time limits for a CAT.
Test makers can again come to the appropriate time limits through the
previous set of experiments. They make the same student take both exams
(PBT and CAT) and see under what kind of time limits the student is achieving
similar scores in both forms.
A PBT of 30 questions need only 30 questions. A
CAT of 30 questions need more than 100 questions!
Let us look at the case of an exam with three CAT levels; 1,2,3. Let us
assume that 1 is the easiest and 3 is the most difficult. Every student
starts with level 2. If he/she answers correctly he goes to level 3.
Let us say, the student remains there (answers correctly) the remaining
questions. In such a case, the test database should consist of 1 question
from level 2 and 29 from level 3.
Let us consider another student who answers the first one wrong and the
rest all wrong such that he/she stays on level 1. In such a case, the
test needs to have 1 at level 2 and 29 from level 1.
Let us consider another case where a student answers alternate questions
correctly. If the first question is correct, the second one will be level
3. As it is going to be wrong, third one is level 2.
This way, we need 15 questions of level 2 and 15 of level 3. Similarly,
in the other case where first question is wrong, the exam needs 15 of
level 1 and 15 of level 2.
In this case it is assumed that if you and your friend perform exactly
similarly, you should get the same questions. But, it may not happen in
real GRE. The reason for this is that they have a random selection of
questions from the level. So, the first question itself will be different
for different test takers.
In this way, the test taker should have to prepare a much larger pool
of questions than the actual number of questions that the test taker has
to answer.
Algorithm
Algorithm is the logic that the exam uses to switch
the difficulty levels. Now, this is where CATs of different sources differ
significantly. The simple example described before is of course a very
easy to implement.
Its logic is as follows.
-
Ask
a question from level 2, if no questions were asked till then.
-
Else
ask more difficult question than previous one if the answer was right
and if the level was not 3.
-
Ask
level 3 question if previous question was answered correctly and its
level was 3.
-
Ask
less difficult question than previous one if the answer was incorrect
and if the level was not one.
-
Ask level
1, if previous question was correct and its level was 1.
What
are the problems with this algorithm?
Well, there are many. But let me point out two of
them.
ETS
algorithm is a lot more complicated.
It is not available for inspection. I personally
answered a number of tests to reverse engineer it as much as possible.
I will discuss them in a later section and show you how to make most of
it.
Even though the procedure is complex, the algorithm fixes you with a particular
final level after taking you through questions at various levels.
It builds some inertia in this process.
Let me explain what that
means:
If you answer a series of questions correctly and then make a mistake
it punishes you lot less than if you make a mistake after answering only
a few questions correctly. It assumes that you made a mistake by
mistake.
Similarly, if you answer a row of questions incorrectly and answer one
correctly, it adds a lot less to your score than if you answered that
question correctly after only a few mistakes. It assumes, in this
case, that you got the correct answer by fluke.
Is CAT better than a
PBT?
Why is ETS going for a CAT as opposed to a
PBT?
CAT is a lot more easier to administer. Less wastage of paper, instant
declaration of results, ability to conduct the exams more often are the
main advantages.
It also minimizes mal-practice during exam as people can't copy from each
other.
However, significant amount of prerequisite research, more number of questions
for the same length of test and the need for better infrastructure (faster
computers with CD ROMs) are some of the issues that need to be tackled
before converting any exam in to a CAT.
As ETS has all these, it is going for the
CATs.
|
