In their latest statement released today, the India Institutes of Management claim to have unearthed patterns of problems faced by candidates during the ten-day period of CAT 2009 between November 28 and Dec 8. Asserting that a retest was not needed since only 4% of the test-takers had been affected by the unfair testing conditions, the IIMs have contradicted their own precedent of 2003 when the CAT question paper was leaked to a much smaller number of people and yet a retest was ordered.
The full text of the statement is reproduced below.
“The IIMs opted for computerized CAT 2009 to enhance the quality of test including providing a more conducive environment for testing as compared to paper and pencil test. It is indeed our regret that despite best efforts a number of candidates faced difficulties during this test window of Nov 28-Dec 8, 2009.
About 2.42 lakh candidates registered for CAT 2009. In the first administration window (i.e., from November 28 to Dec 8, 2009) about 2.16 lakh candidates completed the test. About 10% of the total registered applicants (24,000) did not show up for the test – a percentage that is similar to what has been experienced in the past. The remaining 2,000 candidates who were rescheduled remain to be tested.
A very large majority of candidates, over 2 lakhs, completed the test successfully. It is estimated that about 8,000 students may have faced difficulty of various kinds in completing the test. The various problems faced by these candidates have been categorized based on the emails received by the Candidate Care Centre of Prometric and by the IIMs, the telephone calls made by candidates at the Candidate Care Centre, the site and audit reports made by Prometric and NIIT personnel in charge of the labs at the test centre, problems and situations reported by the media, and discussions on blogs and websites. The categories of problems faced by candidates include premature exit due to inadvertent use of “End” button on the review screen in the first few days of testing period; disruptions caused due to malfunctioning of computers; missing graphics and data; and consequent rescheduling of test. Prometric has provided us with data from various sources mentioned above as well as select computer data on the candidates and some video footages. We have also received analysis of computer data identifying candidates who experienced any form of disruption/rebooting of their machine. This is helping us analyze the nature and extent of problems that the candidates faced and identify candidates who were genuinely affected by them.
There are, however, several claims that have been made about many problems and they require some reflection. For instance, the “End” button problem – it appears that barely 500 candidates faced this situation over the 11 day window. More importantly, once a candidate presses the “End” button, a message first appears on the screen which alerts a candidate that there are unanswered questions and asks whether the candidate really wants to exit (i.e., by pressing an “End” button one does not exit automatically). The cursor sits on the reply NO and the candidate has to move the cursor to YES and press it before he/she exits. Besides, from day 2 onwards, all candidates were being alerted through repeated announcements to be careful in using these buttons.
Similarly, video footage exists from every room and every test administration and is now being reviewed to identify labs that were affected, as claimed. It may be stated that the question sequence in any lab is also randomized. Consequently, any two candidates sitting on adjacent computers are most likely to be seeing very different questions at any given point of time. This reduces any chance of gaining from any form of cheating (which would also be identified through the video review).
Unlike testing on a single day (as CAT was in the past), whenever multiple day testing is undertaken, a large number of questions have to be developed with the same level of difficulty. This requires a different technology for generation of tests and establishing their equivalence. All the tests over this 11 day period were dissimilar while asking questions that test similar concepts. However, by design, the test included a few questions that were common in at most two tests. This is a standard practice followed by all tests that are offered over multiple days (including GRE, GMAT, TOEFL etc) to develop equivalence in terms of difficulty between two tests. It helps in making appropriate adjustments for varying difficulty levels thereby making the scores comparable across multiple days of testing. A few common situations may occur across different tests forms but key data has always been different in all and each situation was asking a different question or requiring a different analysis. These may have given an impression that questions have been repeated when that may not be true. This kind of testing process and its evaluation system which is common in other established tests such as GRE, GMAT, and TOEFL needs to be understood.
We would like to inform all the candidates that Prometric uses a robust encryption technology for uploading all the data that will not be affected by any virus. We have been assured that computer data on the test from all candidates have been successfully uploaded from all the labs.
The Directors of IIMs met on 13 December 2009 to take stock of the situation. Based on a detailed review, the directors decided to continue with computerized CAT 2009. This would be fair to a very large percentage (96%) of candidates who successfully completed the test. In addition, to be fair to those who experienced difficulties during the test, the directors decided to provide an opportunity to test in the second half of January 2010. On this testing date, candidates who remain to be tested and all those who, through a review process, are found to have been genuinely affected by various glitches during the first testing period will be asked to appear for the test. IIMs are in the process of setting up the criteria that will be used to identify candidates eligible for the test to be held in January. The candidates thus identified will be informed by Prometric in the coming weeks by email.
The IIMs would like to assure all concerned that through the above process fairness would have been achieved in the conduct of CAT 2009.”