Did someone just forget to test the new IIM Prometric (Online Test Taking System)


Oh Boy!! A Saturday morning cannot be more eventful and worrying then waking up to a news (or rather a tweet) which says,

CAT servers crash in Chennai, Bangalore, Bhopal and Lucknow (via @ndtv)

Websites crash all the time and so do one of the best made software’s.But, when a software fails at its very first live execution, it is a little worrisome to say the least. More so, when the test system is used for selecting the best brains of India for admission to one of the most reputed MBA colleges in the whole world.


IIM’s all around this year decided to move to an online test taking system moving away from the usual paper-pen style.The move was to reach to a much wider audience and further streamline the logistics. A good online system makes it easier for the test takers as well as the governing body to ensure a hassle free and speedy results. So, when IIM’s decided to take the route of online tests like the GRE and GMAT, the general perception was that things are moving in the right direction.

The Online test solution provider Prometric was the chosen one for conducting the CAT tests online. From the looks of it, they are experienced players in the domain with a healthy lists of online tests they offer. But, then with CAT exams , did PROMETRIC just forget to load test the system for server optimization.

Why the server crash came as a surprise is the fact that the number of concurrent users was very much known to the IIM’s and the Prometric people. Now, there are cases of websites and software’s that crash with a surprising spike in the traffic with loads of users using the system at the same time. But, the number of users for this system was decided, wasn’t it? Around 250,000 students will be taking the CAT this year and given a 10 day window for the exams, the server load on one particular day should not be too huge.

IIM’s saw a decrease in applications this time around and the application deadline has to be extended to allow more students to register. But with this server crash today and cancellation of exam, my heart goes out to the test takers who would have been preparing for this day with hard work and dedication. Even though the tests will be re-conducted, the tempo and the mindset takes a hit.

Further details are not out as yet. With the servers failing on the very first day of the CAT exam, it will be a daunting task for the IIM’s and prometric to launch a damage control strategy and ensure the smooth functioning of the test system for the rest of the exam window.

The stakes are super high with an examination like CAT. Hence, any change in the procedure calls for a thorough test run and a contingency plan in place. It is surprising that in this case, there is no evidence of even one of them being in place as of now.

In fact, according to Satish Deodhar – IIM faculty and convener of CAT exams,

“This is not an internet-based but computer-based test, so there is no question of any server crash. Prometric has ensured the strictest security measures for the exams.”

What are your thoughts on this ?

The IIM’s and Prometric should have put more stringent quality checks before taking the system live. Moreover, couldn’t there have been backup servers to ensure that the tests could be conducted should something happen to the existing servers

  1. Tina Gupta says

    Hi! this is Tina from Mumbai, ya nice write up, I did my MBA from Sikkim Manipal University of Distance Education, they have awesome courses and the great thing is that they have no entrance exams and the fees are reasonable too and they have their centers all over.

    I enjoyed studying at SMUDE, and their website http://www.smude.edu.in will clear all your doubts surely.


    1. vikrant rao says

      *spam* ^^

  2. Ankit says

    Guys sorry for not being able to join the discussion earlier

    I agree with the fact that be it anything, software is bound to fail at times.Windows has a history to crash on launches.But, then it is the Plan-B which is crucial and strange that the elite IIM management dint think of it.Infact, the statement from the convenor looks cocky.

    A college which helps you make the best B-Plan did not think of a Plan-B :-)

  3. Madhav Shivpuri says

    @Vikrant, Looks like they preach, not practice :-)

    With an incident of this nature IIM’s name will take a (slight?) beating and so it is not easy to restore. Thankfully they are not a software company otherwise this could have been lethal.

  4. Vikrant says

    Heard about load testing? back up plan? Anybody in IIM?

    1. vikrant rao says

      Yes they should obviously have had a back-up plan, im in no way trying to cover for the goofup, obviously they didnt do their homework

  5. vikrant rao says

    Correct me if i am wrong here but what seems to have failed here are the client’s systems, when upon completion of the exam submit the results to the server (this seems to have gone fine) now why the tests failed on the client systems needs to be investigated, was it a software glitch, or hardware malfunction ( we know how well institutes maintain their systems) or both.

  6. sharib says

    what about guys who have to take their end semesters later this month….

    how will they cope up?

  7. Kiran says

    internet based, server based..blah blah blah…a production box is always more complicated than a severely load tested testing box – usually close to impossible to have a smooth production considerin the number of users and probably the complexity of algorithms to choose a question.

    1. Ankit says

      true kiran!! live environments are always full of surprises.BUt, what beats me is that there was no backup plan in place.

      Moreover,what we are talkin bout is test to one of the most premier institute in the whole world.So, things could have been planned and executed better.Moreover it is not an adaptive test like GMAT so the complexity is lesser IMO

  8. Arun Basil Lal says

    On the bright side, since they discovered the flaw even before conducting one exam, its good on the part of the students. It would have been even more painful if the test had to be re-conducted.

    as @diovo pointed out in twitter: “The most painful bugs are caught by the users, not the testers.”

    1. Ankit Agarwal says

      Hell no man, today was the test and thousands of candidates came all prepared to write the test.For them, they will have to choose a new slot to write the test, so for them it is a re-test for sorts.The whole mental make-up is going to take a hit.

      I read @diovo’s tweet and replied.For this reason alone, there are terms like “Beta” and “User Acceptance Testing”.Moreover, this is an issue on server load and not the software failing.So, this could have been pretty much taken care via load testing of servers

  9. raakshas says

    Yea, it’s hard to believe that they hadn’t load tested their servers before such an important exam.

    1. Ankit Agarwal says

      Ya, beats me too!! So much so, when they exactly knew the kind of load that they will be expecting

Leave A Reply

Your email address will not be published.

who's online