It is quite tough to fathom the hold the Gartner folk have on the communications industry. They are, or believe they are, so powerful that even powerful software companies get upset at not being in the right place in one of their Magic Quadrants. Some go to great lengths to try and change it. In the case Netscout last week and ZL in 2009, they reach for their lawyers.
There are three themes that seem to recur with these Magic Quadrant cases.
Is it that Gartner creates their Magic Quadrants at best on subjective data? At worst – allegedly, of course – based on how much consultancy you buy from them, the so called ‘Pay to Play.’ Oddly, they also seem to be fixated on Marketing budgets. According to ‘sources’, a Gartner analyst said to his client that it was going nowhere because it spent virtually nothing on marketing.
They do seem to be attracted to companies with big marketing budgets. Presumably someone has told Gartner that spending millions on marketing in this industry is pretty ineffective (even though most people reluctantly do it at the big shows). What you need to do is begin conversations with key people. Perhaps the analyst meant ‘spent virtually nothing on marketing with Gartner.’
The second is that both Netscout and other companies have been offered the opportunity to see a draft of a Magic Quadrant – and the measurement criteria behind it – and, according to our own sources, when companies have requested amendments to factual information that has been mis-represented or wrong, have been refused or ignored. Another established company was simply left out of the relevant Magic Quadrant. Coincidently, they do not pay Gartner any money. Hardly best practice.
The third is that Gartner may not be as influential as it would like us to think it is. At least not in the BSS arena. Following their original ‘puzzling’ Magic Quadrant, we asked twelve very large operators how influential Gartner was in their IT decision making process. We gave these industry decision makers a simple scale, where ‘1’ equals ‘we completely ignore them’ and ‘5’ was ‘we do exactly what they recommend.’
Not one of those operators we surveyed gave Gartner a 5. Only one gave them a 4 ‘very influential,’ the majority gave them a 3 – ‘we use the information as part of our decision-making process, no more.’
Oddly, part of Gartner’s defence back in 2009 was that the suit was wrong ‘because it attacks opinions expressed by Gartner, Inc. These opinions are constitutionally protected,…’
So, as we thought, Magic Quadrants are apparently just opinions, and in our opinion, opinions formed by junior analysts chatting to people. We wrote to them about this when they produced a second puzzling IRCM Magic Quadrant.
The only thing we can suggest, since picking a fight with a film critic is pretty pointless if you are trying to promote a film, is to give Gartner’s Magic Quadrants the weighting they actually deserve, which, in our part of the industry at least is ‘part of the decision-making process.’ No more.
Unfortunately ‘we use the information as part of our decision-making process, no more’ is likely to mean ‘we use the information to draw up our candidate shortlist’. Why? Because it’s quick, easy and we don’t have to work too hard to justify it to anyone else.
Unfortunately for the vendor, if you don’t make it through that part of ‘the decision-making process’, it could have a serious impct on your business. Not surprising that people take it seriously and are generally wary of riling the judges.