Survey: which analyst firms are moving up (or down)?

AnalystValueSurvey

Everyone in or around the analyst industry knows that the four major firms are Gartner, Forrester, IDC and Ovum. But who are four after that, and the four after that?

To start you off, here’s one of the charts from the presentation.We asked participants which of 30 firms were the most influential on the media and on buyers. We have ranked each firm from 1st place (Gartner) to 30th (ABI – don’t shoot the messenger guys!). You can see that there are some firms, like Tower, who are punching below their weight with the media, and others like 451 who are punching above their weight. Click on the image to enlarge it.

The data come from our Analyst Value Survey (and, to say thank you, participants in the survey get a free seat in the webinar). 198 users of analyst research told us about the topics and analyst firms they are using, which ones they expect to use more, and which firms are the most influential on investors, buyers, the media — and one themselves. Around 20 of those were themselves staff at analyst firms, and we’ve filtered out their responses.

One of the most fascinating questions asks which firms are most, and least, independent of their research findings: some firms are rated highly for both.

Analyst wonks may be wondering how we made the chart. There’s an outline of the Analyst Value Survey at analystequity.com/1834. In the survey we asked respondents to select up to four firms that are most influential on buyers and, in a separate question, which were most influential on the media. We produced an ordinal rank using those data, in which 1st place is higher and 30th place was lowest.

Some of participants in our survey were analysts, either in analyst firms or in-house analysts for vendors or telecoms providers. We’ve subtracted the respondents from analyst firms from the charts. It doesn’t make much of a difference, and we don’t think those analysts were pushing their own firms. However, more folk wanted to see the data without them — and we’re happy to oblige.

P.S. In September 2013 we ran the survey again. To get a copy of the new findings, go to slideshare.

Survey: which analyst firms are moving up (or down)?
0 votes, 0.00 avg. rating (0% score)

18 Comments

  • Reply January 17, 2013

    Bram Weerts

    Great overview Duncan!

  • Reply January 18, 2013

    John Furrier

    That survey is not consistent with the data that we have acquired. I think that your methodology is completely flawed. If you’re not using sample size > 25,000 then its irrelevant and if not broken down by vertical it’s useless.

    Sorry but this is irrelevant survey. For example in a few of the emerging verticals Wikibon dominates the influencer model.. just saying

    • Reply January 19, 2013

      Duncan

      Hey John. I’m honoured to have you visit my blog. Many thanks for commenting. I’m sure there are lot of ideas we can get from you and other readers about how to improve the survey in 2013: we’ve been running it since 2002, and maybe it’s the right time to increase the number of respondents dramatically. I know SiliconANGLE works with really large panels of hundreds of thousands of people and I think it would be great to put those questions out to more people.

  • Reply January 22, 2013

    Duncan Chapple

    I can update readers a little on John’s comment above. I’ve had an exchange of emails with him. He’s a busy guy but I’m hoping to speak with him in a couple of weeks. John’s not surveyed his readers about analyst firms, but his feeling is that the sample size needs to be larger. I raised the idea of a 25,000+ sample with a Gartner alum yesterday who doubts that there are 25,000 seat holders at all the analyst firms added together, and it would be hard to get more than one or two percent of those to respond to an unincentivised survey. Any suggestions?

  • Reply January 22, 2013

    John Furrier

    Duncan,
    Great to meet you and I don’t mean to be critical just want to start a positive conversation. The old guard is losing share. Yes there is no doubt marketshare and leadership both in terms of actual share and mindshare but I see data that shows a rapid decline in the Gartner and IDC shares. Mainly IDC who I am fond of a firm having grown up in Massachusetts. However there is a real trend for “open” free research. The market is changing and so are the firms. New brands are emerging.

  • Reply January 23, 2013

    Duncan Chapple

    Hi John. Can you share with us, or point us to, the data that shows that Gartner’s share is falling? Speaking literally, Gartner’s share is up: it’s $50 now, up from $38 last January; $35 the January before that; $21 the January before that; and $14 the January before then. Gartner’s revenue was $1.5bn last year – a record.

    The challenge is that buyers are more cautious, and they are using more research. It’s hard to track the ‘free’ research. Of course much of it is neither free nor open: for every Wikibon (which is really open) are more vendor funded firms which claim to be open but are not. See http://www.analystequity.com/808

    However, the rise of those small firms doesn’t deny the probability that – since the income of the major analyst firms continues to rise – their influence is growing, at least in their client base.

    What do your data show?

  • Reply January 25, 2013

    Jim Lundy

    Duncan: I’ll add a few comments as the CEO of analyst firm you missed. My firm Aragon Research is only 1.5 years old, but we are all ex-Gartner. You also missed Altimeter Research, slightly older and almost all ex-Forrester. I would also add that your focus seems to be on analyst firms that have a foothold in Europe. Aragon Research and Altimeter are both based in Silicon Valley.

    The ex-factor is something that won’t come out in your surveys because the big firms will never disclose who leaves and that is something I’ve discussed in my blog. The fact is that both Forrester and Gartner have had large defections of their thought leaders, with Forrester having suffered the most losses. I have tracked some of that and will be publishing those results soon.

    The other assumption you make is that because the stock price has gone up, that means they are doing great. It does mean that the firms are now much better managed than even a few years ago. It doesn’t mean that the product or service has gotten better. Ask any vendor that has been on the receiving end of a nasty letter from the Gartner Compliance department.

    After having one of most productive research teams at Gartner (Social Collaboration), I left and started Aragon Research because I knew I could provide clients with better service, the kind of service that used to be provided at my own firm.

    There is more to this story than a short response like this can offer. Look forward to interacting and now you may realize that there is actually more going on in in the analyst firm land than how you have portrayed it.

  • Reply January 25, 2013

    Teri Green

    great info and validation as to what we are also seeing in the marketplace (at least in our market space) – thank you Duncan!

  • Reply January 31, 2013

    Duncan

    Jim raises some important points and – as I’ve mentioned to him – I’ll work up a separate post to discuss them next week. That said, there’s one thing that wasn’t clear in my article. The survey had free text fields so people could add in firms and, indeed, a few firms came up that I didn’t have on the tip of my tounge. So the firms mentioned in the webinar aren’t just our selection: they represent the respondants’. They’re not perfect, but it’s worth saying that it’s not as simple as Lighthouse missing them out or overlooking them.

  • Reply April 13, 2013

    Steve Brown

    In terms of weighing influence on journalists, what types of questions are you asking? And, are you in any way evaluating the number of times analysts are quoted in news stories? I could see a journalist indicated that they are greatly influenced by a boutique analyst but then perhaps turning to one of the heavy-weights for an actual quote. Knowing how you factor and evaluate this would be helpful. Also, what publications are you asking? IT is fairly broad and clarification of the types of publications and reporters would be appreciated.

  • Reply April 14, 2013

    Duncan

    Hi Stuart. We asked participants which analyst firms were most influential on journalists. The participants are users of analyst research: in the main they are not also journalists. In a separate study, the Analyst Impact Model, we do evaluate the profile of analysts in the media, using a rather complex model which accounts for different kinds of coverage.

  • [...] Analyst Value Survey results: what’s the analyst [...]

  • [...] Many of his posts provide helpful insights about the firms. A sample of an interesting post – Survey: Which Analysts Firms are Moving Up or Down (includes a graph with 30 analyst [...]

  • Reply September 5, 2013

    Lena Ahad

    Good overview and considering this information is shared for free I can’t see why some people are complaining! Thank you.

  • Reply October 7, 2013

    Evan Quinn

    I think there are few things that have to be understood when interpreting Duncan’s model:

    1. Do not, as people do with the Magic Quadrant, jump to the conclusion that upper right is “good” and lower left isn’t so good, or a firm not on the chart means the firm isn’t at all relevant. A couple of years ago a study was done by media consultants, and by far the leading analysts in terms of media influence were indie analysts. None of those analysts are even in Duncan’s study. Neither are bloggers who do not fall into media or analyst camps. You must take the study for what it is; what the respondent, purpose and geographic sample is based on, and thus with a big grain of salt.

    2. “Influence” should not equate to “whom to invest in” from an AR/influence management perspective, though that is the easy way out. It is one factor but there are other factors. For example, I am an associate over at HfS Research, one of the more “higher rated” firms – good for us. And if I were wearing my AR shoes and was working in the outsourcing and IT services arena, there is no doubt HfS would be very high on my list. But HfS doesn’t cover, for example, telecommunications, semiconductors, hardware, and only covers software to a limited degree, and we have a limited APJ presence. While CEO Phil Fersht might get a kick out of it, an AR person trying to do business with HfS to cover servers in South Korea should have her/his head examined.

    3. There are many small firms not in this study or that do not do particularly well on the study that do absolutely brilliant work in their niche of choice. Perhaps they specialize in just one tech, or one region, or ROI studies, or write great educational white papers. Here is an example: Wikibon, not in the study, has the “Cube” which seems to be at almost every US tech industry event of value, a forum for interviews and video capture/distribution – nobody does this like Wikibon, best in the analyst business. Suppose your CMO came to you and said, “Is there an analyst firm that does interviews and video at industry events really well?” and you pulled out Duncan’s chart. Oops.

    I think those who do kneejerk AR based purely on “influence” and “pay for play” concepts are still doing AR 101. AR people need to be thoughtful, not look for shortcuts, and to consider all the elements. Usually the first is “What does my company need?” (versus what is easy and comfortable).

    Excellent work by Duncan, but it is just a singular PoV based on a limited sample.

  • Reply October 8, 2013

    Duncan

    Hi Evan. Your comments are really useful, thanks. I especially agree that no-one should see these responses as a sample: we are not making any claim that these data represent anything other than the responses of the participants. That said, I would not describe these data as a “model”. This is a chart to show participant’s responses, not a model.

    I don’t know which study by media consultants you refer to, but it would be interesting to know if they compared firms or individuals. This chart shows global data about 30 top firms: by any measure I think any half-serious study will confirm that this chart shows the vast majority of the analyst firms with the most influence on the media. While this chart shows 30 firms, I think you know that the survey also allows people to write in other firms, so there’s no basis on which you can say that the survey tracks none of the leading analysts in terms of media influence.

    PS. For comparison, I was just looking at data from Apollo Research which shows the individual analysts most quoted in the media. The most recent rankings for Europe and the USA are mostly composed of analysts from Gartner, IDC, Forrester, Yankee Group and Constellation; they were all in the 30 top firms in our 2012 survey.

  • Reply October 8, 2013

    Ludovic Leforestier

    Duncan,

    Pardon me if this is a daft question, but I can’t quite comprehend how firms with no or little end-user clients arrive above the line.

    I can only deduct that the 198 men in the street had not idea about the questions?

  • Reply October 8, 2013

    Duncan

    There are no daft questions Ludovic. Most of the respondents give their address so they can get the full results, and they are by and large an expert audience. You’ll recognise most of the names. I think there are a few reasons why a firm you think has few end-user clients is above the line: some participants could be mistaken; you could be mistaken; some firms could have low scores for both, with one marginally higher; and certainly analysts can influence non-clients both directly and indirectly. There’s a nice paper on analyst influence at http://www.academia.edu/405853/How_strongly_do_industry_analysts_influence_buyers_of_high_technology

Leave a Reply

Leave a Reply