As we have moved to virtual briefings, I have increasingly been excerpting short video segments (with permission) as part of my Analyst Cam series.
This time it is Paul Greenberg, Managing Principal, The 56 Group, LLC. He summarizes the results from his latest CRM Watchlist survey. He is a repeat guest and was last on the Burning Platform episode where we discussed generational shifts in content creation and distribution.
This is the 19th edition of his signature, annual project. He describes how the effort has evolved over the years. Lot of factors go into his scoring and they change every year. As he says, he looks at a vendor’s strategy and scores it on how well they executed on what they said they would. He then measures the impact they had on the market. Impact as a company – so not just their architectural excellence or financial performance or social responsibility posture. And it is based on the vendor’s stated strategy and market focus – which allows smaller vendors to be evaluated fairly depending on how they delivered against their geographic, vertical or size of customer focus.
He does factor in what he calls “intangibles”. His scoring for that category is a bit less transparent considering how much he guides vendors about the others. But he shares one nugget - he deducts points for “competition bashing”. Good for him.
He is tough and thorough, but very fair. Does not matter if you are a client of his or a close friend. This year, he evaluated 59 vendors reading every word of the 4,682 pages (yes!) if they were submitted to his guidelines. It is a testament to his market impact that vendors spend so much time and effort on their responses.
He has three winning categories -Winners, Winners with Distinction, and Elite. Only a handful have qualified for the Elite designation over all these years. There were 4 winners with Distinction this time around – Oracle, ServiceNow, Thunderhead and Zoho. He discusses another 12 in the Winners tier. Zoho scored highest in total this year. It also submitted the longest response at 259 pages. Paul insists there is no correlation between size of the response and his scoring.
He also shares the vendors who scored highest for each of his categories and summarizes with 4 promising and 2 concerning trends he gleaned from this year's responses.
I have run competitive evaluations of software vendors and outsourcers for many a client. At Gartner, I ran similar structured reviews for Magic Quadrants and other long-form content. I feel no shame saying I did not ever come close to evaluating 50+ vendors or reading 4000+ response pages for a single deliverable.
Next year, Paul says will be the final Watchlist. I suspect he will get an even larger population of vendors who pay homage by participating. In the blog post he said “Since this upcoming one is the very last one so, if I'm you, I'm registering to take a crack at it. Even if most companies don't win the award, they more often than not appreciate the exercise that you go through to put together the submission because it, when completed, gives you a holistic look at your company that you otherwise do not have. It's worth it for the workout.”
I will absolutely bring him back for to describe the process for the 20th in the series. In the meantime, I need to have him back around his other passion – professional sports. We can discuss Tom Brady’s “un-retirement”, the return of baseball after a near shutout and so much else.
