My friend Wil Reynolds from SEER Interactive alerted me to a Harvard Business Review study published in 2011 which claims that “a one-star increase on Yelp leads to a 5 to 9 percent increase in revenue.”
The author of the study looked at ratings on Yelp for Seattle restaurants over the course of several years and compared them to the revenues these restaurants reported to Washington State.
While I don’t dispute the possibility of an increase in a restaurant’s average Yelp rating having this kind of effect on bottom line revenue, I am surprised that the author thinks he can draw causation conclusions from this study.
Yelp is certainly a popular destination for consumers looking to decide where to eat, drink, and be merry. But it’s far from the only opportunity for restaurants to market themselves online, and I don’t think that obsessing over an extra star on Yelp is worth the increase in an average restaurateur’s blood pressure when so many other less stressful opportunities exist that can improve revenue just as much, if not more. Obsessing over Yelp ratings is also a very dangerous game, as the site’s notoriously strict review filter and aggressive stance against review solicitation lead to lots of frustration for business owners.
Yelp ratings simply can’t be considered in a vacuum as the author of this study attempts to do, as far as I can tell. There are so many questions that this method leaves unanswered for me:
- What other kinds of marketing is a restaurant doing?
- Is it the star rating that causes the increase in revenue, or improvements in other factors that increase rankings (and thus visibility to prospective customers) in Yelp’s internal algorithm?
- What does their profile look like on other sites across the internet (say, Google for example)? In his appendices, he hardly looks at Yelp’s strongest competitors (Zagat prior to their purchase by Google, Seattle Times, and Food-and-Wine).
- How is he demonstrating that 5-9% of increased revenue is coming from people who actually saw the restaurant on Yelp?
To be fair, as a non-statistician, I don’t fully understand the “Regression Discontinuity Estimate” the author says he applies to label this effect as causal rather than correlational. It seems like a highly-flawed study, though, and I’m surprised Harvard would put this out.
As a result, I have a couple of questions for the Local U Community:
- Can someone who understands statistics better than I do explain why this is a viable study?
- Do you buy the author’s conclusion of a causal effect of an increase in star ratings?
- Would you recommend that a restaurant owner start his marketing with Yelp?
