The deceptive nature of amateur reviews

Life used to be simple and somewhat orderly. Hotels were classified as one through five stars (or diamonds) by reputable agencies such as AAA or Mobil/Forbes. Classification was based upon a very specific set of criteria for both service and physical operations. Annual inspections were exciting times on property with staff anxiously awaiting any changes in classification, for better or for worse.

No one believes that inspections only take place once or twice a year. Each and every guest that crosses your transom is an inspector, fully capable of providing an instantaneous blow-by-blow of their stay. Furthermore, these ratings often include personal or emotional bias, something that would never be included by a professional evaluator. I have read many of these oft slapdash or maligned reviews, and perhaps you have your own war stories. 

Take, for example, a one-time billing incident, leading to an assignment of a three-star rating, even though the guest admitted it was a terrific stay. But one issue – one not even highly or marginally repeatable – nevertheless lowered the overall score significantly. Or another example, where the hotel rating is substantially downgraded by the restaurant’s shortcomings, even though the F&B outlet is a completely independent standalone entity. (And even if the restaurant’s ownership was the same, what does this have to do with the physical accommodation’s efficacy?)

Many of us tend to look at the average score or ranking. Take wine: The higher the score, generally the better the product. We know how highly Wine Spectator’s ratings (out of 100) are prized by wine producers. A wine that delivers a score close to 100 (perfect) is voraciously sought after. Hoteliers are looking for this perfect score too. I have been in planning meetings where managers quote their TripAdvisor rating as if they were quoting Robert Parker (of “The Wine Advocate” fame by the way)!

Like wine, most hotel ratings are simple to understand, with a maximum score of five stars. The challenge is that all hotels and resorts are lumped together in the same database, just like wine. So, it is quite possible to have a Holiday Inn Express ranked in direct comparison to a Four Seasons Hotel within the same city. Even without passing judgment on either brand, any hotelier knows that the guest experiences at these two properties will not be the same. Yet, it is possible to have the two rated very similarly, or the HIE rated (numerically) even higher. Why? One easy explanation: The Four Seasons’ guests have much higher expectations than those habituating the HIE, and they will punish the luxury hotelier for any perceived impropriety.

To understand why ratings are deceiving, consider two hotels: Property A and Property B, respectively 4.2 stars and 3.8 stars on an online review site. Say you’re traveling to a city where you have zero experience. Now, for a moment, pretend you are not a hotelier, and you have very limited knowledge of hotel brands (say they were all in a foreign language).

Based on the two ratings above, would you choose Property A or Property B? Without knowing the details, I believe we all would gravitate towards the higher rated property. That’s common sense. Next, add the possibility that Property A is lower priced than Property B, and you might think that you have uncovered a bargain. Remember, you know nothing about either property other than what you see on the review site.

Now try again: Property A is a limited service hotel whereas Property B is full-service luxury. Same ratings, but now you have a little bit more information. So, which one would you choose? Are you still prepared to go with the higher rated property, even while recognizing that it offers considerably less services? How about if I told you that there was no statistical difference between a 4.2 and 3.8 star rated hotel. Would this change your mind?

The bottom line is this: These are amateur ratings, not done by professional reviewers. Your standards for a 5-star may be different than mine. They are not perfect, and can be misleading to the viewer: so worry less about your actual number, and more about what the individual rater says in the desccription. It’s the qualitative that counts millions over the quantitative.

Think about the motivation for amateur review writing. Some are written by review hounds who delight in helping others with a seemingly objective review. But many arise due to emotional extremes – someone who is immensely distraught or someone who is overwhelmingly enamored, both instances where the guest is borderline obligated to write about the hotel. Knowing the motivation – that they write uncompensated and on their own free time – tells you that what they write about will almost certainly be what’s top of mind from their stay.

Emotional reviews might not be objective, but they will definitely shine a light on any issues that are especially bothersome to guests. In other words, if a visitor takes the time to draft a review online, then what they comment on should be considered imperative to improving your hotel. Thinking qualitatively (and, of course, look for commonalities in the reviews) and you will undoubtedly boost your overall rating and increase bookings.