At a Glance
Introduction
Consumer Choice Awards (CCA) approached Polly in 2018 to revamp the selection process for their business category winners. The catalyst for the switch was an increased bias in their results when scraping sites like Yelp & TripAdvisor from employees and trolls gaming the reviews. Polly brought the awards’ focus back to the consumer and touted CCA as the only awards organization that gathers consumer opinions online.
Methodology
Polly properly curates data collected from Google Reviews with a focus on balancing the ranking data to not be biased towards the most popular or largest business in the category, and directly address the following effects:
Halo Bias
Customers will be more likely to rate a business positively if they have a happy experience. Polly’s language model detects the context in which the review was made and adjusts according to the experience.
Recency Bias
Recency bias is the tendency towards rating more recent experiences better than ones in the past. Polly can go back in time for a snapshot of how customers engaged with a business on the day of a review, rather than having them complete a survey about a past experience.
Trolls
These individuals have an online behaviour pattern that makes it easy for Polly to identify when looking at the account history. If a user consistently leaves bad reviews for 1+ businesses, Polly will adjust for this type of user input
Size Bias
Larger companies are larger targets for bad reviews. Polly’s algorithm considers the age and size of the business to account for negative customer experiences.
Results
For 6 years, Polly has used her dynamic methodology to produce the most sophisticated, consumer-driven results available. Polly has increased the credibility of the awards organization, assuring that the winners are selected based on in-depth consumer evaluation and reinforcing purchase decisions for consumers in these markets.
