Aug 22, 2012

Rethinking the Rating System

I read this article on why rating systems don't work and it got me into thinking what rating system would work. The current rating systems are based on average rating and thus can be easily gamed. Every business can ask their well wishers to rate high on their listing and offset the few genuine raters.
One alternative is to consider rating during a fixed time period and average based on period rating.

For eg. I would average the rating for every month of the year, and my resultant rating would be the average of all the period ratings.

Say,

January: Ten 5 star ratings. - Average of 5.

February: One 3 star rating. - Average of 3.

Resultant rating in March - 4 (3+5)/2

Using the averages it would be - 5 (10*5 + 3)/11.

Another option is to consider density of ratings with reference to time to create a period which varies on density. Thus a restaurant could be a great place to eat for a year, but if their quality goes down for a month, the resultant rating would easily offset the ratings of the previous year. The math of this is simple to work out and can be fine tuned iteratively.