Re: Sorting scores
- From: torbenm@xxxxxxxxxxxxx (Torben Ægidius Mogensen)
- Date: Tue, 04 Sep 2007 10:07:27 +0200
Bruno Barberi Gnecco <brunobgDELETETHIS@xxxxxxxxxxxxxxxxxxxxx> writes:
I've been looking for ways to sort scores. It's not a rare
problem: in a scale of 0 to 10, if one uses simple average an item
that has only 1 score of 10 is ranked below an item with 5 scores
of 10 and 1 score of 5. I suppose it should be also meaningful for
a score weight to decay with time.
I'm sure this problem must have been studied before, but I
can't find any references. Thanks a lot for any help.
There is no unique best way to do this. One way is to assign a
confidence to every rating, which starts out at, say, 10 and decays
over time. You can also rate your reviewers so those you trust the
most start at a higher confidence and decay more slowly.
So the question is: If you have a set of ratings/confidence pairs, how
do you combine them into a single final score used for ranking?
One possible solution is:
1. Let C be the sum of all confidences.
2. Let R be the sum of rating*confidence for all pairs.
3. Let score = R/C.
This way, if all agree on the same rating, the score will be equal to
this rating, regardless of their confidence. It is still not quite
what you want, as 5 ratings of 10 plus one of 5 scores lower than one
rating of 10 (regardless of their confidences). So you would want to
give a bonus to having more ratings and higher confidences.
One way to achieve this is to add a single artificial review of
average rating and high confidence to the mix. This way, it will
require more reviews to pull the score away from the average.
Let us, for example, assume that 5 is an average rating and
confidences go from 1 to 10.
Example 1 has a single review of rating 10 and confidence 10.
Example 2 has five reviews of rating 10 with confidences 10, 9, 8, 7
and 6 plus one rating of 5 with confidence 10.
When we add the artificial review (5/10) to the first example, we get
C = 10+10 = 20
R = 10*10 + 5*10 = 150
score = 150/20 = 7.5
When we do the same to the second, we get
C = 10+9+8+7+6+10 = 50
R = 10*(10+9+8+7+6) + 5*10 = 450
score = 450/50 = 9
I hope this is at least a little bit helpful.
Torben
.
- References:
- Sorting scores
- From: Bruno Barberi Gnecco
- Sorting scores
- Prev by Date: Sorting scores
- Next by Date: ANN: a new wiki site to organize Calls For Papers (CFP)
- Previous by thread: Sorting scores
- Next by thread: ANN: a new wiki site to organize Calls For Papers (CFP)
- Index(es):
Relevant Pages
|
