如何用众包的排序排名一万张图片 [英] How to rank a million images with a crowdsourced sort

问题描述

我想通过做游戏排名风景图片的集合，由此网站访问者可以评价他们，以便找出图像的人找到最有吸引力的。

I'd like to rank a collection of landscape images by making a game whereby site visitors can rate them, in order to find out which images people find the most appealing.

什么是这样做的好方法？

What would be a good method of doing that?

• 热或-没有风格？即显示一个单一的形象，要求将其从1-10排名用户。在我看来，这可以让我的平均成绩，而我只需要保证我得到的所有图像的均匀分布票。相当容易实现。
• 选择A-或-B ？即显示两个图像，询问用户挑选最好的一个。这是有吸引力的，因为没有具体的排名，它只是一个比较。不过，我将如何实现它？我首先想到的是做一个快速排序，与由人提供的比较操作，完成后，只需重复排序广告无穷。

• Hot-or-Not style? I.e. show a single image, ask the user to rank it from 1-10. As I see it, this allows me to average the scores, and I would just need to ensure that I get an even distribution of votes across all the images. Fairly simple to implement.
• Pick A-or-B? I.e. show two images, ask user to pick the better one. This is appealing as there is no numerical ranking, it's just a comparison. But how would I implement it? My first thought was to do it as a quicksort, with the comparison operations being provided by humans, and once completed, simply repeat the sort ad-infinitum.

如何的您的办呢？

如果你需要的数字，我说的是大约一万张图片，在网站上20,000日均访问量。我想像一小部分可能会玩游戏，为了讨论的，可以说，我可以生成一个每天2000人的排序操作！这是一个非营利性的网站，和临终好奇会觉得通过我的个人资料：）的

推荐答案

正如其他人所说，排名1-10不工作那么好，因为人们有不同的层次。

As others have said, ranking 1-10 does not work that well because people have different levels.

与问题选择A-或-B 方式是，它不保证该系统能传递（A可以击打乙，而B拍C，和C比一个）。 具有不可传递的比较运算符的排序算法。随着快速排序，对这个例子中，没有选择的字母为支点将被错误地排对立起来。

The problem with the Pick A-or-B method is that its not guaranteed for the system to be transitive (A can beat B, but B beats C, and C beats A). Having nontransitive comparison operators breaks sorting algorithms. With quicksort, against this example, the letters not chosen as the pivot will be incorrectly ranked against each other.

在任何时候，你希望所有的图片绝对排名（即使部分/全部都是并列）。你也希望你的排名不会改变的除非有人票

At any given time, you want an absolute ranking of all the pictures (even if some/all of them are tied). You also want your ranking not to change unless someone votes.

我会使用选择A-或-B（或并列）方式，但确定排名类似的的Elo评级系统它用于排名的2人游戏（原国际象棋）：

I would use the Pick A-or-B (or tie) method, but determine ranking similar to the Elo ratings system which is used for rankings in 2 player games (originally chess):

本的Elo球员评分   系统比较球员的比赛记录   针对对手的比赛记录   并且确定的概率   球员赢得了对决。本   概率的因素决定了   点了玩家的等级上升或   向下基于每个的结果   匹配。当玩家击败的   对手具有较高的评价，所述   玩家的等级上升超过当   他或她击败的球员的   低等级​​（因为玩家们   击败对手谁拥有较低   评级）。

The Elo player-rating system compares players’ match records against their opponents’ match records and determines the probability of the player winning the matchup. This probability factor determines how many points a players’ rating goes up or down based on the results of each match. When a player defeats an opponent with a higher rating, the player’s rating goes up more than if he or she defeated a player with a lower rating (since players should defeat opponents who have lower ratings).

的的Elo系统：

1. 在所有新的球员开始时的基础等级的 1600
2. WinProbability = 1 /（10 ^（（对手的额定电流播放器的额定电流）/ 400）+ 1）
3. ScoringPt = 1点，如果他们赢了比赛，0，如果他们输了，和0.5个平手。
4. 玩家的新评价=玩家的旧等级+（K值*（ScoringPt播放器的获胜概率））

1. All new players start out with a base rating of 1600
2. WinProbability = 1/(10^(( Opponent’s Current Rating–Player’s Current Rating)/400) + 1)
3. ScoringPt = 1 point if they win the match, 0 if they lose, and 0.5 for a draw.
4. Player’s New Rating = Player’s Old Rating + (K-Value * (ScoringPt–Player’s Win Probability))

替换图片球员，你必须调整图文并茂评级基于公式的一个简单方法。然后就可以使用这些数字的分数排名。 （K值这里是比赛的级别，它是8-16本地的小型比赛和24-32的较大invitationals /地区性，你可以只使用一个常量像20）。

Replace "players" with pictures and you have a simple way of adjusting both pictures' rating based on a formula. You can then perform a ranking using those numeric scores. (K-Value here is the "Level" of the tournament. It's 8-16 for small local tournaments and 24-32 for larger invitationals/regionals. You can just use a constant like 20).

通过这种方法，你只需要保持一个号码对每个图像比保持每幅画面的各个队伍彼此画面少了很多占用大量内存。

With this method, you only need to keep one number for each picture which is a lot less memory intensive than keeping the individual ranks of each picture to each other picture.

编辑：添加基于注释的多肉少

Added a little more meat based on comments.

这篇关于如何用众包的排序排名一万张图片的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！