精度调用曲线和平均精度之间的混淆 [英] Confusion about precision-recall curve and average precision

问题描述

为了评估我的图像检索系统，我读了很多有关Precision-Recall曲线的文章.特别是，我正在阅读这篇有关功能的文章VLFeat和维基百科页面中的提取器.

I'm reading a lot about Precision-Recall curves in order to evaluate my image retrieval system. In particular I'm reading this article about feature extractors in VLFeat and the wikipedia page about precision-recall.

我知道该曲线对于评估我们的系统性能非常有用.检索到的元素数.因此，我们反复计算精确调用率，以获取top元素，然后获取top 2，top 3等……但是我的问题是:何时停止?

I understand that this curve is useful to evaluate our system performance w.r.t. the number of elements retrieved. So we repeatedly compute precision-recall retrieving the top element, then top 2, top 3 and so on...but my question is: when do we stop?

我的直觉是:当检索到的元素列表的召回率等于1时，我们便停止了操作，因此我们检索了所有相关元素(即没有假阴性，只有真阳性).

My intuition is: we stop when our list of retrieved elements has recall equal to 1, so we retrieve all the relevant elements (i.e. there are no false negatives, only true positives).

相同的问题是平均精度:检索结果中应包含多少个元素以进行计算?如果我以前的直觉是正确的，那么我们只需要找出最小的列表是什么.召回率为1，并将其用于计算AP.

Same question is for average precision: how many elements should be present in the retrieved result for computing it? If my previous intuition is correct, then we just need to find out what is the smallest list s.t. recall is 1 and use it for compute it AP.

我想知道为什么所有用于计算p-r曲线的库都没有显示出它是如何实现的?

I wonder why all the libraries for computing p-r curve don't show how this is implemented?

推荐答案

具有召回1的信息检索系统意味着一个完美的系统，在实践中似乎是不可能的！当您需要比较两个或多个信息检索系统时，Precision-Recall曲线非常有用.它不是要在召回率或精度达到一定值时停止. Precision-Recall曲线显示每个点上的成对召回率和精度值(请考虑前3或5个文档).您可以将曲线绘制到任何合理的点.

An information retrieval system with recall 1 means a perfect system which doesn't seem possible in practice! Precision-Recall curves are good when you need to compare two or more information retrieval systems. Its not about stopping when recall or precision reaches some value. Precision-Recall curve shows pairs of recall and precision values at each point (consider top 3 or 5 documents). You can draw the curve upto any reasonable point.

接近完美Precision-Recall曲线的曲线的性能水平要高于接近基线的曲线.换句话说，另一条曲线上方的一条曲线具有更好的性能水平.两条Precision-Recall曲线代表两个IR系统的性能水平:A和B.根据下图，系统A明显优于系统B.

Curves close to the perfect Precision-Recall curve have a better performance level than the ones closes to the baseline. In other words, a curve above the other curve has a better performance level. Two Precision-Recall curves represent the performance levels of two IR systems: A and B. System A clearly outperforms system B according to the following figure.

记住:Precision-Recall曲线不仅用于评估IR系统.它可以用来显示您的分类器有多好！例如，您可以计算精度，调用二进制分类任务并绘制Precision-Recall曲线，从而可以很好地估计分类器的性能.

Remember: Precision-Recall curve not only used for evaluating IR systems. It can be used to show how much good your classifier is! For example, you can compute precision, recall for a binary classification task and plot the Precision-Recall curve that can give you a good estimate of the performance of your classifier.

例如:

我鼓励您查看此教程来自Coursera.我相信您对Precision-Recall曲线的想法会更加清楚.

I would encourage you to see this tutorial from Coursera. I believe your idea will become more clear about Precision-Recall curve.

这篇关于精度调用曲线和平均精度之间的混淆的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！