Solved – the False Discovery Rate curve

by

I have heard of the "false discovery rate curve" (e.g here) but never seen an example.

If I recall correctly from a conversation with a colleague, the y-axis in the FDR curve measures the FDR itself, defined as $FDR = frac{FP}{TP+FP}$ (i.e. $1 – text{Precision}$), but what goes into the x-axis?

What is a normal shape for this curve, and when is it a good idea to use it?

Contents hide
1 Best Answer
2 Similar Posts:

Best Answer

The standard Receiver Operating Characteristic (ROC) chart displays True Positive Rate (also called Sensitivity or Recall) TPR = TP/(TP+FN) on the vertical axis vs. False Positive Rate, FPR = FP/(FP+TN) on the horizontal axis. The 45 degree diagonal line from (0,0) to (1,1) corresponds to "random", and iso-quality lines parallel to this represent equal values of "distance from random" or Informedness = TPR – FPR.

If we consider the problem from the opposite perspective, then each of these quantities has its converse, as follows:

  • TPR <–> TNR or Specificity, where TNR = TN/(FP+TN),
  • FPR <–> FNR = FN/(FN+TP)
  • Informedness = TPR – FPR <–> TNR – FNR = Informedness = distance from random (same).

Therefore, in principle you could equally well plot TNR vs. FNR instead of the conventional ROC plot of TPR vs. FPR and Informedness = distance-from-random stays the same.

When using a conventional ROC plot of TPR vs. FPR with Informedness = TPR-FPR as an unbiased metric, the other unbiased metrics which are useful but which are not displayed on the plot are :

  • Markedness = TP/(TP+FP) – FN/(TN+FN) = Precision – (1 – Negative Predictive Value)
  • Matthews Correlation Coefficient = SQRT[Informedness * Markedness].

If we again consider the problem from the opposite perspective, then each of the quantities also has its converse, as follows:

  • Precision = Positive Pred.Value = TP/(TP+FP) <–> TN/(TN+FN) = Negative Pred.Value

and therefore:

  • FDR = 1 – Precision = 1 – Positive Pred.Value <–> 1 – Negative Pred.Value = FN/(TN+FN)

I don't now what (if any) particular name is given to FN/(TN+FN), other than 1-NPV, but this would be the converse of FDR.

Generally the other characteristic plot that is used in addition to the ROC plot of TPR vs. FPR (on linear scales) is the Detection Error Tradeoff or DET plot. This plot displays missed detection rate or FNR on the vertical axis vs. FPR on the horizontal axis, both on non-linear scales. The non-linear scaling of the DET plot is based on the inverse of the cumulative normal distribution. Perhaps this might be the plot that your colleague was referring to.

I have not personally used DET plots myself, but I believe that the reason for using them is that the displayed output results appear linear, whereas those of the ROC plot do not.

Similar Posts:

Rate this post

Leave a Comment