I am using LibLinear for a document classification task, in which I would like to calculate the probability of correctness for each prediction. In fact, in the LibLinear, it does provide probability output for logistic regression, but not for default support vector classification task. Furthermore, based on 10-fold cross validation, the logistic regression is nearly 10% worse than support vector classification.
So can anyone tell me, if I continue to use the support vector classification for the solution, is there a method to calculate the probability separately from the program?
Best Answer
You can use a sigmoid function $f(d) = frac{1}{1 + e^{-alpha(d-beta)}}$ to convert your SVM decision value $d = (w, x) + b$ into a number between 0 and 1 which can be treated as probability. You can adjust parameters $alpha$ and $beta$ depending on your data.
For more elaborate approaches, see these papers:
- B.Zadrozny, C. Elkan, Transforming classifier scores into accurate multiclass probability estimates.
- J.Drish, Obtaining calibrated probability estimates from Support Vector Machines.