# Solved – Multiclass Summary Metrics in R’s Caret when predicting Probabilities

It's not clear to me, how the different summary metrics for caret's `train` function are defined, when I predict probabilities of a multiclass problem.

Without a loss of generality, assume that I use a random forest to produce probabilities:

``library(caret) data(iris)  control = trainControl(method="CV", number=5,verboseIter = TRUE,classProbs=TRUE) # iv) tuning parameter  grid = expand.grid(mtry = 1:3) rf_gridsearch = train(y=iris[,5],x=iris[-5],method="ranger", num.trees=2000, tuneGrid=grid, trControl=control) rf_gridsearch  # Output: ....   mtry  Accuracy   Kappa   1     0.9600000  0.94    2     0.9666667  0.95    3     0.9666667  0.95  Accuracy was used to select the optimal model using  the largest value. The final value used for the model was mtry = 2. ``

For the binary case I know how to compute the AUC: https://en.wikipedia.org/wiki/Receiver_operating_characteristic

So my questions are:

1. How is the `Accuracy` computed for mutliclass prediction? (How is it condensed to a single value)

2. Why is mtry=2 chosen, even if mtry=3 is equivalent according to the Accuracy?

3. Why isn't it possible to compute the RMSE for a classification problem? (Throwing the error "Error: Metric RMSE not applicable for classification models" if metric="RMSE" is used. Furthermore, for multiclass prediction the RMSE is still defined – other then the AUC.)
Since, the Brier Score is simply the MSE (for 2 classes), why not allowing to use the RMSE for probability predictions?