I am trying to construct a rule-based classifier on a dataset with 332 instances and 14 features. I am just confused how can I validate the classification model? 10-fold cross validation or holdout method should be used?
Can I just apply the 10-fold cross validation for validation or the model has to be tested by a different set?
Holdout is essentially a 2-fold cross validation. If you perform k-fold cross validation correctly, no extra holdout set is necessary.
Make sure that your predictors are chosen based on the test sets (and not in advance on all the samples). You may also want to think about stratification if appropriate.
- Solved – the difference between k-holdout and k-fold cross validation
- Solved – k-fold cross validation overestimates test error
- Solved – How to perform 10-fold cross validation by manually constructing datasets
- Solved – How to use 10-fold Cross Validation in Feature Selection
- Solved – K fold cross validation; How many epochs to train for