When I read about how to setup your data, one thing I have often come across is that transforming some continuous data into categorical data is not a good idea, since you may very well make the wrong conclusion if the thresholds are poorly determined.
However, I currently have some data (PSA values for prostate cancer patients), where I think the common consensus is that if you are below 4 you probably don't have it, if you are above you are at risk, and then something like above 10 and 20, you probably have it. Something like that.
In that case, would it still be incorrect to categorize my continuous PSA values into groups of let's say 0-4, 4-10, and >10 ? Or is it actually okay since the thresholds are "well determined" so to speak.
Is there a sharp discontinuity at your thresholds?
For instance, suppose you have two patients A and B with values 3.9 and 4.1, and another two patients C and D with values 6.7 and 6.9. Is the difference in the likelihood for cancer between A and B much larger than the corresponding difference between C and D?
If yes, then discretizing makes sense.
If not, then your thresholds may make sense in understanding your data, but they are not "well determined" in a statistically meaningful sense. Don't discretize. Instead, use your test scores "as-is", and if you suspect some kind of nonlinearity, use splines.
- Solved – How does number of thresholds get chosen in roc_curve function in scikit-learn
- Solved – Bayes Network computing conditional probabilities
- Solved – One way Anova Test validation
- Solved – Why do we use the chi-squared test to test independence of 2 categorical variables
- Solved – When would one pre-specify thresholds in SEM/CFA for limited dependent variables