Suppose that you have a logistic model and the predicted probabilities of a $1$ all are in $[0.2,0.3]$. So the predictions would be "0" based on a threshold of $0.5$. What would be a good threshold to choose?
If your logistic model has predicted probabilities that are always in $[0.2, 0.3]$ for class $1$ and you have sufficient inclusion of class $2$ data you have possibly trained it with appropriate data or used appropriate features. The logistic regression model is probabilistic; ie, it spits back probabilities. If you decide that the model saying class $1$ has probability of $0.25$ and class $2$ has probability $0.75$ I feel as though you've eliminated a lot of the strong points and highlights of a probabilistic model as now your inference on features is wonk and difficult to properly interpret (using standard tests, which is really the best argument in favor of probabilistic models over other strategies and their bread-and-butter use-case). You can hack together other models that will probably fit better if you don't want a probabilistic strategy (ie, RF, NNs, etc).
- Solved – How to calculate probability percentage for logistic regression with threshold
- Solved – Why the predicted probalility in logistic regression is small, spreading within (0.0, 0.6), rather than(0.0, 1.0)
- Solved – Finding an optimal class probability threshold for SVM
- Solved – Is this a good strategy to set a threshold on softmax probabilities in a multi-class classification task?
- Solved – How to have a “None of the above” category in a Logistic Regression