Solved – Neutral Networks with Categorical Data

I'm building a neural network in R with the neuralnet package, and the data contains categorical values.

I have converted the columns into dummy variables, but I have a couple of questions:

  • I need to normalize the dummy variables, correct? ( ($X_i$ – column mean)/column standard deviation )
  • Do I need to use the softmax activation function? Only on the output activation function?
  • If so, how do I set this in neuralnet?
  1. You do not need to normalize dummy variables. In my experience NN implementation algorithms work better when weights are in the [-5,5] range. With 0,1 that is exactly the case. In fact many people reccommend scaling to (0,1) range.
  2. You don't need softmax if your predictors are categorical. On the other hand if your predicting variable is multi-class categorical you need softmax because the standard algorithm works with only two categories on the output variable.
  3. You should consider using nnet instead of neuralnet. Altough nnet has less choices, it is probably what you need if you want a simple model. nnet allows for softmax and could be more robust using decay.

Similar Posts:

Rate this post

Leave a Comment