Solved – Garson’s algorithm for fully connected LSTMs

Garson proposed an algorithm, later modified by Goh (1995) for determining the relative importance of an input node to a network. In the case of a single layer of hidden units, the equation is

$$ Q_{ik} = frac{ sum_{j=1}^L | w_{ij} v_{jk} | / sum_{r=1}^N | w_{rj}|}{sum_{i=1}^N sum_{j=1}^Lbig(|w_{ij}v_{jk}| / sum_{r=1}^N|w_{rj}|big)} $$

where $w_{ij}$ is the weight between the $i$th input and the $j$th hidden unit, and $v_{jk}$ is the weight between the $j$th hidden unit and the $k$th output.

I am interested in the case where the neural network is fully connected and has a single output. In this case, the only difference between the $Q_i$s for each input $i$ is the $sum_{j=1}^L |w_{ij}|$, and so if we only care about the relative importance between the inputs, we can define
$$ Q_{ik} = sum_{j=1}^L |w_{ij}|.$$
That is, the only thing that matters are the inputs weights leaving that hidden unit, even if this is generalized to a multi-hidden layer neural network.

I was wondering if the same would hold if the hidden layer was replaced by a layer of LSTM units? My rationale is that since LSTMs are fully connected, we would still be able to say that
$$ Q_{ik} = sum_{j=1}^L |w_{ij}|.$$

Similar Posts:

• Solved – Neural Net Matrix Multiplication
• Solved – word2vec neural network – bias units
• Solved – Weight of neural network and feature importance
• Solved – Does a Neural Network actually need an activation function or is that just for Back Propagation
• Solved – Feedforward Neural networks for Regression confusion