Are Canonical link functions and the link functions the same thing?
If not, can anyone tell the difference between them?
I know a link function is a function that links a linear predictor to the response variable.
Then isn't that a canonical link function too? I'm so confused, could anyone help?
Also if we're taking the distribution of the response variable, say $Y$ as a $lognormal$ distribution, i.e $Y sim LogN(mu,{sigma}^2)$, then what should be the natural link function to be applied?
What I did was, first transform the distribution into $Normal$ distribution and hence use the link function $g(mu) =mu$, but in the solution it's given as $log(mu)$, how is that?
Can anyone help?
Best Answer
The canonical link is a type of link function.
Per Wikipedia: The link function provides the relationship between the linear predictor and the mean of the distribution function…. It makes sense to try to match the domain of the link function to the range of the distribution function's mean.
So there isn't one correct link function for a given distribution – it will depend on the problem.
By linear predictor of the mean of the response, the above quote means $E(Y) = {bf X' beta}$, where ${bf X} = (x_1, …, x_n)$ are the independent variables and ${bf beta} = (beta_1, …, beta_p)$ are the coefficients. Note that the link function is $eta = {bf X' beta}$. This is always true.
For example, if our response, $Y in [0,1]$, then somehow we need to ensure that our linear predictor ${bf X' beta} in [0,1]$. But since $X$, $beta in mathbb{R}$, this means ${bf X' beta} in mathbb{R}$ unless we put some restrictions or apply a transformation. That's a problem because then $E(Y)$ is not guaranteed to be bound between 0 and 1.
To fix this, we can apply a transformation that takes any real number and returns a value between 0 and 1. A common transformation is the logit (or sigmoid or logistic) function:
$$ logit(x) = frac{e^{x}}{1+e^{x}} $$
It also helps that this function monotonically increases – for inputs going from large negative to large positive numbers, the output goes from 0 to 1.
Also note that its symmetric about 0. This could be a desirable property or not. (For example, if your function decays more slowly to zero than it increases to 1, then consider using the complementary log-log link)
We call this a logit link function.
Sometimes you'll see it written in terms of the probability, $pi_i$ of getting a 1, i.e. $P(Y_i = 1|X) = pi_i$. In this case, we write $eta_i = logit(pi_i) = log(frac{pi_i}{1-pi_i}) = {bf X'beta}$. This is the "log odds of $P(Y_i = 1|X)$".
In your question, the canonical link is $eta = mu$. In other words, our response can take any real value. So $E(Y) in mathbb{R}$. That means we don't need to restrict ${bf X'beta}$. So the link function basically has no impact. In other words, $eta_i = mu_i = {bf X'beta}$.
Similar Posts:
- Solved – the difference between a “link function” and a “canonical link function” for GLM
- Solved – What useful properties does the canonical link function have
- Solved – Is the link function in Probit model canonical
- Solved – Canonical link of Gamma Distribution
- Solved – Does a canonical link function always exist for a Generalized Linear Model (GLM)