I am interested in finding the mean and standard deviation of the whole distribution by looking only at a random sample. I don't know anything else about the distribution (for example I don't know if the distribution is normal or not). Is what I'm asking even possible? Best Answer Sure, your best guess of the … Read more
So I always thought of the Student t-distribution as having only 1 parameter, v, the degrees of freedom (as described by wikipedia). When I searched however on how to find the MLE of v I keep coming across questions mentioning mu and sigma as parameters as well. So 1) Is the Student-t Distribution a special … Read more
Is there one word to describe the sections of data on either side of the median? I'd guess "half", but it seems like there ought be something better… Best Answer …as half to median. "Quarter" and "half" are names for parts of something, "quartile" and "median" are quantiles, so the points that divide the data … Read more
I am trying to better understand the connection between the power law distribution and Zipf's distribution (law). There is a neat explanation in [1]. The article suggests that as we can derivate the power law function from Pareto's law, combined with the relationship between Pareto's law and Zipf's law, the power law parameter alpha is … Read more
Assume $ellleft(thetaright)$ is the log-likelihood of parameter vector $theta$ and $widehat{theta}$ is the maximum likelihood estimator of $theta$ then the Taylor series of $ellleft(thetaright)$ about $widehat{theta}$ is begin{align*} ellleft(thetaright) & approxeqellleft(widehat{theta}right)+frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}left(theta-widehat{theta}right)+frac{1}{2}left(theta-widehat{theta}right)^{prime}frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}left(theta-widehat{theta}right)\ frac{ellleft(thetaright)}{partialtheta} & approxeqmathbf{0}+left(mathbf{1}-mathbf{0}right)frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}+frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}left(theta-widehat{theta}right)quadoverset{textrm{set}}{=}quadmathbf{0}\ \ theta-widehat{theta} & =-left[frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}right]^{-}left[frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}right]\ widehat{theta}-theta & =left[frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}right]^{-}left[frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}right]\ widehat{theta}-theta & =left[mathbb{H}left(thetaright)Bigr|_{theta=widehat{theta}}right]^{-}left[mathbb{S}left(thetaright)Bigr|_{theta=widehat{theta}}right] end{align*} As $ theta=widehat{theta}-left[mathbb{H}left(thetaright)Bigr|_{theta=widehat{theta}}right]^{-}left[mathbb{S}left(thetaright)Bigr|_{theta=widehat{theta}}right] $ So begin{align*} theta^{left(m+1right)} & =theta^{left(mright)}-left[mathbb{H}left(theta^{left(mright)}right)right]^{-}mathbb{S}left(theta^{left(mright)}right)quadquadleft({textrm{Newton-Raphson}}right)\ \ … Read more
We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model models the probability of a number of random variables each with its own value of mean? If not, then what is the correct … Read more
I have used the principle of maximum entropy to justify the use of several distributions in various settings; however, I have yet to be able to formulate a statistical, as opposed to information-theoretic, interpretation of maximum entropy. In other words, what does maximizing the entropy imply about the statistical properties of the distribution? Has anyone … Read more
I have used the principle of maximum entropy to justify the use of several distributions in various settings; however, I have yet to be able to formulate a statistical, as opposed to information-theoretic, interpretation of maximum entropy. In other words, what does maximizing the entropy imply about the statistical properties of the distribution? Has anyone … Read more
Let $(X_n)$ be iid random variables and suppose they have mean 0 and follow Cauchy distribution. I know I can set the location parameter to 0. My question is how to find the corresponding scale parameter. Thank you! Best Answer Note that the mean of the Cauchy distribution doesn't exist (so we can't assume it … Read more
Let $(X_n)$ be iid random variables and suppose they have mean 0 and follow Cauchy distribution. I know I can set the location parameter to 0. My question is how to find the corresponding scale parameter. Thank you! Best Answer Note that the mean of the Cauchy distribution doesn't exist (so we can't assume it … Read more