## Solved – Finding the mean and standard deviation of an unknown distribution from a sample

I am interested in finding the mean and standard deviation of the whole distribution by looking only at a random sample. I don't know anything else about the distribution (for example I don't know if the distribution is normal or not). Is what I'm asking even possible? Best Answer Sure, your best guess of the … Read more

## Solved – Student t-distribution parameter/s and MLE

So I always thought of the Student t-distribution as having only 1 parameter, v, the degrees of freedom (as described by wikipedia). When I searched however on how to find the MLE of v I keep coming across questions mentioning mu and sigma as parameters as well. So 1) Is the Student-t Distribution a special … Read more

## Solved – Quarter is to quartile as half is to…

Is there one word to describe the sections of data on either side of the median? I'd guess "half", but it seems like there ought be something better… Best Answer …as half to median. "Quarter" and "half" are names for parts of something, "quartile" and "median" are quantiles, so the points that divide the data … Read more

## Solved – Connection between power law and Zipf’s law

I am trying to better understand the connection between the power law distribution and Zipf's distribution (law). There is a neat explanation in [1]. The article suggests that as we can derivate the power law function from Pareto's law, combined with the relationship between Pareto's law and Zipf's law, the power law parameter alpha is … Read more

## Solved – Taylor series expansion of maximum likelihood estimator, Newton-Raphson, Fisher scoring and distribution of MLE by Delta method

Assume \$ellleft(thetaright)\$ is the log-likelihood of parameter vector \$theta\$ and \$widehat{theta}\$ is the maximum likelihood estimator of \$theta\$ then the Taylor series of \$ellleft(thetaright)\$ about \$widehat{theta}\$ is begin{align*} ellleft(thetaright) & approxeqellleft(widehat{theta}right)+frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}left(theta-widehat{theta}right)+frac{1}{2}left(theta-widehat{theta}right)^{prime}frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}left(theta-widehat{theta}right)\ frac{ellleft(thetaright)}{partialtheta} & approxeqmathbf{0}+left(mathbf{1}-mathbf{0}right)frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}+frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}left(theta-widehat{theta}right)quadoverset{textrm{set}}{=}quadmathbf{0}\ \ theta-widehat{theta} & =-left[frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}right]^{-}left[frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}right]\ widehat{theta}-theta & =left[frac{partial^{2}ellleft(thetaright)}{partialthetapartialtheta^{prime}}Bigr|_{theta=widehat{theta}}right]^{-}left[frac{partialellleft(thetaright)}{partialtheta}Bigr|_{theta=widehat{theta}}right]\ widehat{theta}-theta & =left[mathbb{H}left(thetaright)Bigr|_{theta=widehat{theta}}right]^{-}left[mathbb{S}left(thetaright)Bigr|_{theta=widehat{theta}}right] end{align*} As \$ theta=widehat{theta}-left[mathbb{H}left(thetaright)Bigr|_{theta=widehat{theta}}right]^{-}left[mathbb{S}left(thetaright)Bigr|_{theta=widehat{theta}}right] \$ So begin{align*} theta^{left(m+1right)} & =theta^{left(mright)}-left[mathbb{H}left(theta^{left(mright)}right)right]^{-}mathbb{S}left(theta^{left(mright)}right)quadquadleft({textrm{Newton-Raphson}}right)\ \ … Read more

## Solved – “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model models the probability of a number of random variables each with its own value of mean? If not, then what is the correct … Read more

## Solved – Statistical interpretation of Maximum Entropy Distribution

I have used the principle of maximum entropy to justify the use of several distributions in various settings; however, I have yet to be able to formulate a statistical, as opposed to information-theoretic, interpretation of maximum entropy. In other words, what does maximizing the entropy imply about the statistical properties of the distribution? Has anyone … Read more

## Solved – Statistical interpretation of Maximum Entropy Distribution

I have used the principle of maximum entropy to justify the use of several distributions in various settings; however, I have yet to be able to formulate a statistical, as opposed to information-theoretic, interpretation of maximum entropy. In other words, what does maximizing the entropy imply about the statistical properties of the distribution? Has anyone … Read more

## Solved – How to calculate the scale parameter of a Cauchy random variable

Let \$(X_n)\$ be iid random variables and suppose they have mean 0 and follow Cauchy distribution. I know I can set the location parameter to 0. My question is how to find the corresponding scale parameter. Thank you! Best Answer Note that the mean of the Cauchy distribution doesn't exist (so we can't assume it … Read more

## Solved – How to calculate the scale parameter of a Cauchy random variable

Let \$(X_n)\$ be iid random variables and suppose they have mean 0 and follow Cauchy distribution. I know I can set the location parameter to 0. My question is how to find the corresponding scale parameter. Thank you! Best Answer Note that the mean of the Cauchy distribution doesn't exist (so we can't assume it … Read more