Question
Is there such concept in econometrics/statistics as a derivative of parameter $hat{b_{p}}$ in a linear model with respect to some observation $X_{ij}$?
By derivative I mean $frac{partial hat{b_{p}}}{partial X_{ij}}$ – how would change parameter $hat{b_{p}}$ if we changed $X_{ij}$?
Motivation
I was thinking about a situation when we have some uncertainty in data (e.g. results from a survey) and we have enough money to obtain precise results only in a single observation, which observation should we choose?
My intuition is saying that we should choose the observation that might change parameters the most, which is equivalent to the highest value of the derivative. If there are any other concepts feel free to write about them.
Best Answer
@onestop points in the right direction. Belsley, Kuh, and Welsch describe this approach on pp. 24-26 of their book. To differentiate with respect to an observation (and not just one of its attributes), they introduce a weight, perform weighted least squares, and differentiate with respect to the weight.
Specifically, let $mathbb{X} = X_{ij}$ be the design matrix, let $mathbf{x}_i$ be the $i$th observation, let $e_i$ be its residual, let $w_i$ be the weight, and define $h_i$ (the $i$th diagonal entry in the hat matrix) to be $mathbf{x}_i (mathbb{X}^T mathbb{X})^{-1} mathbf{x}_i^T$. They compute
$$frac{partial b(w_i)}{partial w_i} = frac{(mathbb{X}^Tmathbb{X})^{-1} mathbf{x}_i^T e_i}{left[1 – (1 – w_i)h_iright]^2},$$
whence
$$frac{partial b(w_i)}{partial w_i}Bigg|_{w_i=1} = (mathbb{X}^Tmathbb{X})^{-1} mathbf{x}_i^T e_i.$$
This is interpreted as a way to "identify influential observations, … provid[ing] a means for examining the sensitivity of the regression coefficients to a slight change in the weight given to the ith observation. Large values of this derivative indicate observations that have large influence on the calculated coefficients." They suggest it can be used as an alternative to the DFBETA diagnostic. (DFBETA measures the change in $b$ when observation $i$ is completely deleted.) The relationship between the influence and DFBETA is that DFBETA equals the influence divided by $1 – h_i$ [equation 2.1 p. 13].