# Solved – Differences in differences, fixed effects and standard errors

I have a question on estimating a difference in differences model using Stata. As I understand this, also from other questions, when there are no covariates, estimating the diff in diff using a regular regression (including dummy for year of treatment, dummy for treatment, and interaction) gives the same results as estimating it using a fixed effect command such as Stata's xtreg. It actually is so when I do this with my data, but the standard errors are completely different: when is use Stata's command "reg" i get absolutely no significance, when I use xtreg I get instead a t-statistic of more than 2, with standard errors being almost 4 times smaller. Why is it so? And what does it suggest about the validity of the model and the command to use? What would be best to do when I am also adding covariates later?

Edit: I try to add an example from the code:

``gen y07=1 if year==2017  replace jump=0 if jump!=1 gen did=y07*treat  xtset id year xtreg y y07 did, fe r    Fixed-effects (within) regression               Number of obs     =      4,568 Group variable: id                              Number of groups  =      2,284  R-sq:                                           Obs per group:      within  = 0.0131                                         min =          2      between = 0.0008                                         avg =        2.0      overall = 0.0011                                         max =          2                                                  F(2,2283)         =      12.73 corr(u_i, Xb)  = 0.0069                         Prob > F          =     0.0000                                   (Std. Err. adjusted for 2,284 clusters in id) ------------------------------------------------------------------------------              |               Robust            y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval] -------------+----------------------------------------------------------------          y07 |   .5117687   .1409194     3.63   0.000     .2354253    .7881121          did |   .8282564   .4076776     2.03   0.042     .0287991    1.627714        _cons |   8.272329   .0809889   102.14   0.000      8.11351    8.431149 -------------+----------------------------------------------------------------      sigma_u |  18.188562      sigma_e |  5.4737922          rho |  .91695247   (fraction of variance due to u_i)     reg y treat y07 did, r   Linear regression                               Number of obs     =      4,568                                                 F(3, 4564)        =       1.80                                                 Prob > F          =     0.1441                                                 R-squared         =     0.0013                                                 Root MSE          =     18.597  ------------------------------------------------------------------------------              |               Robust            y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval] -------------+----------------------------------------------------------------        treat |   .6513042   .7340444     0.89   0.375    -.7877781    2.090386          y07 |   .5117687   .6775766     0.76   0.450    -.8166093    1.840147          did |   .8282564   1.161073     0.71   0.476    -1.448009    3.104522        _cons |   8.045057   .4404064    18.27   0.000     7.181647    8.908467 ------------------------------------------------------------------------------   ` ` `   Of course I was imprecise in saying the standard error was four times smaller, it's slightly less than tree, but it's the same thing. Of course year the variable "treat" denotes being assigned to the treatment group.  ``
Contents

You need to compare apples to apples, so use clustering with OLS and clustering with `xtreg, fe` (or robust with `xtreg, fe`, which will default to clustering as Thomas pointed out). These coefficient equivalences are limited to two-period (one pre, and one post) datasets with treatment at the same time for all treated units.

Here's an example of 2×2 DID on a public dataset demonstrating this. Here NJ restaurants are treated (become subject to the minimum wage increase) and PA restaurant are not. February '92 (t=0) is pre and November '92 is post (t=1). The DID parameter is the interaction of t = 1 and NJ = 1. The outcome fte is full-time equivalent employees. Here I will balance the panel in order to get `xtreg, fe` and OLS to give the same coefficient estimates. If the panel is unbalanced (consists of repeated cross-sections), `xtreg, fe` will drop some observations that appear in only one year and the estimates will no longer match OLS or manual calculations. You may want to stick with clustered OLS if you have a repeated cross-section.

Here is the result. Note that you can use factor variable notation to create the interactions rather than hard coding them.

``. use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear (Dataset from Card&Krueger (1994))  . drop if id == 407 // duplicate restaurant (4 observations deleted)  . drop if missing(fte, treated, t, id) (19 observations deleted)  . bysort id: keep if _N==2 // balance the panel (19 observations deleted)  . xtset id t        panel variable:  id (strongly balanced)         time variable:  t, 0 to 1                 delta:  1 unit  .  . /* calculate DID by hand */ . table treated t, c(mean fte N fte) row col  ---------------------------------------- New       | Jersey =  | 1;        | Pennsylva | Feb. 1992 = 0; Nov. 1992 = 1 nia = 0   |        0         1     Total ----------+-----------------------------        PA | 20.17333     17.65  18.91167           |       75        75       150           |         NJ | 17.06927  17.51831  17.29379           |      314       314       628           |      Total | 17.66774   17.5437  17.60572           |      389       389       778 ----------------------------------------  . di %9.3f (17.51831 - 17.06927) - (17.65 - 20.17333)     2.972  .  . /* fit models */ . eststo ols_robust:   qui   reg fte i.treated##i.t, robust  . eststo xtreg_robust: qui xtreg fte i.treated##i.t, fe robust  . eststo xtreg_clust:  qui xtreg fte i.treated##i.t, fe cluster(id)  . eststo ols_clust:    qui   reg fte i.treated##i.t, cluster(id)  .  . capture ssc install estout  . esttab *, se(%9.7f) noomitted drop(0.treated 0.t 0.treated#0.t) modelwidt(15) mtitles label varwidth(35)  ---------------------------------------------------------------------------------------------------------------                                                 (1)                (2)                (3)                (4)                                             ols_robust       xtreg_robust        xtreg_clust          ols_clust    --------------------------------------------------------------------------------------------------------------- NJ                                           -3.104*                                                  -3.104*                                           (1.4475664)                                              (1.4484988)     Feb. 1992 = 0; Nov. 1992 = 1=1               -2.523             -2.523*            -2.523*            -2.523*                                           (1.6371048)        (1.2498119)        (1.2498119)        (1.2506190)     NJ # Feb. 1992 = 0; Nov. 1992 = 1=1           2.972              2.972*             2.972*             2.972*                                           (1.7822146)        (1.3337493)        (1.3337493)        (1.3346107)     Constant                                      20.17***           17.67***           17.67***           20.17***                                         (1.3591695)        (0.2232501)        (0.2232501)        (1.3600450)    --------------------------------------------------------------------------------------------------------------- Observations                                    778                778                778                778    --------------------------------------------------------------------------------------------------------------- Standard errors in parentheses * p<0.05, ** p<0.01, *** p<0.001 ``

Clustering in DID settings is a good idea for reasons outlined in Bertrand, Duflo, and Mullainathan's 2004 QJE paper. Clustering at the level of treatment is also a good idea, but here that is not feasible since there are not enough clusters (since treatment is a state law and we have data from two states only) for that to work well. Generally your SEs will go up when you cluster in DID, but if the errors are negatively correlated within cluster, they might shrink. See this post for the reasons why.

Code:

``estimates clear cls use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear drop if id == 407 // duplicate restaurant drop if missing(fte, treated, t, id) bysort id: keep if _N==2 // balance the panel xtset id t  /* calculate DID by hand */ table treated t, c(mean fte N fte) row col di %9.3f (17.51831 - 17.06927) - (17.65 - 20.17333)  /* fit models */ eststo ols_robust:   qui   reg fte i.treated##i.t, robust eststo xtreg_robust: qui xtreg fte i.treated##i.t, fe robust eststo xtreg_clust:  qui xtreg fte i.treated##i.t, fe cluster(id) eststo ols_clust:    qui   reg fte i.treated##i.t, cluster(id)  capture ssc install estout esttab *, se(%9.7f) noomitted drop(0.treated 0.t 0.treated#0.t) modelwidt(15) mtitles label varwidth(35) ``

Rate this post