I've analyzed effects of temperature and sample thickness on drying process via response surface methodology. I've faced to four high leverage points (my four axial runs) that stick together on normal probability plot. these points aren't the outlier (on residual versus run plot).
Just because a high leverage point isn't an outlier doesn't mean all is well.
A single sufficiently influential point can pull the line essentially right through it (so its residual is 0).
A pair of influential points can easily make each other's externally studentized residuals zero / nearly zero.
So you can have a few points that behave as if they're from a completely different relationship from that you see in (say) 99% of the data, but they're sufficiently influential, they will have small residuals and even their studentized deleted residuals (/externally studentized residuals) can be small.
This can happen with Cooks Distance (and a variety of other measures of influence) — the effect of another outlier can make Cook's distance look fine for both.
If you remove all the influential points – as Michael suggests in comments – and the line is little changed, then there should be little to worry about.
- Solved – Interpreting case influence statistics (leverage, studentized residuals, and Cook’s distance)
- Solved – What effect(s) would removing the outliers have in a linear regression
- Solved – Cook’s distance and $R^2$
- Solved – Regression analysis vs outlier detection
- Solved – Outlier removal before Boosted Decision Tree Regression