I think it's useful to review what we know about cross-validation. Statistical results around CV fall into two classes: efficiency and consistency.

Efficiency is what we're usually concerned with when building predictive models. The idea is that we use CV to determine a model with asymtptotic guarantees concerning the loss function. The most famous result here is due to Stone 1977 and shows that LOO CV is asymptotically equivalent to AIC. But, Brett provides a good example where you can find a predictive model which doesn't inform you on the causal mechanism.

Consistency is what we're concerned with if our goal is to find the "true" model. The idea is that we use CV to determine a model with asymptotic guarantees that, given that our model space includes the true model, we'll discover it with a large enough sample. The most famous result here is due to Shao 1993 concerning linear models, but as he states in his abstract, his "shocking discovery" is opposite of the result for LOO. For linear models, you can achieve consistency using LKO CV as long as $k/n rightarrow 1$ as $n rightarrow infty$. Beyond linear mdoels, it's harder to derive statistical results.

But suppose you can meet the consistency criteria and your CV procedure leads to the true model: $Y = beta X + e$. What have we learned about the causal mechanism? We simply know that there's a well defined correlation between $Y$ and $X$, which doesn't say much about causal claims. From a traditional perspective, you need to bring in experimental design with the mechanism of control/manipulation to make causal claims. From the perspective of Judea Pearl's framework, you can bake causal assumptions into a structural model and use the probability based calculus of counterfactuals to derive some claims, but you'll need to satisfy certain properties.

Perhaps you could say that CV can help with causal inference by identifying the true model (provided you can satisfy consistency criteria!). But it only gets you so far; CV by itself isn't doing any of the work in either framework of causal inference.

If you're interested further in what we can say with cross-validation, I would recommend Shao 1997 over the widely cited 1993 paper:

• An Asymptotic Theory for Linear Model Selection (Shao, 1997)

You can skim through the major results, but it's interesting to read the discussion that follows. I thought the comments by Rao & Tibshirani, and by Stone, were particularly insightful. But note that while they discuss consistency, no claims are ever made regarding causality.

I think it's useful to review what we know about cross-validation. Statistical results around CV fall into two classes: efficiency and consistency.

Efficiency is what we're usually concerned with when building predictive models. The idea is that we use CV to determine a model with asymtptotic guarantees concerning the loss function. The most famous result here is due to Stone 1977 and shows that LOO CV is asymptotically equivalent to AIC. But, Brett provides a good example where you can find a predictive model which doesn't inform you on the causal mechanism.

Consistency is what we're concerned with if our goal is to find the "true" model. The idea is that we use CV to determine a model with asymptotic guarantees that, given that our model space includes the true model, we'll discover it with a large enough sample. The most famous result here is due to Shao 1993 concerning linear models, but as he states in his abstract, his "shocking discovery" is opposite of the result for LOO. For linear models, you can achieve consistency using LKO CV as long as $k/n rightarrow 1$ as $n rightarrow infty$. Beyond linear mdoels, it's harder to derive statistical results.

But suppose you can meet the consistency criteria and your CV procedure leads to the true model: $Y = beta X + e$. What have we learned about the causal mechanism? We simply know that there's a well defined correlation between $Y$ and $X$, which doesn't say much about causal claims. From a traditional perspective, you need to bring in experimental design with the mechanism of control/manipulation to make causal claims. From the perspective of Judea Pearl's framework, you can bake causal assumptions into a structural model and use the probability based calculus of counterfactuals to derive some claims, but you'll need to satisfy certain properties.

Perhaps you could say that CV can help with causal inference by identifying the true model (provided you can satisfy consistency criteria!). But it only gets you so far; CV by itself isn't doing any of the work in either framework of causal inference.

If you're interested further in what we can say with cross-validation, I would recommend Shao 1997 over the widely cited 1993 paper:

• An Asymptotic Theory for Linear Model Selection (Shao, 1997)

You can skim through the major results, but it's interesting to read the discussion that follows. I thought the comments by Rao & Tibshirani, and by Stone, were particularly insightful. But note that while they discuss consistency, no claims are ever made regarding causality.