Solved – How is True risk equal to the expected value of the empirical risk

We know that the empirical risk is :
$L_s = frac{1}{n} sum_{i=1}^{n} l(f(x_i),z_i)$

where, $n$ = number of samples,$l(f(x),z)$ is a loss function, $S = (z_1,…,z_n)$ are the provided samples to testtrain on, $f(x_i)$ is the output of a learning algorithm and $z_i in Z $ is $(x_i, y_i)$

Some current literature like this (page 3 section 3.2), indicate that the true risk to be the expected value with respect to joint distribution of $n$ samples.
$L_mu = E_{Z^n}[L_s]$ $_{(1)}$

I do not understand how can the true risk be just the expected value of the empirical risk

This is how far I got :

We know that true risk is defined as :
$L_u = int l(f(x),z) dP(x,y) $

Now to write it in a summation form I can assume $n$ to be arbitrarily large, $L_u = frac{1}{n} sum_{i=1}^{n} l(f(x_i),z_i).P(x_i,y_i)$ $_{(2)}$

Now have no idea about the next step to reconcile equation $(2)$ and $(1)$

You don't need to assume $n$ to be arbitrarily large. It's just linearity of expectation:$DeclareMathOperator{E}{mathbb E}$ begin{align} E[text{empirical risk}] &= Eleft[ frac1n sum_{i=1}^n l(f(x_i), z_i) right] \&= frac1n sum_{i=1}^n Eleft[ l(f(x_i), z_i) right] \&= frac1n sum_{i=1}^n int l(f(x_i), z_i) ,mathrm{d}P(x_i, z_i) \&= frac1n sum_{i=1}^n text{true risk} \&= text{true risk} .end{align}

Similar Posts:

Rate this post

Leave a Comment