# Solved – Confidence interval of ratio estimator

As an example, consider a program that executes on two computers, A and B.
Measuring the execution time of 3 executions each shows the following results:

System A: 10s, 10s, 4s
System B: 8s, 8s, 2s

With these values, we can calculate the mean and the confidence interval for both systems (considering Student's t distribution for a confidence of 90%):

A: 8s +- 5.84
B: 6s +- 5.84

We can now calculate an average speedup between System A and B as 8s/6s =
1.33.

My question is: Is it possible to calculate an error or confidence interval for this speedup? If yes, how?

Contents

You should take a look at the theory of 'ratio estimators' , there are references to find via google (e.g. http://www.math.montana.edu/~parker/PattersonStats/Ratio.pdf). The idea is that you can compute the mean and the variance of a ratio of random variables and then use the mean and variance to define confidence intervals.

But I think you will need larger samples.

@Matthias Diener

So for your example: \$y = (10, 10, 4), bar{y}=8\$, \$x=(8,8,2), bar{x}=6\$

the ratio estimator for your speedup is \$r=frac{bar{y}}{bar{x}}=frac{8}{6}=1.333\$ and the variance of the estimator is \$sigma_r^2=frac{1}{bar{x}^2}frac{s_r^2}{3}\$, with \$s_r^2=frac{1}{3-1}sum_{i=1}^3 (y_i – rx_i)^2\$.

The R-code looks like:

``y<-c(10, 10, 4) x<-c(8, 8 , 2)  m.x<-mean(x) m.y<-mean(y)  m.x m.y  r<-m.y/m.x r  s.r.2<-1/(length(x)-1) * sum(( y - r * x ) * ( y - r * x))  variance.r<- 1/m.x^2 * s.r.2/length(x)  stand.deviation.r<-sqrt(variance.r)  cat(paste("estimate for speedup:", round(r, digits=3), "with stand. deviation:", round(stand.deviation.r, digits=3))) ``

Rate this post