Proportionate reduction of error

If both ${\displaystyle x}$ and ${\displaystyle y}$ vectors have cardinal (interval or rational) scale, then without knowing ${\displaystyle x}$, the best predictor for an unknown ${\displaystyle y}$ would be ${\displaystyle {\bar {y}}}$, the arithmetic mean of the ${\displaystyle y}$-data. The total prediction error would be ${\displaystyle E_{1}=\sum _{i=1}^{n}{(y_{i}-{\bar {y}})^{2}}}$ .

If, however, ${\displaystyle x}$ and a function relating ${\displaystyle y}$ to ${\displaystyle x}$ are known, for example a straight line ${\displaystyle {\hat {y}}_{i}=a+bx_{i}}$, then the prediction error becomes ${\displaystyle E_{2}=\sum _{i=1}^{n}{(y_{i}-{\hat {y}})^{2}}}$. The coefficient of determination then becomes ${\displaystyle r^{2}={\frac {E_{1}-E_{2}}{E_{1}}}=1-{\frac {E_{2}}{E_{1}}}}$ and is the fraction of variance of ${\displaystyle y}$ that is explained by ${\displaystyle x}$. Its square root is Pearson's product-moment correlation ${\displaystyle r}$.

There are several other correlation coefficients that have PRE interpretation and are used for variables of different scales:

More information η ...

predict	from	coefficient	symmetric
nominal, binary	nominal, binary	Guttman's λ[2]	yes
ordinal	nominal	Freeman's θ[3]	yes
cardinal	nominal	η${\displaystyle ^{2}}$[4]	no
ordinal	binary, ordinal	Wilson's e [5]	yes
cardinal	binary	point biserial correlation	yes

Close