-	About
-	Contract
-	Library
-	-	Books
-	-	-	AJGP
-	-	Code
-	-	Courses
-	-	Links
-	-	Media
-	-	Tutorials
-	People
-	-	David
-	-	-	Résumé
-	-	Shannon
-	Portfolio
-	Update

CroftSoft / Library / Tutorials

The Cross-product
is the Dot Product

David Wallace Croft

2005-06-01

Abstract

This tutorial shows the relationship between the Pearson product-moment correlation coefficient from statistics and the dot product from linear algebra.

Motivation

The standard deviation seems a bit odd when you compare it to the more intuitive average absolute deviation. Worse, the standard deviation stretches to reach out to highly deviant scores which you might consider throwing out of your data anyway. One wonders why the standard deviation formula is used at all when it seems somewhat arbitrary.

If you record sample data for two variables such as x and y and then compute the Pearson product-moment correlation coefficient, it will always fall between -1 and +1. This bit of magic occurs because the correlation coefficient is the average cross-product of z scores and the z scores are scaled by those stretchy standard deviations. This is a clue that the standard deviation is not as arbitrary as it first seems.

Bear in mind that the cross-product of z scores is not the same as the cross product of two three-dimensional vectors. The cross-product of z scores comes from statistics as used in the correlation coefficient and the other cross product comes from linear algebra as used in computer graphics and physics. It turns out, however, that the cross-product of statistics is the dot product of linear algebra.

The formula for the standard deviation looks almost like the length of multidimensional vector centered at the mean. Could the standard deviation be a way of converting a data vector into a normalized unit vector? Yes, if you note that the square root of the number of samples n in the denominator of the standard deviation cancels out when you compute the correlation coefficient. You then have the projection of one n-dimensional unit vector onto another. This is the cosine of the angle between them and this always falls between -1 and +1.

Proof

Mean
$m_{x} = (Σ_{i = 1}^{n} x_{i}) / n$
Variance
$σ_{x}^{2} = (Σ_{i = 1}^{n} {[x_{i} - m_{x}]}^{2}) / n$
Standard Deviation
$σ_{x} = \sqrt{σ_{x}^{2}}$
Z Score
$z_{x,i} = (x_{i} - m_{x}) / σ_{x}$
Correlation Coefficient
$r = \frac{Σ_{i = 1}^{n} [z_{x,i} * z_{y,i}]}{n}$
Average Cross-product
$r = \frac{Σ_{i = 1}^{n} [(\frac{x_{i} - m_{x}}{σ_{x}}) * (\frac{y_{i} - m_{y}}{σ_{y}})]}{n}$
Move the constants
$r = \frac{1}{σ_{x} * σ_{y} * n} * Σ_{i = 1}^{n} [(x_{i} - m_{x}) * (y_{i} - m_{y})]$
Dot Product
$r = \frac{1}{σ_{x} * σ_{y} * n} * [(x - m_{x}) \cdot (y - m_{y})]$
Standard Deviation in Terms of Norm
$σ_{x} = \sqrt{\frac{(Σ_{i = 1}^{n} {[x_{i} - m_{x}]}^{2})}{n}} = \frac{‖ x - m_{x} ‖}{\sqrt{n}}$
The n cancels out
$r = \frac{\sqrt{n} * \sqrt{n}}{‖ x - m_{x} ‖ * ‖ y - m_{y} ‖ * n} * [(x - m_{x}) \cdot (y - m_{y})]$
Projection of Unit Vectors
$r = \frac{(x - m_{x})}{‖ x - m_{x} ‖} \cdot \frac{(y - m_{y})}{‖ y - m_{y} ‖} = \cos (θ)$
Proportionate Reduction in Error
$r^{2} = \cos^{2} (θ) = 1 - \sin^{2} (θ)$
Perpendicular to Projection
$\sin (θ)$

Links

The Cross-productis the Dot Product

Abstract

Motivation

Proof

Links

The Cross-product
is the Dot Product