Linear algebra for programmers

Inspired by Eric Meijer I thought I would list some of the things that have been useful to me recently in transforming a Machine Learning article to actual running C++. So the idea is a kind of phrase book for programmers venturing into the strange lingo of Linear Algebra. Most examples are taken from this paper since I recently implemented it.

In the examples I will assume no familiarity with Linear Algebra so we will take some shortcuts, but we will get the job done. The idea is to transfer these strange terms into something familiar to any developer. I am pretty sure that you would fail horribly trying to do any math with this understanding of Linear Algebra, but it should work fine for coding.

Scalar

int, float, double, or objects if you please. Basic arithmetic should be implemented (+ - / *).

Vector

An ordered list of scalars.  scalar[]

Matrix

An ordered list of vectors [scalar[]]. Can also be multidimensional. [[[scalar[]]]. All the innermost lists have the same length. Normally we only need the list of lists. The length of the lists is called the dimensionallity of the matrix. An n x m Matrix is a list of size n containing lists of size m.

Dot product / Inner product

The most normal way to multiply two vectors. Takes the sum of the product of the elements. You can only take the dot product of two vectors of the same length:

int dotProduct(int[] a, int[] b){

    if (a.size()!=b.size) throw exception;

    int result =0;

    for(int i=0; i<a.size();++i){

        result += a[i]*b[i];

    }

    return result;

}

Notice that multiplying two vectors returns a scalar.


Diagonal matrix

Mathematicians spend a lot of time swapping elements or building strange structures like the diagonal matrix because Linear Algebra relies heavily on the rules for multiplying different elements. Say I want to multiply two vectors like this:

int[] DiagonalProduct(int[] a, int[] b){

    if (a.size()!=b.size) throw exception;    

    int[] result.reserve(a.size());.

    for(int i=0; i,a<size();++i){

        result[i] = a[i]*b[i];

    }

    return result;

}

You could call it element-wise multiplication. There is no rule like that in Linear Algebra between two vectors. So what is done is put the a vector on the diagonal of a matrix A and fill the rest of the spaces with 0's. A is then called a diagonal matrix. Then the above corresponds to multiplying A*b with the rules for Matrix vector multiplication.

Permutation matrix

The same is the case for the permutation matrix. What it does is swap the position of the list in a matrix. Consider a permutation matrix P=[[0,1],[1,0]]. What it does is swap the first and second row of a matrix. Example P*[[2,4],[6,5]] = [[6,5],[2,4]]. I C++ you can use the swap method to easily do the same job.

I hope to write more entries to this some day, but for now my 30 minutes are up.