This is an idea that crops up in many contexts. Starting from Linear Relationships which was a big topic year 9 in school. That is the kind of relationship between 2 values that shows up as a straight line on a graph in cartesian space with those values as the axes. Then extended to many variables.

The equation \(\ ax+by+cz=d\ \) describes a linear relationship between `x`, `y` and `z` when `a`, `b`, `c` and `d` are constants. This relationship could be shown as a plane in a 3D graph. A Linear Relationship graphs as a flat shape one dimension smaller than the number of variables. Some of the constants might be 0, you will still get a plane in 3D space (or nothing or everywhere if \(a=b=c=0\!\) ).

\(\ ax+by+cz\ \) is a Linear Combination of three variables `x, y` and `z`. Much more generally a Linear Combination of a set of things is made by multiplying each by a scalar (possibly zero or one) and adding the results. If there are `n` elements in the set `X` then we could write this \(\ \sum_{i=1}^n a_i x_i\ \) where `a_i` are scalars and `x_i` are the elements of `X`.

We sometimes use the name Linear Space rather than Vector Space. It refers to the kind of independence each dimension of the space has. Any set of Basis vectors must be Linearly Independent, that is none of them are a Linear Combination of the others. This is closely related to Dimension or the Degrees of Freedom within some model. Variables are rather like Degrees of Freedom, you can adjust each quite independently of the others. The space in which the relationship above was graphed is 3-dimensional, you need to pick three values to get to any point. The relationship given could be considered a function of `x` and `y` by putting `z` as the subject, defined as \(\ {\rm f}(x,y)=-\frac1c(ax+by-d)\ \). Now we see there are two variables input with an output for each possible input pair. A way of considering the graph of it is \({\rm f}(x,y)\) as the height above (or negative, below) a cartesian plane with axes `x` and `y`. It will be the same surface in 3D space as the equation gave, but we no longer have any mention of `z`. It is a 2D surface because it is the output of a function with 2 Degrees of Freedom, two variables that may be independently varied. It is a flat surface, a plane, because \(\ ax+by\ \) is a linear combination of `x` and `y` (which is then subtracted from `d` then scaled by \(\frac1c\!\) ).

This idea of independence , dimension and degrees of freedom is deeply part of Vector Space.

Matrices have a great deal to do with that.

is from space to space preserving addition and scalar multiplication. By this we mean we can do the transformation then the operation or do the operation first then do the transformation and the result will be the same. \[{\rm f}:V\mapsto W\quad\text{is linear iff}\quad{\rm f}({\bf a}+{\bf b})={\rm f}({\bf a})+{\rm f}({\bf b})\quad\text{and}\quad{\rm f}(c{\bf a})=c{\rm f}({\bf a}),\] so linear combinations are preserved (since addition and scalar multiplication each associative).

eg \(\ {\bf v}\mapsto c{\bf v}\ \) is linear. But \(\ x\mapsto x^2\ \) and \(\ x\mapsto x+1\ \) are not linear. The last is affine instead. For a cartesian or vector space any rotation, reflection, stretching or skewing which preserves the origin is a linear map, a linear transformation. For the appropriate space of functions differentiation, definite integral and indefinite integral with a fixed starting point are linear maps, linear operations on those functions producing new functions. Expected value of a random variable is a linear function (but variance is not). I’ve seen HSC questions that are very quickly answered if you understand the meaning of this last sentence, though most students just use particular memorised properties from the ‘expected value’ topic — properties that are actually true of any linear function, and much much more useful than that single, specific, topic. Many probably could not name or use the much more general and widespread idea and think of the topic in isolation as one more group of methods to memorise.

We can use matrix algebra and notation to work with transformations including linear and affine ones.

Now consider square matrices. For matrix multiplication with `n`-dimenional square matrices we have an Identity, the Unit for multiplication, like `1` in the scalars, \[I=\left[\matrix{ 1&0&\cdots&0\cr 0&1&\ddots&\vdots\cr \vdots&\ddots&\ddots&0\cr 0&\cdots&0&1\cr }\right]\ \text{ because }\ IA=AI=A=1A.\] Consider: ` X=xI=[ [x,0],[0,x] ],` 2\({\times}\)2 matrix `A` and the 2D vector where `V` is \(\bf v\) written as single column matrix. \(\ xA=XA=AX\ \) and \(x\bf v\) corresponds to \(XV\).

`X` is the simple linear transformation in 2D geometric space of scaling by `x` around the origin. In 2D vector space it is stretching the vectors by `x`, identical to scalar multiplication. Multiplying another 2\({\times}\)2 matrix results in composing the transformations into a single transformation, in the case of `X` it is the same as scalar multiplication by `x`.

Within a space, given a basis, each linear transformation can be represented by a square matrix and every appropriately sized matrix is a linear transformation. Transformations can be thought of as mappings, or as functions. Matrices can represent these very usefully. In two dimensional space here are some examples:

\(\left[\matrix{0 & 0 \cr 0 & 1 }\right],\quad\) Projection onto vertical axis

\(\left[\matrix{1 & 2 \cr 0 & 1 }\right],\quad\) Horizontal shear

\(\left[\matrix{3 & 0 \cr 0 & \frac12 }\right],\quad\) Squeeze (asymetrically)

\(\left[\matrix{\cos2\theta & \sin2\theta \cr \sin2\theta & -\cos2\theta}\right],\quad\) Reflection across line through origin, angle `theta` to horizontal

Now consider square matrices. For matrix multiplication with `n`-dimenional square matrices we have an Identity, the Unit for multiplication, like `1` in the scalars, \[I=\left[\matrix{ 1&0&\cdots&0\cr 0&1&\ddots&\vdots\cr \vdots&\ddots&\ddots&0\cr 0&\cdots&0&1\cr }\right]\ \text{ because }\ IA=AI=A=1A.\] Consider: ` X=xI=[ [x,0],[0,x] ],` 2\({\times}\)2 matrix `A` and the 2D vector where `V` is \(\bf v\) written as single column matrix. \(\ xA=XA=AX\ \) and \(x\bf v\) corresponds to \(XV\).

`X` is the simple linear transformation in 2D geometric space of scaling by `x` around the origin. In 2D vector space it is stretching the vectors by `x`, identical to scalar multiplication. Multiplying another 2\({\times}\)2 matrix results in composing the transformations into a single transformation, in the case of `X` it is the same as scalar multiplication by `x`.

Within a space, given a basis, each linear transformation can be represented by a square matrix and every appropriately sized matrix is a linear transformation. Transformations can be thought of as mappings, or as functions. Matrices can represent these very usefully. In two dimensional space here are some examples:

\(\left[\matrix{0 & 0 \cr 0 & 1 }\right],\quad\) Projection onto vertical axis

\(\left[\matrix{1 & 2 \cr 0 & 1 }\right],\quad\) Horizontal shear

\(\left[\matrix{3 & 0 \cr 0 & \frac12 }\right],\quad\) Squeeze (asymetrically)

\(\left[\matrix{\cos2\theta & \sin2\theta \cr \sin2\theta & -\cos2\theta}\right],\quad\) Reflection across line through origin, angle `theta` to horizontal

Continued in the main discussion.