For associativity let

(2)
\begin{align} A, B, C \in \end{align}

the set of matrices of the form stated above.

with

(3)
\begin{align} A = \left[ {\begin{array}{ccc} 1 & a & b \\ 0 & 1 & c \\ 0 & 0 & 1 \\ \end{array} } \right] B = \left[ {\begin{array}{ccc} 1 & d & e \\ 0 & 1 & f \\ 0 & 0 & 1 \\ \end{array} } \right] C = \left[ {\begin{array}{ccc} 1 & g & h \\ 0 & 1 & k \\ 0 & 0 & 1 \\ \end{array} } \right] \end{align}

(4)
\begin{align} \left[ {\begin{array}{ccc} 1 & a & d+e \\ 0 & 1 & k \\ 0 & 0 & 1 \\ \end{array} } \right] \end{align}

We want to show (AB)C=A(BC).

This is true since the set is a subgroup of M_{n}(R) which is a group.

The identity matrix is in this set so an identity exists.

This set has inverses since it is an upper row echelon matrix. The Det(A)=k where k cannot equal 0.

The set of matrices of the form

(5)
\begin{align} \left[\begin{array}{ccc}1 & a & b \\0 & 1 & c \\0 & 0 & 1\end{array}\right] \end{align}

with real entries under matrix multiplication has the subgroup

A=

(6)
\begin{align} \left[\begin{array}{ccc} 1 & 0 & b \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \end{align}

Where b is an element of Z.

What is a left coset of the subroup above?

For a left coset we multiply

(7)
\begin{align} \left[\begin{array}{ccc}1 & 0 & b \\0 & 1 & 1 \\0 & 0 & 1\end{array}\right] \end{align}

on the left of matrix A. Doing this we get

(8)
\begin{align} \left[ {\begin{array}{ccc} 1 & 0 & b+b \\ 0 & 1 & 1 \\ 0 & 0 & 1 \\ \end{array} } \right] \end{align}

Doing this again we get

(9)
\begin{align} \left[ {\begin{array}{ccc} 1 & a & b+b+b \\ 0 & 1 & 2 \\ 0 & 0 & 1 \\ \end{array} } \right] \end{align}

This leads us to define the left cosets as

(10)
\begin{align} \left[ {\begin{array}{ccc} 1 & a & b1+b2+b3+bn \\ 0 & 1 & c \\ 0 & 0 & 1 \\ \end{array} } \right] \end{align}

Where c is an element of Z and b is an element of Z.