By computing the matrix product, we have
[tex]A^2 = \left[\begin{array}{cc}ab&b^2\\-a^2&-ab\end{array}\right] \cdot\left[\begin{array}{cc}ab&b^2\\-a^2&-ab\end{array}\right]=\left[\begin{array}{cc}a^2b^2-a^2b^2&ab^3-ab^3\\-a^3b+a^3b&-a^2b^2+a^2b^2\end{array}\right][/tex]
and as you can see, all the entries of this matrix are terms of the form [tex]x-x[/tex], so the matrix is composed by nothing but zeroes.