Let X = [ cases 0 & 1 & 0 0 & 0 & 1 0 & 0 & 0 cases ], Y = + + γ X...

Given, X = \left[\begin{cases}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{cases}\right], so {X}^{2} = \left[\begin{cases}0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0\end{cases}\right]

Now, finding $Y = \alphal + \betaX + \gamma{X}^{2}$ &$Z = {\alpha}^{2}I - \alpha\betaX + \left({\beta}^{2} - \alpha\gamma\right){X}^{2}$ by putting the value of $X \text{ and } {X}^{2}$ we get,

Y = \left[\begin{cases}\alpha & \beta & \gamma \\ 0 & \alpha & \beta \\ 0 & 0 & \alpha\end{cases}\right] \text{ and } Z = \left[\begin{cases}{\alpha}^{2} & - \alpha\beta & {\beta}^{2} - \alpha\gamma \\ 0 & {\alpha}^{2} & - \alpha\beta \\ 0 & 0 & {\alpha}^{2}\end{cases}\right]

We know that $Y \cdot {Y}^{- 1} = I$

⇒ \left[\begin{cases}\alpha & \beta & \gamma \\ 0 & \alpha & \beta \\ 0 & 0 & \alpha\end{cases}\right]\left[\begin{cases}\dfrac{1}{5} & \dfrac{- 2}{5} & \dfrac{1}{5} \\ 0 & \dfrac{1}{5} & \dfrac{- 2}{5} \\ 0 & 0 & \dfrac{1}{5}\end{cases}\right] = \left[\begin{cases}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{cases}\right]

⇒ \left[\begin{cases}\dfrac{\alpha}{5} & \dfrac{- 2\alpha}{5} + \dfrac{\beta}{5} & \dfrac{\alpha}{5} - \dfrac{2\beta}{5} + \dfrac{\gamma}{5} \\ 0 & \dfrac{\alpha}{5} & \dfrac{- 2\alpha}{5} + \dfrac{\beta}{5} \\ 0 & 0 & \dfrac{\alpha}{5}\end{cases}\right] = \left[\begin{cases}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{cases}\right]

On comparing L.H.S and R.H.S we get,

$⇒ \dfrac{\alpha}{5} = 1 ⇒ \alpha = 5$

$⇒ - \dfrac{2}{5}\alpha + \dfrac{\beta}{5} = 0 ⇒ \beta = 10$

$⇒ \dfrac{\alpha}{5} - \dfrac{2\beta}{5} + \dfrac{\gamma}{5} = 0 ⇒ \gamma = 15$

So, ${\left(\alpha - \beta + \gamma\right)}^{2} = {\left(5 - 10 + 15\right)}^{2} = 100$