Given ${\left(A^T\right)}^{-1}=A, \Rightarrow A{A}^T=A^{T}A=I.$

Now, $X=AB{A}^T$

$\Rightarrow X^{2021}={\left(AB{A}^T\right)}^{2021}$

$\Rightarrow X^{2021}=\left(AB{A}^T\right)\left(AB{A}^T\right)\left(AB{A}^T\right)...\left(AB{A}^T\right)$

We know that, the matrix multiplication is associative i.e. for three matrices, we have $\left(PQ\right)R=P\left(QR\right).$

$\Rightarrow X^{2021}=AB\left(A^{T}A\right)B\left(A^{T}A\right)B\left(A^{T}A\right)...\left(A^{T}A\right)B{A}^T$

Using the given condition, we get

$\Rightarrow X^{2021}=AB\left(I\right)B\left(I\right)B\left(I\right)...\left(I\right)B{A}^T$

$\Rightarrow X^{2021}=ABBB...B{A}^T$

$\Rightarrow X^{2021}=A{B}^{2021}{A}^T$

$\Rightarrow A^T{X}^{2021}A=A^{T}A{B}^{2021}{A}^{T}A$

$\Rightarrow A^T{X}^{2021}A=I{B}^{2021}I=B^{2021}.$

Now $B^2=BB=\left[\begin{array}{cc}1& 2\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 2\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 4\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 2\times 2\\ 0& 1\end{array}\right].$

And $B^3=B^{2}B=\left[\begin{array}{cc}1& 4\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 2\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 6\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 2\times 3\\ 0& 1\end{array}\right].$

And $B^4=B^{3}B=\left[\begin{array}{cc}1& 6\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 2\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 8\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 2\times 4\\ 0& 1\end{array}\right].$

Hence, $B^{2021}==\left[\begin{array}{cc}1& 2\times 2021\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}1& 4042\\ 0& 1\end{array}\right].$

Hence, $A^T{X}^{2021}A=\left[\begin{array}{cc}1& 4042\\ 0& 1\end{array}\right].$