$\begin{aligned} & \text A \cdot B=\left[\begin{array}{lll}b & a & 0 \\ c & 0 & b \\ a & a & b\end{array}\right]\left[\begin{array}{lll}0 & a & b \\ b & 0 & c \\ b & a & a\end{array}\right]=\left[\begin{array}{ccc}2 & 2 & 7 \\ 1 & 8 & 5 \\ 3 & 6 & 10\end{array}\right] \\ & \Rightarrow\left[\begin{array}{ccc}a b & a b & b^2+a c \\ b^2 & a c+a b & b c+a b \\ a b+b^2 & a^2+a b & 2 a b+a c\end{array}\right]=\left[\begin{array}{ccc}2 & 2 & 7 \\ 1 & 8 & 5 \\ 3 & 6 & 10\end{array}\right] \\ & \therefore \quad b^2=1 \\ & a b=2 \Rightarrow a=2 \\ & a^2 b^2=4 \\ & \therefore \quad a^2=4\end{aligned}$
$\begin{array}{ll} & b^2+a c=7 \Rightarrow a c=6 \\ \therefore \quad & c=3 \\ & a^2+b^2+c^2=2^2+1^2+3^2=14 .\end{array}$