Given that, $\left|\begin{array}{ccc}x+1& x& x\\ x& x+\lambda & x\\ x& x& x+{\lambda }^2\end{array}\right|=\frac{9}{8}\left(103x+81\right)$

Put $x=0$ as $x\in R$

$\Rightarrow \left|\begin{array}{ccc}1& 0& 0\\ 0& \lambda & 0\\ 0& 0& {\lambda }^2\end{array}\right|=\frac{9}{8}\left(103\left(0\right)+81\right)$

On expanding the determinant we get,

$\Rightarrow {\lambda }^3=\frac{9}{8}\times 81$

$\Rightarrow {\lambda }^3=\frac{9^3}{2^3}$

$\Rightarrow \lambda =\frac{9}{2}$ and $\frac{\lambda }{3}=\frac{9}{6}$

Now the required quadratic equation is $x^2-\left(\frac{9}{2}+\frac{9}{6}\right)x+\frac{9}{2}\times \frac{9}{6}=0$

$\Rightarrow x^2-\left(6\right)x+\frac{27}{4}=0$

$\Rightarrow 4x^2-24x+27=0$

Hence, the required quadratic equation is $4x^2-24x+27=0$