Given equation is
$2 x^2-10 x y+12 y^2+5 x+\lambda y-3=0$
Here, $a=2, h=-5, b=12, g=\frac{5}{2}, f=\frac{\lambda}{2}, c=-3$
For pair of lines $\left|\begin{array}{lll}a & h & g \\ h & b & f \\ g & f & c\end{array}\right|=0$
$\begin{array}{cc}\Rightarrow & \left|\begin{array}{rrr}2 & -5 & 5 / 2 \\ -5 & 12 & \lambda / 2 \\ 5 / 2 & \lambda / 2 & -3\end{array}\right|=0 \\ \Rightarrow & 2\left(-36-\frac{\lambda^2}{4}\right)+5\left(15-\frac{5 \lambda}{4}\right) \\ & +\frac{5}{2}\left(\frac{-5 \lambda}{2}-30\right)=0 \\ \Rightarrow & -72-\frac{\lambda^2}{2}+75-\frac{25 \lambda}{4}-\frac{25 \lambda}{4}-75=0 \\ \Rightarrow & \lambda^2+25 \lambda+144=0 \\ \Rightarrow & (\lambda+9)(\lambda+16)=0 \\ \Rightarrow & \lambda=-9 \quad(\because|\lambda| < 16)\end{array}$