Given, density of sphere, $p=A r^\alpha$ (where, $r=$ radial distance and $A$ and $\alpha$ are constants)
Consider an elemental spherical shell of radius $r$ and thickness $d r$


Mass of elemental spherical shell,
$$
\begin{aligned}
& d m=\text { Volume } \times \text { Density } \\
& d m=\left(4 \pi r^2\right) d r \cdot A r^\alpha=4 \pi A r^{2+\alpha} d r
\end{aligned}
$$
Mass of entire solid sphere,
$$
\begin{aligned}
& M=4 \pi A \int_0^R r^{2+\alpha} d r \\
& M=4 \pi A\left[\frac{r^{3+\alpha}}{3+\alpha}\right]_0^R=\frac{4 \pi A}{3+\alpha} \cdot R^{3+\alpha}
\end{aligned}
$$
Now, moment of inertia of elemental spherical shell is
$$
d I=\frac{2}{3}(d m) \cdot r^2=\frac{2}{3}\left(4 \pi A r^{2+\alpha} d r\right) r^2
$$
Moment of inertia of entire solid sphere,
$$
I=\int_0^R d I=\frac{2}{3} 4 \pi A \int_0^R r^{4+\alpha} \cdot d r
$$


$$
\begin{aligned}
& I=\frac{2}{3} 4 \pi A\left[\frac{r^{5+\alpha}}{5+\alpha}\right]_0^R \Rightarrow I=\frac{2}{3} 4 \pi A\left[\frac{R^{5+\alpha}}{5+\alpha}\right] \\
& I=\frac{2}{3}\left(\frac{4 \pi A}{3+\alpha} \cdot R^{3+\alpha}\right) \cdot \frac{R^2(3+\alpha)}{5+\alpha}
\end{aligned}
$$
From Eq. (ii), we get
$$
I=\frac{2}{3} M R^2 \cdot\left(\frac{3+\alpha}{5+\alpha}\right)
$$


It is given in the question,
$$
I=\frac{6}{7} M R^2
$$
On comparing Eqs (iii) and (iv), we get
$$
\begin{aligned}
\frac{2}{3} M R^2 \cdot\left(\frac{3+\alpha}{5+\alpha}\right) & =\frac{6}{7} M R^2 \Rightarrow \frac{3+\alpha}{5+\alpha}=\frac{9}{7} \\
21+7 \alpha & =45+9 \alpha \\
\alpha \quad \alpha & =-12
\end{aligned}
$$