Given,

$\overrightarrow{a}=\hat{i}+2\hat{j}+3\hat{k}$,  $\overrightarrow{b}=\hat{i}+\hat{j}-\hat{k}$,  $\overrightarrow{a}·\overrightarrow{c}=11$, $\overrightarrow{b}·\left(\overrightarrow{a}\times \overrightarrow{c}\right)=27$ and $\overrightarrow{b}·\overrightarrow{c}=-\sqrt{3}\left|\overrightarrow{b}\right|$

Now finding, $\left(\overrightarrow{b}\times \overrightarrow{a}\right)$ we get,

$\left(\overrightarrow{b}\times \overrightarrow{a}\right)=\left|\begin{array}{ccc}\hat{i}& \hat{j}& \hat{k}\\ 1& 1& -1\\ 1& 2& 3\end{array}\right|=5\hat{i}-4\hat{j}+\hat{k}$

Let $\overrightarrow{c}=c_1\hat{i}+c_2\hat{j}+c_3\hat{k}$

Now solving, $\overrightarrow{a}·\overrightarrow{c}=11$ we get,

$c_1+2c_2+3c_3=11 .......\left(1\right)$

Now solving, $\overrightarrow{b}·\overrightarrow{c}=-\sqrt{3}\left|\overrightarrow{b}\right|$ we get,

$c_1+c_2-c_3=-\sqrt{3}\sqrt{3}$

$c_1+c_2-c_3=-3 ..........\left(2\right)$

Now solving $\left(\overrightarrow{b}\times \overrightarrow{a}\right)·\overrightarrow{c}=27$ we get,

$5c_1-4c_2+c_3=27...........\left(3\right)$ 

Now on solving equation $\left(1\right), \left(2\right) \& \left(3\right)$ we get,

$\overrightarrow{c}=3\hat{i}-2\hat{j}+4\hat{k}$

Hence, ${\left|\overrightarrow{a}\times \overrightarrow{c}\right|}^2={\left|\begin{array}{ccc}\hat{i}& \hat{j}& \hat{k}\\ 1& 2& 3\\ 3& -2& +4\end{array}\right|}^2$$={\left|14\hat{i}+5\hat{j}-8\hat{k}\right|}^2$

$\Rightarrow |\overrightarrow{a}\times \overrightarrow{c}{|}^2={14}^2+5^2+8^2=285$