The remainder when $2021$ divided by $7$ is $-2$. Hence, the problem reduces to finding the remainder when ${\left(-2\right)}^{2023}$ is divided by $7$.

$=\frac{{\left(-2\right)}^{2022}\times \left(-2\right)}{7}$

$=-\frac{2\times 2^{2022}}{7}$

$=\frac{-2\times {\left(2^3\right)}^{674}}{7}$

$=\frac{-2\times {\left(8\right)}^{674}}{7}$

$=\frac{-2\times {\left(1+7\right)}^{674}}{7}$

Using binomial theorem, we get

$=\frac{-2\times \left(1+7k\right)}{7}$ where $k$ is an integer.

$=\frac{-2-14k}{7}$

Clearly, the remainder when $-14k$ is divisible by $7$ is $0$.

$=\frac{-2}{7}$

$=\frac{-2-5+5}{7}=\frac{5}{7}$

Hence, the remainder is $5$.