Let the wire is cut into 2 pieces of length $y \& 20-y$

Therefore side of square will be $=\frac{y}{4}$ and hexagon will be $=\frac{20-y}{6}$

Area of square$={\left(\frac{y}{4}\right)}^2$

Area of hexagon$=6\times \frac{\sqrt{3}}{4}{\left(\frac{20-y}{6}\right)}^2$

Total area$=\frac{y^2}{16}+\frac{\sqrt{3}{\left(20-y\right)}^2}{24}$

On differentiating,

$A^{\text{'}}\left(y\right)=\frac{2y}{16}+\frac{2\sqrt{3}\left(20-y\right)\left(-1\right)}{24}$

$A^{\text{'}}\left(y\right)=0 at y=\frac{40\sqrt{3}}{3+2\sqrt{3}}$

Length of side of regular hexagon$=\frac{1}{6}\left(20-y\right)=\frac{1}{6}\left(20-\frac{40\sqrt{3}}{3+2\sqrt{3}}\right)=\frac{10}{3+2\sqrt{3}}$