\(x^2+2y^2+3xy=5\)
=>\(x^2+xy+2xy+2y^2=5\)
=>\(x\left(x+y\right)+2y\left(x+y\right)=5\)
=>\(\left(x+y\right)\left(x+2y\right)=5\)
=>\(\left(x+y\right)\left(x+2y\right)=1\cdot5=5\cdot1=\left(-1\right)\cdot\left(-5\right)=\left(-5\right)\cdot\left(-1\right)\)
TH1: \(\left\{{}\begin{matrix}x+y=1\\x+2y=5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}x+y-x-2y=1-5=-4\\x+y=1\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}-y=-4\\x+y=1\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}y=4\\x=1-y=1-4=-3\end{matrix}\right.\)
TH2: \(\left\{{}\begin{matrix}x+y=5\\x+2y=1\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}x+y-x-2y=5-1\\x+y=5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}-y=4\\x+y=5\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}y=-4\\x=5-y=5-\left(-4\right)=9\end{matrix}\right.\)
TH3: \(\left\{{}\begin{matrix}x+y=-1\\x+2y=-5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}x+y-x-2y=-1-\left(-5\right)\\x+2y=-5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}-y=-1+5=4\\x+2y=-5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}y=-4\\x=-5-2y=-5-2\cdot\left(-4\right)=-5+8=3\end{matrix}\right.\)
TH4: \(\left\{{}\begin{matrix}x+y=-5\\x+2y=-1\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}x+y-x-2y=-5-\left(-1\right)\\x+y=-5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}-y=-5+1=-4\\x+y=-5\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}y=4\\x=-5-y=-5-4=-9\end{matrix}\right.\)