\(5x^2-8xy+8y^2-12x+12=0\)
\(\Leftrightarrow5x^2-\left(8y+12\right)x+\left(8y^2+12\right)=0\left(1\right)\)
\(\Delta=\left(8y+12\right)^2-20\left(8y^2+12\right)\)
\(=64y^2+192y+144-160y^2-240\)
\(=-96y^2+192y-96\)
Để \(\left(1\right)\) có nghiệm nguyên thì \(\Delta\) là số chính phương
Đặt \(-96y^2+192y-96=k^2\)
\(\Leftrightarrow y^2-2y+1=-\dfrac{k^2}{96}\)
\(\Leftrightarrow\left(y-1\right)^2=-\dfrac{k^2}{96}\)
mà \(\left\{{}\begin{matrix}\left(y-1\right)^2\ge0,\forall y\in Z\\-\dfrac{k^2}{96}\le0,\forall k\in Z\end{matrix}\right.\)
\(\Rightarrow\left\{{}\begin{matrix}\left(y-1\right)^2=0\\-\dfrac{k^2}{96}=0\end{matrix}\right.\) \(\Leftrightarrow\left\{{}\begin{matrix}y=1\\k=0\end{matrix}\right.\)
\(\left(1\right)\Rightarrow5x^2-20x+20=0\)
\(\Leftrightarrow x^2-4x+4=0\)
\(\Leftrightarrow\left(x-2\right)^2=0\Leftrightarrow x=2\in Z\)
Vậy \(\left(x;y\right)=\left(2;1\right)\) thỏa mãn yêu cầu đề bài