Áp dụng BĐT cauchy ta có:\(\left\{{}\begin{matrix}x^2+y^2\ge2xy\\y^2+z^2\ge2yz\\x^2+z^2\ge2xz\end{matrix}\right.\)
\(P\le\dfrac{1}{4xy+4x+4}+\dfrac{1}{4yz+4y+4}+\dfrac{1}{4xz+4z+4}=\dfrac{1}{4}\left(\dfrac{1}{xy+x+1}+\dfrac{1}{yz+y+1}+\dfrac{1}{xz+x+1}\right)\)
xét biểu thức \(\dfrac{1}{xy+x+1}+\dfrac{1}{yz+y+1}+\dfrac{1}{zx+z+1}=\dfrac{1}{xy+x+1}+\dfrac{x}{1+yx+x}+\dfrac{xy}{x+1+xy}=\dfrac{xy+x+1}{xy+x+1}=1\)do đó \(P\le\dfrac{1}{4}\)
dấu = xảy ra khi x=y=z=1
Trước tiên ta tính:
\(\dfrac{1}{x+xy+1}+\dfrac{1}{y+yz+1}+\dfrac{1}{z+zx+1}\)
Đặt: \(\left\{{}\begin{matrix}x=\dfrac{a}{b}\\y=\dfrac{b}{c}\\z=\dfrac{c}{a}\end{matrix}\right.\left(a,b,c\ne0\right)\)
Thì ta có: \(\dfrac{1}{\dfrac{a}{b}+\dfrac{a}{b}.\dfrac{b}{c}+1}+\dfrac{1}{\dfrac{b}{c}+\dfrac{b}{c}.\dfrac{c}{a}+1}+\dfrac{1}{\dfrac{c}{a}+\dfrac{c}{a}.\dfrac{a}{b}+1}\)
\(=\dfrac{bc}{ab+ac+bc}+\dfrac{ca}{ab+bc+ca}+\dfrac{ab}{ab+bc+ca}=1\)
Quay về bài toán ban đầu. Ta có:
\(P=\dfrac{1}{\left(x+2\right)^2+y^2+2xy}+\dfrac{1}{\left(y+2\right)^2+z^2+2yz}+\dfrac{1}{\left(z+2\right)^2+x^2+2xz}\)
\(=\dfrac{1}{x^2+4x+4+y^2+2xy}+\dfrac{1}{y^2+4y+4+z^2+2yz}+\dfrac{1}{z^2+4z+4+z^2+2xz}\)
\(=\dfrac{1}{\left(x-y\right)^2+4x+4xy+4}+\dfrac{1}{\left(y-z\right)^2+4y+4yz+4}+\dfrac{1}{\left(z-x\right)^2+4z+4zx+4}\)
\(\le\dfrac{1}{4x+4xy+4}+\dfrac{1}{4y+4yz+4}+\dfrac{1}{4z+4zx+4}\)
\(=\dfrac{1}{4}.\left(\dfrac{1}{x+xy+1}+\dfrac{1}{y+yz+1}+\dfrac{1}{z+zx+1}\right)=\dfrac{1}{4}\)