Lời giải:
Ta có: \(5x^2+6xy+5y^2=3(x^2+y^2+2xy)+2(x^2+y^2)\)
\(=3(x+y)^2+2(x^2+y^2)\geq 3(x+y)^2+(x+y)^2\) (theo BĐT AM-GM)
\(\Leftrightarrow 5x^2+6xy+5y^2\geq 4(x+y)^2\Rightarrow \sqrt{5x^2+6xy+5y^2}\geq 2(x+y)\)
Thực hiện tương tự với những biểu thức còn lại suy ra:
\(P\geq \frac{2(x+y)}{x+y+2z}+\frac{2(y+z)}{y+z+2x}+\frac{2(z+x)}{z+x+2y}\)
\(P\geq 2\left(\frac{x+y}{x+y+2z}+\frac{y+z}{y+z+2x}+\frac{z+x}{z+x+2y}\right)=2\left(\frac{(x+y)^2}{(x+y+2z)(x+y)}+\frac{(y+z)^2}{(y+z+2x)(y+z)}+\frac{(z+x)^2}{(z+x+2y)(z+x)}\right)\)
Áp dụng BĐT Cauchy-Schwarz:
\(P\geq 2.\frac{(x+y+y+z+z+x)^2}{(x+y+2z)(x+y)+(y+z+2x)(y+z)+(z+x+2y)(z+x)}\)
\(\Leftrightarrow P\geq 2. \frac{4(x+y+z)^2}{2(x+y+z)^2+2(xy+yz+xz)}=\frac{4(x+y+z)^2}{(x+y+z)^2+xy+yz+xz}\)
\(\geq \frac{4(x+y+z)^2}{(x+y+z)^2+\frac{(x+y+z)^2}{3}}=3\) (theo AM-GM \(xy+yz+xz\leq \frac{(x+y+z)^2}{3}\))
Vậy \(P\geq 3\Leftrightarrow P_{\min}=3\)
Dấu bằng xảy ra khi \(x=y=z\)