a) ĐKXĐ: \(\left\{{}\begin{matrix}a>0\\b>0\\a\ne b\end{matrix}\right.\)
P = \(\dfrac{a-\sqrt{ab}+b+3\sqrt{ab}}{\left(\sqrt{a}+\sqrt{b}\right)\left(a-\sqrt{ab}+b\right)}.\left[\left(\dfrac{a+\sqrt{ab}+b-3\sqrt{ab}}{\left(\sqrt{a}-\sqrt{b}\right)\left(a+\sqrt{ab}+b\right)}\right):\dfrac{a-b}{a+\sqrt{ab}+b}\right]\)= \(\dfrac{\left(\sqrt{a}+\sqrt{b}\right)^2}{\left(\sqrt{a}+\sqrt{b}\right)\left(a-\sqrt{ab}+b\right)}.\left[\dfrac{\left(\sqrt{a}-\sqrt{b}\right)^2}{\left(\sqrt{a}-\sqrt{b}\right)\left(a+\sqrt{ab}+b\right)}.\dfrac{a+\sqrt{ab}+b}{a-b}\right]\)
= \(\dfrac{\sqrt{a}+\sqrt{b}}{a-\sqrt{ab}+b}.\dfrac{\sqrt{a}-\sqrt{b}}{a-b}\)
= \(\dfrac{1}{a-\sqrt{ab}+b}\)
b) có a = 16 và b = 4 (thoả mãn ĐKXĐ)
Thay a = 16, b =4 vào P có:
P = \(\dfrac{1}{16-\sqrt{16.4}+4}\)= \(\dfrac{1}{12}\)
Vậy tại a =16, b = 4 thì P = \(\dfrac{1}{12}\)