We have
[tex]\dfrac{x^2}{x+1}=\dfrac{(x+1)^2-2(x+1)+1}{x+1}=(x+1)-2+\dfrac1{x+1}=x-1+\dfrac1{x+1}[/tex]
So
[tex]\displaystyle\lim_{x\to\infty}\left(\frac{x^2}{x+1}-ax-b\right)=\lim_{x\to\infty}\left(x-1+\frac1{x+1}-ax-b\right)=0[/tex]
The rational term vanishes as x gets arbitrarily large, so we can ignore that term, leaving us with
[tex]\displaystyle\lim_{x\to\infty}\left((1-a)x-(1+b)\right)=0[/tex]
and this happens if a = 1 and b = -1.
To confirm, we have
[tex]\displaystyle\lim_{x\to\infty}\left(\frac{x^2}{x+1}-x+1\right)=\lim_{x\to\infty}\frac{x^2-(x-1)(x+1)}{x+1}=\lim_{x\to\infty}\frac1{x+1}=0[/tex]
as required.