Sorry, but polyint cannot integrate a rational polynomial, and certainly not over an infinite region. Depending on the roots of the polynomial, it may or may not have an integral.
The simplest way to do your integration is to use the symbolic toolbox.
syms x
P = x^4 + 8*x^3 + 23*x^2 + 28*x + 12;
int(x/P,x,[1,inf])
ans =
log((3*2^(1/2))/8) + 2/3
If you have only the coefficients of the polynomial, you can still do it.
pcoeff = [1 8 23 28 12];
roots(pcoeff)
ans =
-3
-2
-2
-1
So a happy list of roots.
int(x/poly2sym(pcoeff,x),[1,inf])
ans =
log((3*2^(1/2))/8) + 2/3
