In general, you can't.
This is not aquestion of "dividing a matrix by a polynomial". In fact, that makes no sense at all. You have a list of points (x) that you are evaluating a polynomial at, and now, you want to recover the original x. Thus, given a point or set of points x, you have computed the value y for each x.
y = P(x)
Now, for each y, you want to recover the original value of x. Sorry, but you cannot do that operation uniquely, unless the polynomial is a linear one, in which case it is trivial. Even if the function is a quadratic polynomial, not the cubic one that you show, there will be multiple solutions, so no unique solution. You can never know which of those multiple roots to choose. This is no different from asking the question:
If I know the value of y=x^2, then which of the possible solutions,
x = sqrt(y)
or
x = -sqrt(y)
did I start with?
