First, do you have a matlabpool open? If no matlabpool is open then the parfor loop will not be run in parallel anyway.
Here the data transfer and communication overhead cost more than the computation. parfor will do the most good when the data transfer is low and the calculation time is high.
(And I assume you are aware of: x = rand(N,1) % ;) )