When you say "classifier ensemble", I assume you mean "decision tree ensemble". The time it takes to grow a tree depends on the data, both predictors and labels. If the classes are barely separable after you add noise, the tree may bail out early because it fails to find a good split. Individual trees are saved in the Trained property of the ensemble object. You can check how deep they are with and without label noise.
If the classification accuracy of the ensemble with added noise is at the level you would expect, I don't see why you need to worry about training being too fast.