Getting different results training on the same datasets each time

Question

Katy 2023-8-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2003867-getting-different-results-training-on-the-same-datasets-each-time

编辑： Ayush Anand 2023-8-29

I'm using 'fitrgp' to train my GPR model. I've specified hyperparameters I'd like it to optimize, but I'm getting slightly different results each time I train it on the same dataset. I know this is a feature of ML in general, but I'd like to know if there's any way to get a reproducible result each time.

I'm already using rng("default") and using the "expected-improvement-plus" acquisition function to improve the reproducibility.

I'd really appreciate any insight into if this is possible!

3 个评论
显示 1更早的评论隐藏 1更早的评论

Katy 2023-8-3

在 MATLAB Online 中打开

From https://www.mathworks.com/help/stats/predict-battery-soc-using-machine-learning.html
rng("default")
Mdl = fitrgp(trainData,"SOC", ...
    OptimizeHyperparameters=["BasisFunction","KernelFunction","Standardize"], ...
    HyperparameterOptimizationOptions= ...
    struct(AcquisitionFunctionName="expected-improvement-plus",UseParallel=true))

My code is similar to this as its based off the based off the example here.

One thing I investigated was whether this had to do with the parallel processing. However, even after trying to set the random seeds for the workers, I still am not getting reproducble results either.

https://www.mathworks.com/help/parallel-computing/control-random-number-streams-on-workers.html

the cyclist 2023-8-5

Sorry I did not see this reply earlier.

This documentation discusses reproducibility in parallel computations. It seems to have some distinct recommendations from the page you linked.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ayush Anand 2023-8-29

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2003867-getting-different-results-training-on-the-same-datasets-each-time#answer_1295471

编辑：Ayush Anand 2023-8-29

Hi Katy,

I understand that you are doing Gaussian process regression and getting slightly different results every time, you train the model on the same dataset.

The code that you posted shows that you are using parallel computation option while training. Since MATLAB parallel computing toolbox is inherently non-deterministic in nature, generally it is not sought out when reproducible results are required, and serialization is preferred.

In this case, for producing serialized results, you can set “UseParallel” to false and keep the “rng(“default”)” statement. This disables parallel computations and setting the seed to default every time ensures reproducible results.

The MathWorks Documentation linked below is referring to a workaround to produce reproducible results with parallel computing toolbox, but it doesn’t seem to be possible to inculcate this with the “fitrgp” function:

https://in.mathworks.com/help/stats/reproducibility-in-parallel-statistical-computations.html

Also, as you mentioned in the question, most of the ML algorithms (including Gaussian Process regression) inherently have some randomness due to re-initializations and samplings, so you are bound to get slightly different results.

I hope this helps.