Randomness is lost in parfor loop on GPU?

2022 4 12
1 个回答
更新时间：2022 4 15
4 次查看（30 天）

0 个投票

I have a code with a following structure:
% my_code_seed#.m
myoutputs1 = my_function1(myinputs1);
seed = #;
rng(seed);
parfor iter = 1:50
myouputs2 = my_gpu_function2(myoutputs1, my_function_of_Gaussian);
result(iter).name = myoutputs2;
end
save(file_name, 'result');
What I want to do is to model some random behavior of something by using this code repeatedly. What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2". I run these codes in a parallel way on a GPU cluster to save the computation time. However, it seems like the results from the codes are very similar, even though I'm giving different values of seed.
Do you have an idea why rng(seed); is not working properly in this case? I appreciate any help.

请先登录，再回答此问题。

回答（1 个）

Steven Lord
Steven Lord 2022-4-13

0 个投票

What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2".
So if later on you find a bug in my_code_seed42.m you're going to go back and modify the previous 41 files? That's inefficient. Instead write the code once as a function that accepts a seed value and call that function with the various seeds as input.
As for the randomness in a parfor, see this documentation page on repeating random numbers in a parfor loop and the page on controlling randomness to which its first paragraph links.

4 个评论
显示 2更早的评论 隐藏 2更早的评论

Daigo
Daigo 2022-4-13
编辑：Daigo 2022-4-13
Thanks for your advice. Yes, I will make a function that takes a seed as an input.
As for the randomness, I still couldn't figure out. The randomness in the parfor loop is not a problem. The 50 results from the parfor loops are different from each other, which is exactly what I want. My problem is, the jobs are using the same random number stream even though I'm giving the different seed values. I'd like to know how to fix that.
Edric Ellis
Edric Ellis 2022-4-13
In the example you show, you're calling rng(seed) on the client, this does not affect the workers operating on the body of the parfor loop. If you're using gpuArray in the body of the parfor loop, then you'll need a combination of these two bits of documentation:
Daigo
Daigo 2022-4-13
I appreciate a lot, Edric! I didn't fully understand what the client and workers are, but now it's all clear. I will take a look at the documentations.
Daigo
Daigo 2022-4-13
编辑：Daigo 2022-4-15
@Edric Ellis I ended up using parallel.gpu.RandStream.create to control the rng state for gpuArray inside of forloop. Here is a toy example:
function [r, time] = example_random_gpu(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(data(ii,:))/n;
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc;
end
function c = func_random(a, n, stream)
c = randn(stream, 1, n) + 1i*a*randn(stream)*gpuArray(linspace(-2,2,n));
end
The function example_random_gpu performs as exactly desired - the same r for the same val_seed, but a different r for a different val_seed.
However, I have another question regarding the data access. In this toy example, I have multiple parfor-loops accessing the same constant: data. Is there anyway to allow each worker to access the value of data more efficiently? I tried doing the following but the computation became much slower:
function [r, time] = example_random_gpu2(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
C = parallel.pool.Constant(data); % <----- ADDED
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(C.Value(ii,:))/n; % <---- Modified
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc; % ---> Computation became slower ...
end

请先登录，再进行评论。

类别

在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息

产品

版本

R2019a

标签

提问:

Daigo
Daigo
2022-4-12

编辑：

Daigo
Daigo
2022-4-15

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by