Randomness is lost in parfor loop on GPU?

1 次查看（过去 30 天）
Daigo
Daigo 2022-4-12
编辑： Daigo 2022-4-15
I have a code with a following structure:
% my_code_seed#.m
myoutputs1 = my_function1(myinputs1);
seed = #;
rng(seed);
parfor iter = 1:50
myouputs2 = my_gpu_function2(myoutputs1, my_function_of_Gaussian);
result(iter).name = myoutputs2;
end
save(file_name, 'result');
What I want to do is to model some random behavior of something by using this code repeatedly. What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2". I run these codes in a parallel way on a GPU cluster to save the computation time. However, it seems like the results from the codes are very similar, even though I'm giving different values of seed.
Do you have an idea why rng(seed); is not working properly in this case? I appreciate any help.

请先登录，再回答此问题。

回答（1 个）

Steven Lord
Steven Lord 2022-4-13
What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2".
So if later on you find a bug in my_code_seed42.m you're going to go back and modify the previous 41 files? That's inefficient. Instead write the code once as a function that accepts a seed value and call that function with the various seeds as input.
As for the randomness in a parfor, see this documentation page on repeating random numbers in a parfor loop and the page on controlling randomness to which its first paragraph links.
  4 个评论
显示 2更早的评论
Daigo
Daigo 2022-4-13
I appreciate a lot, Edric! I didn't fully understand what the client and workers are, but now it's all clear. I will take a look at the documentations.
Daigo
Daigo 2022-4-13
编辑：Daigo 2022-4-15
@Edric Ellis I ended up using parallel.gpu.RandStream.create to control the rng state for gpuArray inside of forloop. Here is a toy example:
function [r, time] = example_random_gpu(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(data(ii,:))/n;
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc;
end
function c = func_random(a, n, stream)
c = randn(stream, 1, n) + 1i*a*randn(stream)*gpuArray(linspace(-2,2,n));
end
The function example_random_gpu performs as exactly desired - the same r for the same val_seed, but a different r for a different val_seed.
However, I have another question regarding the data access. In this toy example, I have multiple parfor-loops accessing the same constant: data. Is there anyway to allow each worker to access the value of data more efficiently? I tried doing the following but the computation became much slower:
function [r, time] = example_random_gpu2(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
C = parallel.pool.Constant(data); % <----- ADDED
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(C.Value(ii,:))/n; % <---- Modified
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc; % ---> Computation became slower ...
end

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Parallel Computing Fundamentals 的更多信息

标签

产品


版本

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by