Error when using sptensor in parfor

2022 8 5
1 个回答
回答已采纳
更新时间：2022 8 9
13 次查看（30 天）

0 个投票

I am trying to edit a sparse tensor from the Tensor Toolbox inside a parfor loop. However I am getting the "Unable to classify the variable 'tau' in the body of the parfor-loop. I thought this might be a slicing issue, however when I tried testing using a 3D matrix with smaller dimensions the parfor loop runs. Unfortunately the needed tensor size is too big for me to use a native 3D matrix, and so I am trying to stay with a sparse tensor if possible. Any ideas how I can correct this? A simplified code is provided:
tau = sptensor([],[],[N N M]);
opts = parforOptions(gcp,'RangePartitionMethod','fixed','SubrangeSize',100);
parfor (i = 1:N-1, opts)
mytemp_p_k = sptensor([],[],[N M]);
mytemp_p_k(a,b) = <edit matrix>
tau(i,:,:) = mytemp_p_k;
end

1 个评论
显示 -1更早的评论 隐藏 -1更早的评论

Matt J
Matt J 2022-8-5
However I am getting the "Unable to classify the variable 'tau' in the body of the parfor-loop.
Do you mean a Code Analyzer warning? I, for one, get no such warning.

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

Matt J
Matt J 2022-8-5

0 个投票

You could try with ndSparse() instead of sptensor()
https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays
The following modified code ran fine for me.
[N,M]=deal(100);
tau = ndSparse.spalloc([N N M],N^2*M/100);
opts = parforOptions(gcp,'RangePartitionMethod','fixed','SubrangeSize',100);
parfor (i = 1:N-1, opts)
mytemp_p_k = sprand(N,M,0.01);
tau(i,:,:) = mytemp_p_k;
end

6 个评论
显示 4更早的评论 隐藏 4更早的评论

Ezra Altabet
Ezra Altabet 2022-8-5
Hi Matt,
Thank you for your help. With your code, when I try initializing tau with N = 1000189 and M = 343 (which are the values my code needs) I get the following error: "Sparse matrix sizes must be non-negative integers less than MAXSIZE as defined by COMPUTER." I didn't have this issue with sptensor, so I'm wondering if a settings thing I can change. Any ideas as to how I can fix this?
Matt J
Matt J 2022-8-5
编辑：Matt J 2022-8-9
The error should go away if you reshape your matrix as NxMxN and loop across the 3rd dimension instead of the first. Looping across the first dimension of tau is a less favorable memory access pattern anyway (probably for sptensor as well).
[N,M]=deal(1000189, 343);
tau = ndSparse.spalloc([N, M, N], 2e6);
opts = parforOptions(gcp,'RangePartitionMethod','fixed','SubrangeSize',100);
parfor (k = 1:N-1, opts)
mytemp_p_k =...;
tau(:,:,k) = mytemp_p_k;
end
I should also mention that is very inefficient to fill a sparse array cumulatively in a loop, as you are doing currently. This is true for any sparse type implementation. It would be much better if you used the loop to accumulate only the tabular I,J,K,S data needed to populate the array. Then at the end, you would build the actual sparse array:
[I,J,K,S]=deal(cell(N-1,1));
opts = parforOptions(gcp,'RangePartitionMethod','fixed','SubrangeSize',100);
parfor (k = 1:N-1, opts)
mytemp_p_k =...;
[I{k},J{k},S{k}]=find(mytemp_p_k);
K{k}=repmat(k,size(S{k}));
end
I=cell2mat(I(:));
J=cell2mat(J(:));
K=cell2mat(K(:));
S=cell2mat(S(:))
tau=ndSparse.build([I,J,K],S);
Ezra Altabet
Ezra Altabet 2022-8-9
Thank you for your help! I'm trying to implement your method and I appear to be getting some discrepencies in my data. I would think this would produce an NxNxM matrix but instead it outputs an NxMxN matrix. Further, as a test, when I saved mytemp_p_k into a cell after each iteration I got an NxM matrix. Normally this wouldn't be concerning, but ti looks like I'm getting different data then when using my test code that jut saves the value of each sparse matrix as a new cell. Any help would be appreciated. This is the code that I am using:
[I,J,K,S]=deal(cell(N-1,1));
mycell = {};
for i = 1:N
mycell{i} = 0; % preallocate number of cells
end
parfor (i = 1:N-1)
mytemp_p_k = sparse(N, M);
mytemp_p_k(a,b) = <edit matrix>
mycell{i} = mytemp_p_k;
[I{i},J{i},S{i}]=find(mytemp_p_k);
K{i}=repmat(i,size(S{i}));
end
I=cell2mat(I(:));
J=cell2mat(J(:));
K=cell2mat(K(:));
S=cell2mat(S(:));
tau=ndSparse.build([I,J,K],S);
Matt J
Matt J 2022-8-9
Correct, we want it to be an NxMxN matrix to avoid the MAXSIZE error you discovered above, and also because acessing tau(:,:,i) is more efficient than accessing tau(i,:,:) due to memory locality.
Ezra Altabet
Ezra Altabet 2022-8-9
Thank you. Just out of curiosity, is there a way that I could use just the tabular data format so that I can write to any element in the parfor loop instead of just a slice, or does that violate the limits of parfor? In my specific example, elements that need to be changed only get set to 1, if that simplifies the problem.

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

产品

版本

R2021a

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by