Why doesn't parfeval(@splitapply) improve splitapply's performance?

10 次查看（过去 30 天）
Simon
Simon 2023-8-31
评论： Simon 2023-9-6
I want to readtable many html-files to extract tables. I wrote a function extract_sheet to do just that. I had used parfor to perform this task, and it runs decently fast. Then it occurs to me that those html-files can be grouped according to their foder and filename segments. So, I try splitapply(extract_sheet, input variables, groupNumber), and it works. Then I want to see if parfeval would improve the speed. I do something like parfeval(@splitapply, extract_sheet, input variables, groupNumber.)
For a small testing file list, both methods spend almost the same amount of elapsed time, around 27.5 +/- .1 seconds. My question is why parfeval doesn't improve the performance?

请先登录，再回答此问题。

采纳的回答

Matt J
Matt J 2023-8-31
编辑：Matt J 2023-8-31
It probably means that Matlab's internal parallellization already does what parfeval does.
  6 个评论
显示 4更早的评论
Simon
Simon 2023-9-1
@Sam Marshalik, the splitapply doc page does say it is one of the functions that support multi-thread and suggest the users to read parfeavl doc. That's all I found in the doc. The doc is too stingy with providing parallelism instruction materials for Matlab-users (not software developers).
Sam Marshalik
Sam Marshalik 2023-9-5
编辑：Sam Marshalik 2023-9-5
@Matt J: You bring up a good point that the doc page is lacking information on this topic. I put in an enhancement request to improve that. In the meantime, I would suggest to call our Technical Support - they can investigate this further and reach out to the relevant Dev team.

请先登录，再进行评论。

更多回答（1 个）

Matt J
Matt J 2023-9-5
编辑：Matt J 2023-9-5
If you're going to be using PCT functions anyway, I wonder if a parfor loop might do better than splitapply. I.e., instead of,
splitapply(func,X,G)
one might instead do,
I=splitapply(@(x){x}, 1:numel(G), G);
parfor j=1:numel(I)
results{j}=func( X(I{j}) );
end

类别

在 Help Center 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by