Why doesn't parfeval(@splitapply) improve splitapply's performance?

5 次查看（过去 30 天）
Simon
Simon 2023-8-31
评论： Simon 2023-9-6
I want to readtable many html-files to extract tables. I wrote a function extract_sheet to do just that. I had used parfor to perform this task, and it runs decently fast. Then it occurs to me that those html-files can be grouped according to their foder and filename segments. So, I try splitapply(extract_sheet, input variables, groupNumber), and it works. Then I want to see if parfeval would improve the speed. I do something like parfeval(@splitapply, extract_sheet, input variables, groupNumber.)
For a small testing file list, both methods spend almost the same amount of elapsed time, around 27.5 +/- .1 seconds. My question is why parfeval doesn't improve the performance?

请先登录，再回答此问题。

采纳的回答

Matt J
Matt J 2023-8-31
编辑：Matt J 2023-8-31
It probably means that Matlab's internal parallellization already does what parfeval does.
  6 个评论
显示 4更早的评论
Simon
Simon 2023-9-1
@Sam Marshalik, the splitapply doc page does say it is one of the functions that support multi-thread and suggest the users to read parfeavl doc. That's all I found in the doc. The doc is too stingy with providing parallelism instruction materials for Matlab-users (not software developers).
Sam Marshalik
Sam Marshalik 2023-9-5
编辑：Sam Marshalik 2023-9-5
@Matt J: You bring up a good point that the doc page is lacking information on this topic. I put in an enhancement request to improve that. In the meantime, I would suggest to call our Technical Support - they can investigate this further and reach out to the relevant Dev team.

请先登录，再进行评论。

更多回答（1 个）

Matt J
Matt J 2023-9-5
编辑：Matt J 2023-9-5
If you're going to be using PCT functions anyway, I wonder if a parfor loop might do better than splitapply. I.e., instead of,
splitapply(func,X,G)
one might instead do,
I=splitapply(@(x){x}, 1:numel(G), G);
parfor j=1:numel(I)
results{j}=func( X(I{j}) );
end

类别

在 Help Center 和 File Exchange 中查找有关 Performance and Memory 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by