How do I share data between my parallel pool workers?
显示 更早的评论
I am running a script in parallel using the Parallel Computing Toolbox. As part of this, I would like to save important data from one worker such that it is readable from another worker in the same job.
Thus far, I have tried to write files containing the data to my device's disk and have the other workers read those files as necessary. However, this often fails to work as expected. Is there a better way to do this?
采纳的回答
MathWorks Support Team
2024-3-20
编辑：MathWorks Support Team
2024-3-20
0 个投票
As of MATLAB R2022a, MATLAB parallel jobs automatically generate a "ValueStore" object. This object stores data values and associated keys out of system memory, providing easy access across workers in a parallel job or the opportunity to retrieve intermediate results.
This object provides many benefits regarding your desired workflow, including the following:
- Key-value pairs allow you to control exactly how your data is stored and accessed.
- The values in a "ValueStore" object are not stored in system memory, meaning they can store larger results.
- Compared to writing and reading to files, a "ValueStore" may increase execution speed and decrease the opportunity for errors to arise.
For more information on this property, as well as how to utilize it, please see the following documentation page:
Additionally, should you want to save files rather than data, I recommend looking into the "FileStore" object. This object mirrors the "ValueStore", but saves files rather than individual data values. For more information, see the following documentation:
Note that these properties may not apply to every workflow involving data management. For instance, in some cases, a "datastore" or "parallel.pool.Constant" may provided superior utility for any given workflow. To help determine the best method to manage your data when running in parallel, reference the following documentation:
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Third-Party Cluster Configuration 的更多信息
2024-3-12
2024-3-20
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
