parfor with amazon-s3

2 次查看（过去 30 天）
Gideon Kogan
Gideon Kogan 2019-12-24
评论： Gideon Kogan 2019-12-24
I am trying perform parllel computation with amazon-s3.
As long as I understand, the only way to load data from amazon-s3 is using the fileDatastore object.
The fileDatastore is also kind of iterator wich does not work with parfor and requires the use of partition.
Now, instead of iteration over files I have to iterate over pertitietions and then use each pertitition as iterator (two loops).
Adding additional loop has two sideefects:
  • changing the flow which coplicates the adaptation of any code iterating over local files to files loacted in the cloud.
  • creating clumsy code.
When I used python, there was no big diference wherever the iteration is over a list of loaclly stored files or a list of files stored in the cloud. Is there any similar solution in MATLAB?
  2 个评论
Kojiro Saito
Kojiro Saito 2019-12-24
What is your file type stored in S3? Shpreadsheat (.xlsx), CSV, or any binary data?
And what MATLAB version are you using?
Gideon Kogan
Gideon Kogan 2019-12-24
I am using mat-files;
version
ans =
'9.6.0.1174912 (R2019a) Update 5'

请先登录，再进行评论。

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Licensing on Cloud Platforms 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by