How to efficiently write complex structures to a file using memmapfile?

14 次查看（过去 30 天）
Raul Onrubia
Raul Onrubia 2023-4-11
编辑： Raul Onrubia 2023-4-18
I am writing a code that reads complex structures from files using memmapfile, then you can modify the data, and last you write the data again to disk with memmapfile. I am struggling with the last part.
As a example, I read the first part of a file using:
filename = 'original.bin';
dataFormat = {'uint32',1,"Source_GRIB_code";
'uint32',1,"Target_GRIB_code"};
Nrepeat = 100;
offset = 0;
m = memmapfile(filename, 'Format', dataFormat, 'offset', offset, 'Repeat', Nrepeat);
tdata = m.Data;
tdata is a 100×1 struct array with fields:
Source_GRIB_code
Target_GRIB_code
After I modify tdata, I want to write it to another file:
filename = 'modified.bin';
dataFormat = {'uint32',1,"Source_GRIB_code";
'uint32',1,"Target_GRIB_code"};
Nrepeat = 100;
offset = 0;
m2 = memmapfile(filename, 'Format', dataFormat, 'offset', offset, 'Repeat', Nrepeat, 'Writable', true);
I expected to be able to execute:
m2.Data = modified_tdata;
but I get:
Error using memmapfile/hParseStructDataSubsasgn
When the Data field of a memmapfile object is a structure, it may not be replaced by assignment.
Error in memmapfile/hDoDataSubsasgn (line 492)
[valid, s, newval] = hParseStructDataSubsasgn(obj, s, newval);
Error in indexing (line 749)
hDoDataSubsasgn(obj, s, newval);
I can not even do:
m2.Data(1) = modified_tdata(1);
I get the same error.
I also tried :
ttt = num2cell(uint32(vertcat(modified_tdata.Source_GRIB_code)));
[m2.Data.Source_GRIB_code] = ttt{:};
But I get:
Error using indexing
A subscripting operation on the Data field attempted to create a comma-separated list. The memmapfile class does not support the use of comma-separated lists
when subscripting.
Since it is going to be really slow, I want to avoid doing a for loop that does something like:
for ii=1:Nrepeat
m2.Data(ii).Source_GRIB_code = modified_tdata(ii).Source_GRIB_code;
m2.Data(ii).Target_GRIB_code = modified_tdata(ii).Target_GRIB_code;
end
Is there an alternative to the for loop?
Thank you

请先登录，再回答此问题。

采纳的回答

Raul Onrubia
Raul Onrubia 2023-4-18
I ended up using fwrite. I reordered the data and casted it to uint8, and then writting it to disk using fwrite. I also tried the same approach but using memmapfile to store the uint8 data instead fwrite, but it was slower.

更多回答（1 个）

Srija Kethiri
Srija Kethiri 2023-4-18
Hi Raul,
I understand that you want to write the modified data from one memory mapped file to another.
The reason for the following error is “Error using memmapfile/hParseStructDataSubsasgn When the Data field of a memmapfile object is a structure, it may not be replaced by assignment.”
After the creation of “memmapfile” object, it doesn’t support changing the value of its “Data” property.
The reason for the following error is “Error using indexing. A subscripting operation on the Data field attempted to create a comma-separated list. The memmapfile class does not support the use of comma-separated lists when subscripting.” is,
In “memmapfile” object accessing a nonscalar structure array as “m.Data.fieldname” is not supported. A nonscalar structure array can only be accessed as m.Data(index).fieldname.
The alternative way of writing the data into “memmapfile” is, you can modify the data in the “modified.bin” file and then memory map that file.
To know more about “memmapfile” function, refer to the following documentation:
https://in.mathworks.com/help/matlab/ref/memmapfile.html
Hope this helps!

类别

在 Help Center 和 File Exchange 中查找有关 Performance and Memory 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by