Is there a limit to the number of elements in an array in Matlab

Question

Steve Francis 2021-1-26

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/726953-is-there-a-limit-to-the-number-of-elements-in-an-array-in-matlab

评论： Walter Roberson 2021-1-26

I have a binary file that comprises 16,814,160 bytes of 16-bit data in little endian format. I wish to open the file in Matlab and convert the 16-bit values to decimal in a 1 x 8407080 array. The first 8 bytes of the file in hex are: E0 0B 23 0C AD 0B 23 0C.

When I edit the file so that it comprises only the first 8 bytes, and then run the commands below, it returns [3040,3107,2989,3107] in the Workspace. This is as I would expect.

However, when I run the same program with all 16,814,160 bytes, the output is shown in the Workspace as a 1x4147200 double, of which the first four numbers are [56802,57570,56800,56801 and so on]. I would have expected the first four bytes to have remained unchanged!

Why is the array truncated to 4147200 elements and why have the decimal values been changed? The only difference is the input file size.

filename='my_binary_file(first_8_bytes).raw';
%filename='my_binary_file(all_bytes).raw' %uncomment this when reading in file size 16,814,160 bytes
fileID = fopen(filename, 'r', 'ieee-le'); % open the file for low level I/O function and obtain file identifier
if fileID == -1, error('Cannot open file: %s', filename); end
precision = 'uint16';
Data = fread(fileID, [1 8407080], precision); %reads data at byte level
fclose(fileID);

Thanks for your help.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jan 2021-1-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/726953-is-there-a-limit-to-the-number-of-elements-in-an-array-in-matlab#answer_606458

Check again, if you are really reading the file your expect you do. Use absolute path names to control this.

If the file starts with the bytes HEX(E0 0B 23 0C AD 0B 23 0C), importing it as UINT16 will reply [3040, 3107, 2989, 3107] in every case. If you get [56802,57570,56800,56801] and an unexpected number of elements, your are reading another file. Everything else would be pure magic, but Matlab works deterministically.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Steve Francis 2021-1-26

This solved my problem! In the end, I was making a very basic error :) Thank you for your help.

请先登录，再进行评论。

Answer 2

Steven Lord 2021-1-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/726953-is-there-a-limit-to-the-number-of-elements-in-an-array-in-matlab#answer_606368

If you want the data to be read in as uint16 and then returned as uint16 as well, use '*uint16' as the precision input in your call to fread. The asterisk is very important. See the description of the precision input on the documentation page for the fread function for more information about the difference.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Steve Francis 2021-1-26

Thanks for your response. I didn't appreciate the importance of the asterisk so that is valuable. Unfortunately, it didn't solve this particular problem although it may contribute.

请先登录，再进行评论。

Answer 3

Walter Roberson 2021-1-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/726953-is-there-a-limit-to-the-number-of-elements-in-an-array-in-matlab#answer_606418

The limit to the number of elements in an array in MATLAB is 2^48-1 which is roughly 2.8e14.

This is a design limit and Mathworks would need a Good Reason to increase it.

The limit was chosen because the Intel x64 architecture is designed with only 48 address bits. There are no publicly known implementations of x64 architecture that have more address lines.

The only publicly known implementation that approaches having that much memory is a server from HP, but you would need several of them tied together with unified address space in order to reach the limit. More of them than exist in the world, I believe.