What are faster alternatives than fread for costumn import patterns?
8 次查看(过去 30 天)
显示 更早的评论
Hallo Everyone,
I have to import point cloud 'video' - data from a binary file. Each file contains an unknown amount of frames, while each frame contains a number of points that is provided in the 'header' section of the frame.
Currently, I'm using fread to import the data, but as you all know, it is rather slow in loops.
My code looks currently like this:
frame_index = 1;
while ~feof(file_id)
% -- begin body --
temp.frame_width = fread(file_id,1,'*int');
temp.frame_height = fread(file_id,1,'*int');
temp.nPoint = fread(file_id,1,'*int');
% -- begin point cloud --
if (temp.nPoint > 0)
for i = 1:temp.nPoint
temp.indexes(i) = fread(file_id,1,'*int');
temp.user_indexes(i) = fread(file_id,1,'*int8');
temp.points(i,1) = fread(file_id,1,'*int16');
temp.points(i,2) = fread(file_id,1,'*int16');
temp.points(i,3) = fread(file_id,1,'*int16');
end
else
temp.indexes = [];
temp.user_indexes = [];
temp.points = [];
end
% -- point cloud end --
% -- body end --
depthClouds(frame_index) = DepthPointCloud(temp.frame_width, temp.frame_height, ...
temp.nPoint, temp.indexes, temp.user_indexes, ...
temp.points);
frame_index = frame_index +1;
end
I'm happy for every suggestion that could help to advance the speed of the import. Thanks in Advance,
Karen
0 个评论
回答(2 个)
Charu
2025-3-27
Hello,
To improve the speed of importing point cloud data from a binary file, instead of reading each data point individually. You can read larger chunks of the data at once to reduces the overhead of the repeated function.
Additionally, you can utilize the MATLAB's ability to handle operations on entire arrays at once, rather than iterating through elements with loops.
Below is a code snippet which uses larger chunks of data at once and pre-allocates the arrays:-
if (temp.nPoint > 0)
% Preallocate arrays
temp.indexes = zeros(temp.nPoint, 1, 'int32');
temp.user_indexes = zeros(temp.nPoint, 1, 'int8');
temp.points = zeros(temp.nPoint, 3, 'int16');
% Batch read all data at once
temp.indexes = fread(file_id, temp.nPoint, '*int32');
temp.user_indexes = fread(file_id, temp.nPoint, '*int8');
temp.points = fread(file_id, [3, temp.nPoint], '*int16')';
end
For more information, refer to the following MathWorks Documentation:
“fread” function:
1 个评论
Walter Roberson
2025-3-27
This is incorrect.
The original byte stream after the header looks like
4 byte int, 1 byte int, 2 byte int, 2 byte int, 2 byte int
4 byte int, 1 byte int, 2 byte int, 2 byte int, 2 byte int
4 byte int, 1 byte int, 2 byte int, 2 byte int, 2 byte int
... N times
Whereas you have coded for
N by 4 byte int, N by 1 byte int, 3*N by 2 byte integer
To have a chance of working properly, you would need to ftell() the current location, frewind() to the saved location, fread() with the "skip" parameter set, frewind to the saved location, move forward 4 bytes, fread() with the "skip" parameter set, frewind to the saved location, move forward by 5 bytes, fread() with the "skip" parameter set, and so on. The three 2 byte int groups would have to be done seperately.
Walter Roberson
2025-3-27
After the reading of temp.nPoint,
bytes = fread(file_id, 11*nPoint, '*uint8');
indices_uint32 = uint32(bytes(1:11:end))*16^0 + uint32(bytes(2:11:end))*16^1 + uint32(bytes(3:11:end))*16^2 + uint32(bytes(4:11:end))*16^3;
indices = typecast(indices_uint32, 'int32');
user_indices = typecast(bytes(5:11:end), 'int8');
points(:,1) = typecast(uint16(bytes(6:11:end))*16^0 + uint16(bytes(7:11:end))*16^1, 'int16');
points(:,2) = typecast(uint16(bytes(8:11:end))*16^0 + uint16(bytes(9:11:end))*16^1, 'int16');
points(:,3) = typecast(uint16(bytes(10:11:end))*16^0 + uint16(bytes(11:11:end))*16^1, 'int16');
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!