Import CSV file to 2012b make Windows 8 Entire UI hang!

2 次查看（过去 30 天）
mingming wang
mingming wang 2012-10-16
I just try to import CSV file, which has 1054419 rows and 16 cols. However, during import, windows 8 has no response at all, the entire UI hang.
I try many times but fail. 2012b preformance is too bad.
I already try to use csvread(), however, it also return error.
Error using dlmread (line 139) Mismatch between file and format string. Trouble reading number from file (row 1u, field 3u) ==> x0000,0,0x0012,1,0,0,3,66,20,2012,6,21,10,44,4\n
Error in csvread (line 48) m=dlmread(filename, ',', r, c);
here is the sample data: 2nd and 4th cols data type is HEX:
18251,0x0000,0,0x0012,1,0,0,3,66,20,2012,6,21,10,44,4 18251,0x0000,10,0x0010,1,1,38,0,226,2432,2012,6,21,10,44,4 18251,0x0000,12,0x000e,0,0,0,20,28,84,2012,6,21,10,44,4 18251,0x0000,13,0x0000,0,1,85,0,91,5440,2012,6,21,10,44,4 18251,0x0000,15,0x000e,0,1,23,0,25,1472,2012,6,21,10,44,4 18251,0x0000,16,0x0000,0,1,44,0,51,2816,2012,6,21,10,44,4 18251,0x0000,18,0x000e,0,0,0,15,17,72,2012,6,21,10,44,4 18251,0x0000,23,0x000e,0,0,0,15,21,68,2012,6,21,10,44,4 18251,0x0000,23,0x000e,0,1,1,0,1,64,2012,6,21,10,44,4 18251,0x0000,25,0x000e,0,1,55,0,225,3520,2012,6,21,10,44,4 18251,0x0000,30,0x000e,0,1,121,0,244,7744,2012,6,21,10,44,4 18251,0x0000,36,0x000e,0,0,1,20,30,92,2012,6,21,10,44,4 18251,0x0000,37,0x0000,0,1,47,0,56,3008,2012,6,21,10,44,4 18251,0x0000,38,0x000e,0,1,78,0,104,4992,2012,6,21,10,44,4 18251,0x0000,42,0x0000,0,0,86,0,87,2680,2012,6,21,10,44,4 18251,0x0000,42,0x0000,0,1,747,0,1490,47608,2012,6,21,10,44,4 18251,0x0000,43,0x0000,0,0,0,15,55,64,2012,6,21,10,44,4
  2 个评论
Andreas Goser
Andreas Goser 2012-10-18
Only a comment as not necessarily related to your issue. Win 8 is not a supported OS for R2012b: http://www.mathworks.com/support/sysreq/current_release/
mingming wang
mingming wang 2012-10-18
However, Before I upgrade to 2012b, I use 2012a on windows 8 and import this csv with no issue. Though Matlab itself may has no response during import, it does not lead to the whole windows 8 UI hang, other programs run normally. For matlab, I always care performance first, since I only process large amount data. I also do not understand why Matlab use Jave to develop UI. Java == Slow.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（3 个）

mingming wang
mingming wang 2012-10-16
编辑：Walter Roberson 2012-10-18
I wish data import wizard could provide more powerful data convert rules. Not only replace Blanks with NaN, but also like Hex data convert to Decimal; replace specific string to other value...
I just try to import data by wizard, exclude 2nd and 4th Hex data cols only. Windows 8 UI still Hang. So this is still a performance issue.
Lisa Justin
Lisa Justin 2012-10-18
编辑：Lisa Justin 2012-10-18
you can try fopen, feof, fgets,regexprep,sscanf and fclose
or
fid=fopen(filename.csv')
a = fread(fid,1,'uint32','l');
fprintf('%X',a) b = hex2dec(a);
a=dec2hex(fread(fid))
  1 个评论
mingming wang
mingming wang 2012-10-18
I try your code, but meet error:
>> fid=fopen('csvnew.csv');
>> a = fread(fid,1,'uint32','l');
>> fprintf('%X',a)
35323831>> b = hex2dec(a);
Error using hex2dec (line 38)
Input string found with characters other than 0-9, a-f, or A-F.
I open that hex2dec.m file and see the source code:
% Check for out of range values
if any(any(~((h>='0' & h<='9') | (h>='A'&h<='F'))))
error(message('MATLAB:hex2dec:IllegalHexadecimal'));
end
You can see that if statement. It only judge 0-9, A-F, but not include a-f. However your error message show: a-f. I update my CSV file chars: a-f to upper case: A-F, then re-run the code. But I meet the same error :(

请先登录，再进行评论。

Walter Roberson
Walter Roberson 2012-10-18
fopen(), textscan(), fclose(). If you use textscan(), read the hex fields as strings and convert them after using sscanf() with a %i format item.
Alternately, fopen(), fscanf(), fclose(). Use %i for the format items: it will convert the 0x fields but leave the others as decimal.

类别

在 Help Center 和 File Exchange 中查找有关 Text Files 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by