Parse csv with complex numbers written by Python numpy

Question

LR 2024-7-16，23:07

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2137882-parse-csv-with-complex-numbers-written-by-python-numpy

评论： dpb 2024-7-17，0:06

I am not actually sure which Python features use this format (I think both numpy and pandas use it), or perhaps this is a part of a larger standard, but basically I have csvs with text that looks like

(-0.0053973628685668375-0.004476730131734169j),(0.005108157082444198-0.005597795916657765j),,,,,,,-298.0,-298.0,37293,-0.7617709422297042,0.7202575393833991,(0.001506298444580933-0.0035885955125266656j)

and I want to parse into a numeric array.

The real-valued scalar entries are easy (well I can do a str2double and it's not super fast but it's acceptable). The blanks are also not too bad because after a simple textscan with a comma delimeter I can find emptys and set to a desired value. But what the heck do I do with these rediculous complex number strings?

There's loopy solutions with regexp or finding the real and imag components but they are too slow when dealing with hundreds of thousands of entries. I could also do things like find entries containing a "j" and process them separately, but is there something better?

2 个评论
显示无隐藏无

Stephen23 2024-7-16，23:34

Please upload a sample data file by clicking the paperclip button.

Note that scipy has functions for saving and loading MAT files, so that might be a worthwhile alternative.

dpb 2024-7-17，0:06

在 MATLAB Online 中打开

The embarrassing problem with C i/o is it doesn't know anything about complex variables. Why Mathworks hasn't extended MATLAB to deal with them in 40 years is also beyond comprehension. For a scientific language purporting to be user friendly, this is and has always been a glaring oversight.

Ages ago I built a mex function that passed input off to a Fortran mex file with a format string and let the Fortran compiler i/o library that does know complex data types do the parsing. Unfortunately, I lost access to it when left last employer and they had already wiped the machine by the time I thought to ask about it and never got around to rebuilding it.

But

s='(-0.0053973628685668375-0.004476730131734169j),(0.005108157082444198-0.005597795916657765j)';
fmtC='(%f%fj)';
fmt=[fmtC ',' fmtC];
v=reshape(sscanf(s,fmt),[],2).'
v = 2x2
   -0.0054   -0.0045
    0.0051   -0.0056
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>
v=complex(v(:,1),v(:,2))
v = 
  -0.0054 - 0.0045i
   0.0051 - 0.0056i

For the full file, one has to go through the very painful with C process of building the format string to match the input record; tedious but doable if the file structure is regular.

Even the new(ish) readmatrix and friends with the import options won't let you set a variable to a complex type. Disgraceful imo. Back when I was still consulting I used to grouse regularly but joy has never ensued....which is why I wrote the mex routine.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Stephen23 2024-7-16，23:39

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2137882-parse-csv-with-complex-numbers-written-by-python-numpy#answer_1486407

编辑：Stephen23 2024-7-16，23:40

在 MATLAB Online 中打开

test.txt

Forget about fiddling about with strings. Define the parentheses as delimiters and import as numeric:

format short G
M = readmatrix('test.txt', 'Delimiter',{'(',',',')'}, 'ConsecutiveDelimitersRule','join',...
    'LeadingDelimitersRule','ignore', 'TrailingDelimitersRule','ignore')
M = 
  Columns 1 through 6

   -0.0053974 -  0.0044767i    0.0051082 -  0.0055978i         -298 +          0i         -298 +          0i        37293 +          0i     -0.76177 +          0i

  Columns 7 through 8

      0.72026 +          0i    0.0015063 -  0.0035886i

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

LR 2024-7-16，23:52

Great solution thanks! I didn't realize readmatrix could do things like auto join consecutive delimiters and handle converting text data like "0.005108157082444198-0.005597795916657765j".

请先登录，再进行评论。