Hi,
I am trying to read through and sort two large .txt files, around 300 mb at the largest.
Originally, for each line of code I read, I would re create the matrix lile this:
strarray.full = [strarray.full ; new_info]
strarray.newdate = [strarray.newdate ; new_info ]
This slowed down considerably once the files reached around 20 mb. I've seen that Pre Allocating matrices prevent MATLAB from having to re create the growing matrix each iteration. So now I have the following:
strarray.newdate = NaT(2000000,1);
strarray.full = strings(2000000,1);
where I have a counting varaible ' j ' that counts each time something should be added into the matrix.
strarray.full(j,1) = new_info;
strarray.newdate(j,1) = new_info;
When I did this, the code slowed down considerably, both starting off slower and slowing down faster as time progressed. After running a profiler, it says that nearly all the time is spent putting the info into the pre-allocated matrix.
I've got permission to attach the file, but I cant attach the .txt files directly so I have to strip it down here.
.txt Format 1:
Datetime2 ~ *string* ~ *string* ~ *string*
*string*
Datetime2 ~ *string* ~ *string* ~ *string*
*string*
*string*
*string*
*string*
Datetime2 ~ *string* ~ *string* ~ *string*
*string*
*string*
.txt Format 2:
datetime1 ~ *string* ~~~ *string* ~~~ *string* ~*~
datetime1 ~ *string* ~~~ *string* ~~~ *string* ~*~
datetime1 ~ *string* ~~~ *string* ~~~ *string* ~*~
Thanks.
