hi, I used data with huge size, so I did preallocation A(147611,30977)=zeros But , I got this message: ??? Maximum variable size allowed by the program is exceeded. what i have to do in this case? thanks in advance

problem in preallocation

huda nawaf 2011-11-10

thanks

what I have to do in this case?

what are alternatives?

Titus Edelhofer 2011-11-10

Either use sparse matrices or process your data in chunks instead of one large matrix ... Other then this general advice would require some words where your huge data comes from and what you want to do with it ...

huda nawaf 2011-11-10

my data is from social networks .

look

I opened folder with 1.92GB :

f=fopen('D:\social net\dataset\flixster\flixster\ratings.txt','r+')

this what cause the problem.

I just want to arrange my data in arrays.

these data regarding users in one of social networks.

I'm have not processed it yet.

okay regarding chunks , do u mean I have to splite my data each set is processed alone?

if so, what is sparse matrix?

please if have any information about it provide me.

thanks

Titus Edelhofer 2011-11-10

Hi,

1.92GB is large, but still far away from 34. How did you get to the size 147611x30977? Are the numbers in the file only integers? If you read them from file, you don't need to preallocate (depending on how you read).

Titus

huda nawaf 2011-11-10

in file there are integers and floating, but currently I read just integer.

I used just 1.92GB.

look please, in fact I convert that file into array

the structure of my file:

user id(integer) movie id(integer) rating(float)

in my file the id of users is frequent, so I try to place each user in row and draw his movie id and his rating .

So, the size of each row is different , and because i used preallocate i have to specify the size. I know the no. of rows but I don't know the no. of columns. So I give the largest(30977).

I'm on right or not?

please help me, I have in the same point since some days.

thanks

the array (147611*30977)

Walter Roberson 2011-11-10

I do wonder whether the "social network"'s web site's Terms and Conditions allow this use of the data?

Anyhow, sounds to me like a good place to use cell arrays, or perhaps struct.

problem in preallocation

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

6 个评论
显示 4更早的评论隐藏 4更早的评论

标签

Community Treasure Hunt

problem in preallocation

0 个评论 显示 -2更早的评论 隐藏 -2更早的评论

回答（1 个）

6 个评论 显示 4更早的评论 隐藏 4更早的评论

标签

另请参阅

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

6 个评论
显示 4更早的评论隐藏 4更早的评论