Large For Loop Executions

2014 10 27
1 个回答
回答已采纳
更新时间：2014 10 28
7 次查看（30 天）

0 个投票

Hello Everybody,
I need to execute a for loop for a matrix (34 x 24) which will result in a matrix with 6e36 rows and 24 columns. Of course when I try to do that, matlab gives me the truncating error.
I know its very huge but I want to know if there is a way to execute that large "for loops"?? And if there is a faster way of course that would be great.
Thanks in advance,

6 个评论
显示 4更早的评论 隐藏 4更早的评论

Adam
Adam 2014-10-27
Whether or not there is a faster way probably depends entirely on what you are doing in the loop. Judging from the numbers you give I assume your output is 34^24 in size.
The problem would appear not to be the for loop (although this may be slow), but the memory usage. I would imagine you need to do the result in chunks and save each to file.
What does your for loop index look like? If it is 1:6e36 then certainly that is too large and you need to break it down, if it just just 1:34 then the problem is just with memory and not for loop truncation.
the cyclist
the cyclist 2014-10-27
I think it would be helpful to post a highly scaled-down version of your problem, ideally with executable code, and than indicate which dimensions you need to scale up.
Ayman Esmat
Ayman Esmat 2014-10-27
I understand this.. I will give u an example to what I'm trying to do exactly.
Lets say I have x =
1 4 7
2 5 8
3 6 9
I wrote a code that will place the values in a new matrix like this (where every probability for combination of values in each row is placed) 1 4 7 1 4 8 1 4 9 1 5 7 1 5 8
etc....
The result will be 1 4 7 1 4 8 1 4 9 1 5 7 1 5 8 1 5 9 1 6 7 1 6 8 1 6 9 2 4 7 2 4 8 2 4 9 2 5 7 2 5 8 2 5 9 2 6 7 2 6 8 2 6 9 3 4 7 3 4 8 3 4 9 3 5 7 3 5 8 3 5 9 3 6 7 3 6 8 3 6 9
I hope u get what I'm trying to do. What I want here is that instead of matrix being 3 x 3, i want it 34 x 24 to produce the 6e36 rows and 24 columns.
Hope I explained it well.
Ayman Esmat
Ayman Esmat 2014-10-27
I'm sorry, the result of the matrix should be something like this,
1 4 7
1 4 8
1 4 9
1 5 7
1 5 8
1 5 9
1 6 7
1 6 8
1 6 9
2 4 7
2 4 8
2 4 9
2 5 7
2 5 8
2 5 9
2 6 7
2 6 8
2 6 9
3 4 7
3 4 8
3 4 9
3 5 7
3 5 8
3 5 9
3 6 7
3 6 8
3 6 9
1 5 8
1 5 9
Adam
Adam 2014-10-27
Well, as Mohammad Abouali points out below, I really don't see how you can hope to do this on the data size you say - the disk space required is monolithic. There must be massive redundancy in what you are trying to do...to go from 24*36 to data sizes that are bigger than human comprehension cannot be a solution to whatever problem you are trying to solve.
Even if you had enough disk size to store this, no human nor algorithm could hope to make use of such a vast amount of data.
Ayman Esmat
Ayman Esmat 2014-10-27
Well like I said, for me its all about all the different combinations of these numbers. Further in my code, I'm going to decide which combination I will choose and which I will eliminate. But it seems that my approach to the problem was wrong. I need to rethink it.

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

Mohammad Abouali
Mohammad Abouali 2014-10-27

1 个投票

How do you want to store that in memory?
6e36*24 elements stored at 1B an element (uint8) requires more than 1.34e29GB of memory. I don't think you can even store that in a hard drive, let alone on memory.
Are you sure about your sizes?

10 个评论
显示 8更早的评论 隐藏 8更早的评论

Ayman Esmat
Ayman Esmat 2014-10-27
unfortunately I'm very sure about the size.
Ayman Esmat
Ayman Esmat 2014-10-27
I explained what my code is doing above. If u check it u will understand what I'm trying to do.
Mohammad Abouali
Mohammad Abouali 2014-10-27
编辑：Mohammad Abouali 2014-10-27
I suggest instead of producing all those combination and storing it somewhere (which by the way, you won't be able to store it with todays disk sizes even on the largest supercomputer available), try an alternative approach.
Write a function that gives the n-th combination. So, given your example (the 3x3 matrix) if I call myFun(A,1) it gives you 1 4 7 and if you type myFun(A,2) it gives you 1 4 8.
Then you can pool any combination that you want and do your further computation.
Ayman Esmat
Ayman Esmat 2014-10-27
I'm sorry but u can you further elaborate on this.. I don't understand ur example.
Mohammad Abouali
Mohammad Abouali 2014-10-27
Let's say there is a storage facility that can host that much of data, i.e. 6e36*24 at 1B per element, So let's say you have access to such storage facility. If your write speed is 1GB per second it would take you more than 4.2e21 years to finish writing it to a disk.
Ayman Esmat
Ayman Esmat 2014-10-27
I see. Well I don't have anything to do so I might as well get started. (Kidding)
Thanks for the help, I will try to figure out another way to approach my problem
Mohammad Abouali
Mohammad Abouali 2014-10-27
编辑：Mohammad Abouali 2014-10-27
So something like this"
A= [1 4 7; ...
2 5 8; ...
3 6 9];
% let's say you want the 8-th combination i.e. row 8 of the output you listed above or [1 6 8]
n=8
% now getting the n-th combination.
[k,j,i]=ind2sub(size(A,1)*ones(1,size(A,2)),n);
[A(i,1) A(j,2) A(k,3)]
in your case which you have 24 columns, instead of i,j,k you need i1,i2, i3, ... i24; and notice how the order is reversed.
Ayman Esmat
Ayman Esmat 2014-10-27
Thank you very much for this help.. I will try this and get back to you.
Thanks again
Mohammad Abouali
Mohammad Abouali 2014-10-27
you are welcome
Ayman Esmat
Ayman Esmat 2014-10-28
Dear Sir, I tried ur code and it worked properly. Thank u very much for ur help.
Although, I have a question. In my original code, I constructed the result matrix column by column. And then after the matrix is fully constructed. I start another loop checking the values of each row according to my conditions and omitting the rows that are not feasible to me. (Number of columns doesn't change) Now with ur code, the result could be constructed row by row which is very helpful to me; I can check each row the moment its given to me in the loop and I can decide whether its feasible or not. Matlab won't have to store all the values of the matrix like my old code did, now it will store only the feasible values I want. My question is, since I won't need to store all the row values in case of my huge matrix of 6e36 rows (the 24 columns can't be changed). I only need certain rows which are feasible to me. Can I put the condition of omitting the unwanted rows in ur code so it will store the feasible rows and not store all the possible rows and run the matlab for loop for n=1:6e36 . Can this be done or am I circling in the same dilemma again?
Thanks again for ur help..

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息

产品

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by