I want to solve a linear system in MATLAB whose size is (263169x263169). The major obstacle I am facing is creating the matrix of this size itself. I am working in a cluster with around 10TB soft quota. When I try to create this matrix, I get the following error:
n = 128;
C = zeros((2*n+1)^2);
x = C\b;
"Error using zeros
Requested 263169x263169 (516.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See <a href="matlab: helpview([docroot '/matlab/helptargets.map'], 'matlab_env_workspace_prefs')">array size limit</a> or preference panel for more information."
The problem is that no single node I am allocated can handle 516.0 GB memory alone. So I need to distribute the data somehow between different nodes available to me. The "distribute" function in MATLAB takes already created matrix or a data containing csv file and distributes it to different nodes. However, I do not have the data in a file as I can only fill the elements in the matrix after running the MATLAB code. The only idea I have for now is to save the values of 'C' matrix directly into a csv file as I run the required code and later distribute the csv file using 'distribute' command for solving the linear system in MATLAB. But this process seems to take a lot more time than I expected.
Are there any ways in MATLAB to create a matrix which can be distributed between different nodes during the matrix creation itself? Further, can the entire problem be solved more efficiently in MATLAB with other functions?
Thank you.
