generate more random numbers
显示 更早的评论
Hi everyone, when i try to generate more 100 million random sample, I memory issue with matlab.
Is there a way to generate more than 100 million samples?
8 个评论
JL
2019-8-20
dpb
2019-8-20
Whether they're random or not has no bearing on the size of array you can hold in memory -- or at least only very marginally based on size of code loaded. You'll have same problem if use zeros or nan or repelem.
If you don't need the full precision of double you could cast to single but you may run into issue that has to generate as double first...not sure if the optional input argument 'single' causes singles to be used internally or just casts the output. For RNG, one would presume it's throughout so you might almost double the size possible. But 10X or more likely ain't agonna happen.
James Tursa
2019-8-20
What are you doing with these random numbers? Do you really need all of them in memory at the same time?
You could process whatever algorithm you're running in chunks or you could use tall arrays. A 3rd idea would be to generate chunks of random vars, store them in a text file (or multiple text files if needed) and read them in as needed.
dpb
2019-8-20
"generate chunks of random vars, store them in a text file (or multiple text files if needed) and read them in as needed."
If doing this, don't use a text file, use stream (aka "binary") files.
JL
2019-8-20
Nice, I wasn't familiar with that method. Thanks, dpb!
dpb
2019-8-21
For importance sampling you would have to have the translation adjustment for the normalization from the importance-sampled distribution.
Unless it takes multiple samples to actually compute the results, there's nothing preventing doing the accumulation dynamically or from processing output disk file. Would, of course, require binning to a number of bins that is computable but even there the bins could theoretically be maintained in tall array that isn't all in memory at once.
回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Random Number Generation 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
