清空筛选条件
清空筛选条件

Python-like generator for reading large CSV file?

9 次查看（过去 30 天）
roudan
roudan 2018-4-16
回答： roudan 2018-4-16
I have a large csv file, typically I just read everything into a array like this
num =csvread(filename,2,2)
if csv file is big, the num array will be big as well and sometime it blowout my memory.
In python, there is a way to read the file but without holding the data in the memory. the methods use generator object. For example,
Base case(w/o using generator object):
value=[len(x) for x in open('filename.tx')] this case will hold the data in the memory
another case(use generator object:
it=(len(x) for x in open('filename.tx'))
the it is a generator object and it only hold the memory size for the generator expression without holding memory for all the data, later, we just extract the data by using next(it).
I am wondering if Matlab has similar generator object as Python when reading csv file to save the memory? Thanks

请先登录，再回答此问题。

采纳的回答

Guillaume
Guillaume 2018-4-16

更多回答（1 个）

roudan
roudan 2018-4-16
Thank you, Guillaume. That is awesome! I appreciate it.

类别

在 Help Center 和 File Exchange 中查找有关 Standard File Formats 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by