Regression Learner: Importing batches from a database as input data
1 次查看(过去 30 天)
显示 更早的评论
My problem is that I have a large sqlite database and I would prefer not to import all the rows in memory at once and instead feed the data in batches.
I was experimenting with the fetch() method, but the Regression Learner only accepts tables from workspace and again, I would prefer if I dont have to import the full database into a table in workspace.
Is there a way to achieve this in Matlab?
0 个评论
回答(1 个)
Ishaan Mehta
2022-6-27
Hi Dominik
As I understand, your problem can be solved by importing the table's rows in batches, instead of fetching the entire table at once. This can be done using the LIMIT clause in SQLite.
LIMIT clause accepts two parameters, the number of rows to fetch and the offset i.e. number of rows to skip from the top of your table.
Here is a code snippet to fetch data in batches using a for loop:
for i = 1:5 % set upper limit according to number of rows in your table.
rowsToFetch = 10; % fetch 5 rows at once
offset = (i-1) * rowsToFetch;
sqlquery = sprintf('SELECT * from tablename LIMIT %d,%d', offset, rowsToFetch);
partialResults = fetch(conn, sqlquery)
end
Hope it helps
Ishaan Mehta
0 个评论
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!