What is the best way to add empty rows to an existing table, to pre-allocate for more data?

Question

Leon 2024-1-7

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2067371-what-is-the-best-way-to-add-empty-rows-to-an-existing-table-to-pre-allocate-for-more-data

评论： Matt J 2024-1-8

Let's say I have a table that has 100 rows of ten variables and now I know I am going to need another 100 rows (or perhaps 100,000) in this table. To avoid loads of copying, I should preallocate these rows before populating them. The variables can be of various different types. I tried doing the following without success:

T{101:200,:} = missing;

I also tried:

T(101:200,:) = cell(100, width(T));

"Conversion to char from cell is not possible."

I just want the equivalent of T.addemptyrows(100). Or, if Matlab had an emptyrow(T) function, I could do:

T(101:200,:) = emptyrow(T)

Here's a toy example to play with:

T = table(["One"; "Two"; "Three"], [1; 2; 3], ['the'; 'cat'; 'sat'], {{1 2 3}; {4 5 6}; {7 8 9}})
T{4:10,:} = missing

Error using {}

Unable to perform assignment because value of type 'missing' is not convertible to 'char'.

Caused by:

Error using char

Conversion to char from missing is not possible.

If I remove the char variable, the same happens with cell.

I considered that I could take the first line of the table and duplicate it 100 times, then overwrite it one line at a time, but this is very inefficient, because cells could contain large amounts of data when all I really want is an empty variable. Additionally, this could hide errors since cells could appear to be populated with viable data when they wouldn't have been, and also could pose serious privacy and security risks in some scenarios.

I could, presumably, write a function that loops through the variable types of each and every variable in the existing table, and then creates a new table of the approriate size and variable types, and then copies the variable names from the first table, and then joins the two tables together, but that is a mess. I also worry about clobbering existing meta data or column data (since several variables could be in one column) and other issues I'm not even aware of. Is there not a more elegant solution? This must be a common thing to need to do. Thanks!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2024-1-8

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2067371-what-is-the-best-way-to-add-empty-rows-to-an-existing-table-to-pre-allocate-for-more-data#answer_1384961

在 MATLAB Online 中打开

T = table(["One"; "Two"; "Three"], [1; 2; 3], ['the'; 'cat'; 'sat'], {{1 2 3}; {4 5 6}; {7 8 9}})
T = 3×4 table
     Var1      Var2    Var3       Var4   
    _______    ____    ____    __________

    "One"       1      the     {1×3 cell}
    "Two"       2      cat     {1×3 cell}
    "Three"     3      sat     {1×3 cell}
newlength=10;
T(newlength+1,:)=T(1,:);
T(end,:)=[]
T = 10×4 table
      Var1       Var2    Var3        Var4    
    _________    ____    ____    ____________

    "One"         1      the     {1×3 cell  }
    "Two"         2      cat     {1×3 cell  }
    "Three"       3      sat     {1×3 cell  }
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}

4 个评论
显示 2更早的评论隐藏 2更早的评论

Leon 2024-1-8

编辑：Leon 2024-1-8

在 MATLAB Online 中打开

This achieves what I was looking for and does so without suppressing a warning; I could use this for my code. Thanks. One obvious disadvantage is that if the first row had cells with a lot of data, the unnecessary copying of it to the last line (that is then removed) could be substantial. This might not be so relevant when planning to add hundreds of lines, but imagine a table of two rows where each row contains cells amounting to a few hundred megabytes each and you want to add just three more rows. Apart from avoiding the warning, I wonder if there is an advantage to copying the first line to generate the last line, compared to the way I found and mentioned under Voss's example, which was:

T{end+n,1} = 0 % n is number of rows to add

The 0 strangely enough works for char, cell, string and double at least. For char, it conveniently turns into a char array of spaces of the required length. Is there any class for which 0 wouldn't work?

One minor disadvantage of this 0 method is that if the first variable is a string, the last row will have it as "0" rather than <missing>, which of course wouldn't matter for overwriting. If <missing> were required, your idea of adding an extra line and deleting it could be used.

Leon 2024-1-8

"Copying cells doesn't result in any new memory consumption. It's all done with pointers."

That's great, I didn't realise that. So your solution is very good. Thanks!

Matt J 2024-1-8

You're welcome, but if it resolves your question, please Accept-click the answer.

请先登录，再进行评论。

Answer 2

Voss 2024-1-7

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2067371-what-is-the-best-way-to-add-empty-rows-to-an-existing-table-to-pre-allocate-for-more-data#answer_1384901

编辑：Voss 2024-1-7

在 MATLAB Online 中打开

T = table(["One"; "Two"; "Three"], [1; 2; 3], ['the'; 'cat'; 'sat'], {{1 2 3}; {4 5 6}; {7 8 9}})
T = 3×4 table
     Var1      Var2    Var3       Var4   
    _______    ____    ____    __________

    "One"       1      the     {1×3 cell}
    "Two"       2      cat     {1×3 cell}
    "Three"     3      sat     {1×3 cell}
T{end+100,1} = missing % add 100 rows
Warning: The assignment added rows to the table, but did not assign values to all of the table's existing variables. Those variables are extended with rows containing default values.
T = 103×4 table
      Var1       Var2    Var3        Var4    
    _________    ____    ____    ____________

    "One"         1      the     {1×3 cell  }
    "Two"         2      cat     {1×3 cell  }
    "Three"       3      sat     {1×3 cell  }
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}
    <missing>     0              {0×0 double}

8 个评论
显示 6更早的评论隐藏 6更早的评论

Voss 2024-1-8

在 MATLAB Online 中打开

"is there a special comment that can be added to that one line to suppress the warning?"

Not a comment, but a command. The warning function can be used to turn on and off individual warning messages by ID. To get the ID you can use the lastwarn function.

Demonstration: First, generate the warning in question:

T = table(["One"; "Two"; "Three"], [1; 2; 3], ['the'; 'cat'; 'sat'], {{1 2 3}; {4 5 6}; {7 8 9}});
T{end+100,1} = missing; % warning is issued
Warning: The assignment added rows to the table, but did not assign values to all of the table's existing variables. Those variables are extended with rows containing default values.

Get the warning ID using lastwarn:

[~,wid] = lastwarn()
wid = 'MATLAB:table:RowsAddedExistingVars'

Turn the warning off:

warning('off',wid)

Verify that it worked by trying to generate the same warning again:

T{end+100,1} = missing; % no warning this time

Now that you know the relevant warning ID, in your code you can turn off the warning before any are generated:

warning('off','MATLAB:table:RowsAddedExistingVars')

Leon 2024-1-8

编辑：Leon 2024-1-8

Thanks re. info about lastwarn and warning.

请先登录，再进行评论。

What is the best way to add empty rows to an existing table, to pre-allocate for more data?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

4 个评论
显示 2更早的评论隐藏 2更早的评论

更多回答（1 个）

8 个评论
显示 6更早的评论隐藏 6更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

What is the best way to add empty rows to an existing table, to pre-allocate for more data?

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

4 个评论 显示 2更早的评论隐藏 2更早的评论

更多回答（1 个）

8 个评论 显示 6更早的评论隐藏 6更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

4 个评论
显示 2更早的评论隐藏 2更早的评论

8 个评论
显示 6更早的评论隐藏 6更早的评论