I don't quite understand you example, but for anovan, the expected input is a column vector of numbers (nx1) and a cell array of factors.
To create an input variable for multicolumn data, do the following:
X = data(:)
To create 3 factor variables, A, B and C, define each one separately. Each entry indicates the factor level for the corresponding value in X (0 or 1).
A = [0 0 1 1...];
B = [0 0 0 0 ...];
C = [0 0 0 0 ...];
If it helps, you can visualize it this way:
level | A | B | C | X
(1) | 0 | 0 | 0 | n1
(1) | 0 | 0 | 0 | n2
a | 1 | 0 | 0 | n3
a | 1 | 0 | 0 | n4
...
Finally, I can call the anovan function with the inputs I've created and specify that I want to use the full model (all possible interactions).
anovan(X(:),{A,B,C},'varnames',{'A','B','C'},"model","full")
