Iwas able to have the same result using both syntaxes.
With the mock example above, even if the F-statistic is correct, the anova on the lme model gave no pValues given df2=0 using the 'DFMethod','satterthwaite' syntax.
I attached a larger mock example (DatasetTry2.mat; G:Genotype; M:MouseID; S:SliceID) and I compared the results from the anovan with the experimental design described above and the lme function as follows:
load DatasetTry2
nestedMatrix=[0 0 0; 1 0 0; 0 1 0]; % nested matrix for the anovan function
[p,tbl,stats,terms] = anovan(DatasetTry{:,4},{DatasetTry{:,1},DatasetTry{:,2}, DatasetTry{:,3}},'random', [2,3],'nested',nestedMatrix,'varnames',{'Genotype','MouseID','SliceID'});
lme=fitlme(DatasetTry,'Slope~G+(1|M)','FitMethod','REML'); % I used the Fit method REML(Restricted maximum likelihood estimation) considering that REML is the default in most cases because it provides unbiased estimates, while ML is only unbiased for large designs
statsMM2 = anova(lme,'DFMethod','satterthwaite'); % I used the Satterthwaite approximation since it can be used if the variances are either equal or different.
From the anovan syntax, the Genotype fixed factor had F=193.82 and a pVal=7.15e-0.8. I realized that it is not necessary for my experimental question, namely if the slope values are equal between genotypes (WT and APP in the dataset), to decleare the SliceID variable. Indeed, the follwoing code gave the same F and p values for the Genotype factor:
nestedMatrix=[0 0 ; 1 0];
[p,tbl,stats,terms] = anovan(DatasetTry{:,4},{DatasetTry{:,1},DatasetTry{:,2}},'random', [2],'nested',nestedMatrix,'varnames',{'Genotype','MouseID'}); % omitting the random SliceID variable
Thus, I did not use the random variable Slice (S) in the design of the linear mixed-effect model. I was thus left with Genotype (G: WT and APP as levels) as fixed factor and MouseID (M) as a random factor nested in G.
Accordingly, the lme syntax does not consider S and only consider my fixed variable Genotype (G) and the random variable MouseID (M). Without any Dummy variable, the estimate of the intercept is the mean of the APP group (127.58) and the estimate of G_WT is -112.17 (mean of WT - mean of APP), meaning that the mean of the WT group is 15.41. In other words, the model correctly assign the mean of the APP group to beta0 and the difference between the means to beta1.
The anova run on lme and stored in statsMM2 gives the same F and p vales as the anovan function. In addition, the use of the Satterthwaite approximation is necessary to have the proper df2, namely 10 (indeed, I have a balanced design with five mice for each genotype). Without it, the df2 is equal to the number of observation minues the number of groups of the fixed factors (n. slices-n.genotypes: 24-2=22); it follows that the pVal is much smaller (2.18e-12). In addition, the use of REML is necessary as well to have the same values between the two syntaxes.
If anyone has anything to add and/or point out some fallacy/errors of my reasoning, please post an answer as well
