How do I compare the accuracy of a neural network classifier model to a random classifier?

2 次查看（过去 30 天）
Cai Chin
Cai Chin 2021-4-7
评论： Cai Chin 2021-4-19
Hi, I am attempting to show that the binary predictions made by my bidirectional LSTM neural network classifier are statistically significant, i.e. that the accuracy is more than that which may occur by chance using a 'random' classifier. I know that I can use McNemar's test to compare the accuracies achieved by 2 different models, but I am unsure as to which model to use as the 'random' or 'baseline' classifier.
Any suggestions would be greatly appreciated. Thanks in advance.

请先登录，再回答此问题。

回答（1 个）

Vineet Joshi
Vineet Joshi 2021-4-19
Hi
You can create a ‘random’ classifier by simply using the randsample function. This function can return a randomly sampled class out of provided n classes.
On the other hand, a simple non-random ‘baseline’ classifier can be K nearest neighbor based.
The exact baseline model to use will depend on your use case. For example if you want to show how a bidirectional LSTM neural network can capture the sequential information, you can use a vanilla neural network as a baseline classifier as it will make predictions without any sequence information
Hope this helps.
  1 个评论
Cai Chin
Cai Chin 2021-4-19
Hi Vineet, thanks very much for our suggestions. Regarding the vanilla (shallow) network you have linked, I see that the input is understandably different from that used in a biLSTM network. For instance, the following is used for training a biLSTM network for classifying ECG signals as normal or abnormal:
net201 = trainNetwork(XTrainSD,YTrain_w,layers,options);
Here, 'XTrainSD' represents the signal data which is a 1062 cell array containing 4x28 doubles, whilst 'YTrain_w' is the corresponding labels.
Whereas, the inputs for the shallow network are as follows:
trainedNet = train(net,X,T,Xi,Ai,EW)
I am struggling to understand how I may translate my inputs to fit these arguments, despite reading the page. Could you please suggest how I may achieve this? Thanks again.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by