Hi
I understand that you want to create a smaller mask RCNN. The following workaround might be of good help to you.
You can use ‘removelayers’ to remove a particular layers and “addlayers” to add a layers.
Here is an example which you can refer:
net = resnet50; % resnet object model
lgraph = layerGraph(net); % architecture of a deep learning network
lgraph = removeLayers(lgraph, {'fc1000', 'prob', 'ClassificationLayer_predictions'}); % to remove a layer
feature_layer = 'activation_40_relu'; % feature extraction layer
lgraph = connectLayers(lgraph, 'res5c_relu', feature_layer); % connect all the layers
Refer the following MATLAB documentation page for more information about ‘removelayers’
Refer the following MATLAB documentation page for more information about ‘addlayers’
For more information also refer this MATLAB answer
Thanks