Transform pixelLabel from videoLabeler to readily usable input for training of a Mask R-CNN for instance segmentation

Question

Yi-Ping Hsueh 2021-3-22

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/780157-transform-pixellabel-from-videolabeler-to-readily-usable-input-for-training-of-a-mask-r-cnn-for-inst

There are two parts of my question. I am new to computer vision in MATLAB, so sorry for asking these questions...

Firstly, what is the most efficient way to bridge the gtruth exported from videoLabeler to a 1-by-4 cell array containing the RGB training image, bounding boxes, instance labels, and instance masks?

I used the assisted freehand tool in videoLabeler to label my objects. When I clicked on "export", the data was saved as gtruth, in which the ROIs were stored as png files under gtruth.LabelData. Since this gtruth supposedly contains all the information you need to convert into the valid input to train a Mask R-CNN, just that the format does not match, is there an example to do this efficiently and systematically? Thank you

Second, is it possible to label overlapping instances as pixelLabels instead of polygon?

In ref[1], it seems like overlapping objects can be labelled and the order can be specified (e.g. A on top of B). I tried it myself and found that overlapping instances of the same category is also possible (i.e. A1 on top of A2). However, when labelling as pixelLabels, it seems like overlapping instances are merged into one. In ref[2], the example labeled image shows that a pixel can only be labeled as one category and the overlapped parts of an object are omitted. Since my video always contains two instances of a same category and sometimes they overlap, I am thinking about instance segmentation using Mask R-CNN. Is it possible that I continue to label my objects using pixelLabel or should I use the polygon label instead? It is easier to label using the assisted freehand tool which generates pixel labels...

ref[1] https://www.mathworks.com/help/vision/ug/label-objects-using-polygons.html

ref[2] https://www.mathworks.com/help/vision/ug/label-pixels-for-semantic-segmentation.html#mw_97ed846f-7cfe-497e-9fe3-b0eaf5a0d284

Thank you so much!