label2rgb and rgb2label

Caution: if your values are in floating point format then you risk having problems with floating point round-off. Values in floating point format should seldom be compared for bit-for-bit equality.

Also, the above assumes that there is only one labelcolor, which is generally not the case. You would generally need to use

unique_rgb = uniquetol(reshape(rgblabel,[], 3));

and then work through all of the rows. Now, what you should consider to make it easier is

[label_colormap, label_idx] = uniquetol(reshape(rgblabel, [], 3));
misordered_label_matrix = reshape(label_idx, size(labelcolor,1), size(labelcolor,2));

this gives a color index for each pixel in the original image.

If you just want a pseudo-color image together with a colormap then label_colormap and misordered_label_matrix together will do the job. However, when the task is stated as reversing the effects of label2rgb then generally the person wants the colormap used to be the same as in the original image, and wants the index value to be the same as the original -- which the above does not do. You would have to match each row in label_colormap to the rows of the colormap that were used for label2rgb() to find the closest match (in some sense), and then you would have to map the index that was returned in misordered_label_matrix to the index in the original colormap.

All of this is done for you if you just use rgb2label() passing in the colormap that was used for the label2rgb() call.