You can treat the matrix created as an indexed image and use ind2rgb after defining a custom colormap with 3 levels (one for each value). A small example is shown below for setting the value of 0 to red, 1 to green and 2 to blue.
num = randi(3,100)-1;%randi generates integers from 1-3.
map = [1 0 0
0 1 0
0 0 1];
rgbImage = ind2rgb(num+1, map);
imagesc(rgbImage)
