Setting transparency of markers with gscatter is not possible as of r2020a.
Workarounds
You could use scatter() which does support transparency. Here's one way to translate a gscatter syntax to scatter syntax using arrayfun.
% gscatter syntax
gscatter(x,y,g)
% scatter syntax
cla()
hold on % important
uniqueGroups = unique(group);
h = arrayfun(@(g)scatter(x(group==g), y(group==g), 'filled'), uniqueGroups);
set(h, 'MarkerFaceAlpha', 0.6) % set transparency level
% scatter syntax specifying color, marker, and size
colors = 'rb';
markers = '^v';
uniqueGroups = unique(group);
cla()
hold on % important
h = arrayfun(@(g)scatter(x(group==g), y(group==g), ...
100, colors(g), 'filled', 'Marker', markers(g)), uniqueGroups);
set(h, 'MarkerFaceAlpha', 0.4, 'MarkerEdgeColor', 'k', 'MarkerEdgeAlpha', .2) % set transparency level
Alternatively, you could use gscatter without filled markers.
h = gscatter(x,y,g,colors,'os',10);
set(h,'LineWidth',1)
Why transparency in gscatter doesn't work
The scatter function creates scatter objects. Scatter objects have MarkerFaceAlpha and MarkerEdgeAlpha properties that allow you to set transparency levels.
The gscatter function creates line objects. Line objects do not have these properties. Furthermore, an undocumented method of adding transparancy to some graphics objects by adding a 4th element (0:1) to the RGB color definition does not work with gscatter.
