Looks like you found a bug. The comma option in
'CellLabelFormat', '%,g'
is not available in sprintf() or fprintf() and it doesn't function in heatmap(). Maybe it's a sign of features to come (which has been requested a lot in the forum).
Even when you try to change the formatting after plotting, you get an error.
h = heatmap(...)
h.CellLabelFormat = '%,4.4g'
Warning: Error updating HeatmapChart.
Invalid format.
The bug has been reported.
In the mean time, you could use %.f if you want to get rid of exponential notation.