Think about it and you'll realize why. If they're equal, the difference is zero (or close to it) for all 3 color channels, and zero shows up as black.
Where they're not equal, you'll get the difference between the background and the black handwriting (which is near zero), so you'll get cyanish -- which is high green and blue signals, and low red signal. If you invert that, you'll get low green and blue signals, and high red signal so that's why it then shows up as red.
