text objects have different possible FontUnits. All of the possibilities except Normalized relate to physical display size, and when you zoom because the physical display does not change, the on-screen size stays the same. normalized is proportional to the physical size of the axes, but since that does not change when you zoom, the size of the text does not change.
None of the text size options are in data units.
In order to change the text size when zooming, you have two choices:
- use the zoom postactioncallback to change the font size; or
- make the plot into an image with the text "burned in" to the image, and then the apparent size would change as you zoom.