It's not time efficent, and it can lead to many errors (fake positives), but in your situation, I would track any detected object coordinates (X,Y) in each frame. If two couples of subsequent coordinates are very near each other, I would not count it (probably an object just moved a bit), otherwise yes (probably a new object). As already written, if an object exit the screen and then it come again it, it's gonna be counted again.
Let me know if it's applicable to your application.
Jacopo
