Since you are able to detect the field properly, by imposing the game field lines on to the image will help you to identify the possible location of the stones (all corners). Now in these corner neighborhoods, you can look for particular color to identify the presence of that colored stone or its absence.
If you already know how your game field should look like you can create a matrix with all zeros and can add a label for a particular colored ball in the corresponding location when it is detected.
