This is not possible unless you either have external information about which direction is which in the image, or you have information inside the image that gives you information about the directions.
Remember to take into account that a vector in any one 3-space direction changes between east/west and north/south depending which side of the Earth it is originating from. And watch out for images taken from "below" your assumed coordinate system.
