Find 3D object location with known camera positions and matrices

Question

Here is the problem: I have two cameras looking at the same 3D object. I have the pixel co-ordinates of the object center in the two camera images. I know the camera location and pose in the 3D world. The camera matrix of both the cameras are calibrated.

With all this information, how do I find the object's 3D world location co-ordinates?

Challenges: I don't have the sensor information (sensor height in mm) which helps in getting a more accurate estimate of the location. The objects are tiny and located far away from the camera positions, which makes the localization estimate unreliable. Intuitively, using multiple camera (looking at same object) information should help, but in this case there are multiple instances of the object nearby. So this adds to the problem. Using SfM doesn't work either as the feature information is sparse for small pixel areas.

Answer 1

If you call P1 and P2 the projection of the center of your object for camera 1 and 2, you have to convert your 2 points from Image space, to World Space (you can do this thanks to the projection matrix, or doing a little bit of Thales geometry with the camera parameters).

Then the position of your object in the world is simply the intersection of the lines (C1 P1) and (C2 P2) where C1 and C2 are the world positions in world space of your two cameras.

In an ideal world, this point exists and is unique.

Hope that helps.