How can I find the 3D coordinates of a projected rectangle?

Question

I have the following problem which is mainly algorithmic.

• Let ABCD be a rectangle with known dimensions d1, d2 lying somewhere in space.
• The rectangle ABCD is projected on a plane P (forming in the general case a trapezium KLMN). I know the projection matrix H.
• I can also find the 2D coordinates of the trapezium edge points K,L,M,N.

The Question is the following :

• Given the Projection Matrix H, The coordinates of the edges on the trapezium and the knowledge that our object is a rectangle with specified geometry (dimensions d1, d2), could we calculate the 3D coordinates of the points A, B, C, D ?

I am grabbing images of simple rectangles with a single camera and i want to reconstruct the rectangles on space. I could grab more than one image and use triangulation but this is not desired.

The projection Matrix alone isn't enough since a ray is projected to the same point. The fact that the object has known dimensions, makes me believe that the problem is solvable and there are finite solutions.

If I figure out how this reconstruction can be made I know how to program it. So I am asking for an algorithmic/math answer.

Any ideas are welcome Thanks

Answer 1

You need to calculate the inverse of your projection matrix. (your matrix cannot be singular)

Answer 2

I'm going to give a fairly brief answer here, but I think you'll get my general drift. I'm assuming you have a 3x4 projection matrix (P), so you should be able to get the camera centre by finding the right null vector of P: call it C.

Once you have C, you'll be able to compute rays with the same direction as vectors CK,CL,CM and CN (i.e. the cross product of C and K,L,M or N, e.g. CxK)

Now all you have to do is compute 3 points (u1,u2,u3) which satisfies the following 6 constraints (arbitrarily assuming KL and KN are adjacent and ||KL|| >= ||KN|| if d1 >= d2):

1. u1 lies on CK, i.e. u1.CK = 0
2. u2 lies on CL
3. u3 lies on CN
4. ||u1-u2|| = d1
5. ||u1-u3|| = d2
6. (u1xu2).(u1xu3) = 0 (orthogonality)

where, A.B = dot product of vectors A and B ||A|| = euclidean norm of A AxB = cross product of A and B