Can you use OpenCV solvePNP with a equirectangular image?

Question

Is it possible to use OpenCV's solvePNP with an equirectangular image? I have an equirectangular image and I have four points in this image (red dots) and their pixel coordinates, and then I have 4 corresponding world points e.g. [(0, 0, 0), (2, 0, 0), (2, 10, 0), (0, 10, 0)] how can I estimate the camera pose?

I tried using OpenCVs solvePnp but that expect intrinsics for a Brown camera model so didn't work. Can this be done for a spherical camera?

Answer 1

At first, you should compute the intrinsic parameters of the camera, that is the focal length, the optical center and the distortion parameters, so afterwards you can compute the pose of your camera. Since your image seems to be somehow wide lens, you should use cv.calibrateCamera() as described here. You should enable the CV_CALIB_RATIONAL_MODEL flag at the cv.calibrateCamera() method, which computes for distortion parameters since your image has big distortions.

Now the tricky part In order to obtain high-quality undistorted images, after you have calibrated your camera, you should use a chess pattern that covers your entire image, in order for the distortion parameters to be computed accurately, since the biggest radial distortions reside on the edge of the image. Also, in order to compute accurately the solvePnp parameters, you should pick your red dots all over your image.