What does eigen value of structure tensor matrix denote?

Question

It is known that good feature point across two images can be determined properly, if

the two eigen value of above matrix, are greater than 0. Can someone explain, what does it mean to have both eigen value greater than 0 and why the feature point is not good if either of them is approx. equal to 0.

Answer 1

Note that this matrix always has nonnegative eigenvalues. Basically this rule says that one should favor rapid change in all directions, that is corners are better features than edges or flat surfaces.

The biggest eigenvalue corresponds to the eigenvector pointing towards the direction of the most significant change in the image at the point u.

• If the two eigenvalues are small the image at point u does not change much.
• If one of the eigenvectors is large and the other is small this point might lie on an edge in the image but it will be difficult to figure out where exactly on that edge.
• If both are large, the point is like a corner.

There is a nice presentation with examples in the panoramic stitching slide deck from a course taught by Rajesh Rao at the University of Washington.

Here E(u,v) denotes the Eucledian distance between the two areas in the vicinities of pixels shifted by the vector (u,v) from each other. This distance tells how easy it is to distinguish the two pixels from one another.

Edit The matrix of image derivatives is denoted H in this illustration probably because of its relation to Harris corner detection algorithm.

Answer 2

That is related with the concept of Texturedness in the paper of Thomasi-Shi "Good features to track".

The idea of Textureness is to provide a rating of texture to make features (within a window) identifiable and unique. For instance, lines are not good features since are not unique (see Figure 3.9a)

To solve equation an optical flow equation, it must be possible to invert J (Hessian matrix). In practice next conditions must be satisfied:

1. Eigenvalues of J cannot differ by several orders of magnitude.
2. Eigenvalues of Hessian overcome image noise levels λnoise: implies that both eigenvalues of J must be large.

For the first condition we know that the greatest eigenvalue cannot be arbitrarily large because intensity variations in a window are bounded by the maximum allowable pixel value. Regarding to second condition, being λ1 and λ2 two eigenvalues of J, following situations may rise (See Figure 3.10):

• Two small eigenvalues λ1 and λ2: means a roughly constant intensity profile within a window (Pink region). Problem of figure 3.9-b. • A large and a small eigenvalue: means unidirectional texture patter (Violet or gray region). Problem of figure 3.9-a. • λ1 and λ2 are both large: can represent a corner, salt and pepper textures or any other pattern that can be tracked reliably (Green region).

Some references:

1 - ORTIZ CAYON, R. J. (2013). Online video stabilization for UAV. Motion estimation and compensation for unnamed aerial vehicles. 2 - Shi, J., & Tomasi, C. (1994, June). Good features to track. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR'94., 1994 IEEE Computer Society Conference on (pp. 593-600). IEEE. 3 - Richard Szeliski. Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vis., 2(1):1–104, January 2006.