How to statistically test the accuracy of 3D models?

Question

I have built a 3D model from a 2D image. I want to know how much accurate my model is using some statistical test. I think there are many available methods to do this like correlation and mean squares as mentioned in this question, Is it possible to compare 3D images?.

I couldn't find a clear description of the available tests in other sites. I've found an implementation which compares 2D images using square means here, http://www.mathworks.com/matlabcentral/answers/81048-mse-mean-square-error. I'm not sure if this can be used to calculate the model accuracy. Also, I didn't find an explanation of how the tests work, i.e. what parameters are compared (color, intensity, etc.) ?

EDIT: For more clarity, the 3D model represents every pixel in the 2D image as a voxel which has a color associated with it. The purpose of this model is to reconstruct the different color regions found in the 2D image into the 3D representation. So, the number of pixels that has some color (they represent a region) is calculated from the 2D image. A similar number of voxels will be constructed in the 3D model and given the same color. What matters in this modeling problem is the following,

1- size of the regions (must be almost similar in the 2D image and the model).

2-Connectivity level of a region in the 2D image and its corresponding region constructed in the 3D image must be similar. By connectivity I mean to check if the region components are scattered through the image or they are connected forming one large connected region instead of many small scattered regions of the same color.

EDIT2: I think color correlogram is suitable. I have found a code that implements it, but it is not clear to me. Here is the code,

% Soumyabrata Dev
% E-mail: [email protected]
% http://www3.ntu.edu.sg/home2012/soumyabr001/
I= imread ('img.jpg');
correlogram_vector=[];
[Y,X]=size(rgb2gray(I));

% quantize image into 64 colors = 4x4x4, in RGB space
[img_no_dither, ~] = rgb2ind(I, 64, 'nodither');
% figure, imshow(img_no_dither, map);
%rgb = ind2rgb(img_no_dither, map); % rgb = double(rgb)
distance_vector= [1 3]; 
[~,d]=size(distance_vector);
count_matrix=zeros(64,d);   total_matrix=zeros(64,d);
prob_dist=cell(1,d);

for serial_no=1:1:d
    for x=1:X
        for y=1:Y
            color=img_no_dither(y,x);

            % At the given distance 
            [positive_count,total_count]=get_n(distance_vector(serial_no),x,y,color,img_no_dither,X,Y);
            count_matrix(color+1,serial_no)=count_matrix(color+1,serial_no)+positive_count;
            total_matrix(color+1,serial_no)=total_matrix(color+1,serial_no)+total_count;       
        end
    end

    prob_dist{serial_no}=count_matrix(:,serial_no)./(1+total_matrix(:,serial_no));

end

for serial_no=1:d
    correlogram_vector=cat(1,correlogram_vector,prob_dist{serial_no});
end

end

This is the method get_n,

function [positive_count,total_count]=get_n(n,x,y,color,img_no_dither,X,Y)
% This function is useful to get the validity map of the neighborhood case.
% It can handle any number of neighborhood distances.

% Input
% n=The order of the neighborhood
% x & y= x y co-ordinates of the given pixel
% color= particular quantized color
% img_no_dither= The color quantized image matrix
% X & Y= The original dimensions of the input image

% Output
% positive_count= The number of occurences which have the same color
% total_count= The total number of valid cases for this particular instant


    valid_vector8n=zeros(1,8*n); % This is because of the propoerty of inf-norm. Each distance has 8 times the order
    positive_count=0;   total_count=0;

    nbrs_x=zeros(1,8*n);    nbrs_y=zeros(1,8*n);

    % The counting of the pixels is done in the following manner: From the
    % given pixel, go left-->up-->right-->down-->left-->up
    % Y co-ordinates of nbrs
    nbrs_y(1)=y;
    d=1;
    for k=2:1+n
       nbrs_y(k)=y-d;
       d=d+1;
    end

    nbrs_y(1+n:1:3*n+1)=y-n;

    d=0;
    for k=3*n+1:5*n+1
       nbrs_y(k)=y-n+d;
       d=d+1;
    end

    nbrs_y(5*n+1:1:7*n+1)=y+n;

    d=0;
    for k=7*n+1:1:7*n+1+(n-1)
       nbrs_y(k)=y+n-d;
       d=d+1;
    end

    % X co-ordinates of nbrs
    nbrs_x(1)=x-n;

    nbrs_x(2:1:1+n)=x-n;

    d=0;
    for k=1+n:1:3*n+1
        nbrs_x(k)=x-n+d;
        d=d+1;
    end

    nbrs_x(3*n+1:5*n+1)=x+n;

    d=0;
    for k=5*n+1:7*n+1
        nbrs_x(k)=x+n-d;
        d=d+1;
    end

    nbrs_x(7*n+1:7*n+1+(n-1))=x-n;

    % Assigning the validity of the neighborhood
    for i=1:8*n

        if nbrs_x(i)>0 && nbrs_x(i)<=X && nbrs_y(i)>0 && nbrs_y(i)<=Y
            valid_vector8n(i)=1;

        else
            valid_vector8n(i)=0;

        end

    end


    % Couting the number of common colors in the valid areas of the
    % neighborhood.
    for j=1:8*n
       if valid_vector8n(j)==1
          data= img_no_dither(nbrs_y(j),nbrs_x(j));
          if (data==color)
              positive_count=positive_count+1;
          end
          total_count=total_count+1;
       end
    end

end

Can anyone please clarify how this code works?

The code above is for autocorrelogram not correlogram. I've read that the first is better, but it can only calculate the spatial probability using pixels of the same colors (can't be applied on pairs of pixels which have different colors). Is this right?

Thank You.

Answer 1

TLDR: Classical workflow:

1. find matching features in both models,
2. calculate the distance,
3. ??????,
4. PROFIT!!