How to detect defect bean using python3

Question

normal bean sample

defect bean sample-1

I want to judge whether the bean is normal or defect. I tried to use Canny method(to find edge) and so on... but I failed. I just solve it using shape.(cracked bean & unshaped bean ...) Please, give me some idea to solve it.

Sorry for my English, it's not my first language.

Answer 1

One of the most widely used methods to "fill cracks" in an image is dilate-erode. Simply put, you make your binary image "grow" at the edges, so the cracks get filled, then you reverse the process and make it "shrink" at the edges - but, as the cracks have been filled, there is no information about them left in the image, so they stay filled. Perhaps, you can use that and then look at the difference between your original image and the image after dilate-erode: if there were little to no cracks, there would be little to no difference, and if there were plenty of cracks, there would be plenty of difference.

For example. Let's convert our image to a binary black and white mask:

def get_th_binary_mask(img):
    gs = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    ret, thresh = cv2.threshold(gs, 0, 255, cv2.THRESH_BINARY)
    mask = np.zeros(thresh.shape, np.uint8)
    mask[thresh == 255] = 0
    mask[thresh == 0] = 1
    return mask

Now, just sum the elements of the matrix, which gives us the number of white pixels, dilate-erode, sum again, subtract the sums:

def get_de_difference(binary_image):
    s_before = np.sum(binary_image)
    kernel = np.ones((17, 17), np.uint8)
    d = cv2.dilate(binary_image, kernel, 1)
    d = cv2.erode(d, kernel, 1)
    s_after = np.sum(d)
    return abs(s_after - s_before)

For the "good" bean it gives 72 pixels different, for the "bad" one it gives 1158.

Can be further improved by using a more sophisticated thresholding function, for example, based on Otsu and Grab cut:

def get_gc_binary_mask(img):
    gs = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    ret, thresh = cv2.threshold(gs, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

    mask = np.zeros(thresh.shape, np.uint8)
    mask[thresh == 255] = cv2.GC_PR_BGD
    mask[thresh == 0] = cv2.GC_FGD

    bgdModel = np.zeros((1,65),np.float64)
    fgdModel = np.zeros((1,65),np.float64)

    cv2.grabCut(img, mask, (0, 0, 1365, 767), bgdModel, fgdModel, 5, cv2.GC_INIT_WITH_MASK)

    mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
    return mask2

Using it instead of the previous one gives only 1 pixel difference for the "good" bean (despite an unfortunate artifact - see below), while for the "bad" bean it gives 741. Or, if you can change the background of your pictures, just put some bright green/blue sheet there before taking your pictures and use chroma keying.

This is how it looks, left to right: original images (column 1), basic threshold, dilate, erode, otsu/grabcut threshold, dilate, erode. The difference between columns 2 and 4 and between columns 5 and 7 is what matters.