Segmentation problem for tomato leaf images in PlantVillage Dataset

Question

I am trying to do segmentation of leaf images of tomato crops. I want to convert images like following image

to following image with black background

I have reference this code from Github

but it does not do well on this problem, It does something like this

Can anyone suggest me a way to do it ?

Answer 1

The image is separable using the HSV-colorspace. The background has little saturation, so thresholding the saturation removes the gray.

Result:

Code:

import numpy as np 
import cv2
# load image
image = cv2.imread('leaf.jpg')
# create hsv
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
 # set lower and upper color limits
low_val = (0,60,0)
high_val = (179,255,255)
# Threshold the HSV image 
mask = cv2.inRange(hsv, low_val,high_val)
# remove noise
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel=np.ones((8,8),dtype=np.uint8))
# apply mask to original image
result = cv2.bitwise_and(image, image,mask=mask)

#show image
cv2.imshow("Result", result)
cv2.imshow("Mask", mask)
cv2.imshow("Image", image)

cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 2

The problem with your image is the different coloration of the leaf. If you convert the image to grayscale, you will see the problem for the binarization algorithm:

Do you notice the very different brightness of the bottom half and the top half of the leaf? This gives you three mostly uniformly bright areas of the image: The actual background, the top-half leaf and the bottom-half leaf. That's not good for binarization.

However, your problem can be solved by separating your color image into it's respective channels. After separation, you will notice that in the blue channel the leaf looks very uniformly bright:

Which makes sense if we think about the colors we are talking about: Both green and yellow have very small amounts blue in it, if any. This makes it easy for us to binarize it. For the sake of a clearer image, i first applied smoothing

and then used the iso_data Threshold of ImageJ (you can however use any of the existing automatic thresholding methods available) to create a binary mask:

Because the algorithm has set the leaf to background (black), we have to invert it:

This mask can be further improved by applying binary "fill holes" algorithms:

This mask can be used to crop the original image to extract the leaf:

The quality of the result image could be further improved by eroding the mask a little bit.

For the sake of completeness: You do not have to smooth the image, to get a result. Here is the mask for the unsmoothed image:

To remove the noise, you first apply binary fill holes, then binary closing followed by binary erosion. This will give you:

as a mask.

This will lead to