Cut mask out of image with certain pixel margin numpy opencv

Question

I am playing around with the Mask RCNN (https://github.com/matterport/Mask_RCNN) segmentation program that is trained on the COCO data set. It detects persons (along with many other objects that I further neglect) in an image and returns one or multiple Person masks, i.e. Boolean Numpy arrays containing True values for all pixels that are classified as a 'Person' and False values for all other pixels:

So an inputted image (uint8 array of shape (3900,2922,3)) becomes a mask (Boolean array of shape (3900,2922)) or multiple masks when multiple persons are detected in the picture.

Now I can use this mask to cut the person out of the image with some simply Numpy array indexing:

mask3d = np.dstack([mask]*3)
cut_out_mask = np.invert(mask3d)
res = np.where(cut_out_mask, 0, image)

This returns the following image:

Since the masks that are returned by the Mask_RCNN program are quite tight, I would like to add a margin of a few pixels (let's say 15px), so that I get something like this:

Which Numpy/ OpenCV functions can I leverage to cut out the mask from the original image (similar to np.where), adding a margin of 15 pixels around the mask?

Answer 1

One method to do this is to use cv2.dilate to increase the surface area of the mask. Depending on your mask shape, you can create varying structuring element shapes and sizes with cv2.getStructuringElement.

For instance, if your mask shape is rectangular you might want to use cv2.MORPH_RECT or if your mask shape is circular you can use cv2.MORPH_ELLIPSE. In addition, you can change the kernel size and the number of iterations to dilate. After dilating the mask, you can use cv2.bitwise_and to get your result. Here's a minimum reproducible example:

Original image

Mask

Dilate

Bitwise-and for result

import cv2

# Load image and mask
image = cv2.imread('1.png')
mask = cv2.imread('mask.png')

# Create structuring element, dilate and bitwise-and
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,25))
dilate = cv2.dilate(mask, kernel, iterations=3)
result = cv2.bitwise_and(image, dilate)

cv2.imshow('dilate', dilate)
cv2.imshow('result', result)
cv2.waitKey()