Using Computer vision in R to find rectangles in noisy data

Question

I am trying to automatically detect rectangles in a noisy image matrix. The values are binary, there are an arbitrary number of rectangles, the edges can be a bit fuzzy due to noise. I would like to end up with the coordinates of the rectangles or some other such method allowing me to focus on those areas.

I have been trying to use the Hough transform from the PET package,but don't understand how to interpret the output. I am open to other packages or techniques.

library(PET)

AddRectangle<- function(df, startx, starty, cols, rows){
    RectangleVector <- rep(1:cols, times = rows)
    #Start Point
    RectangleVector <- ncol(df)*(starty-1) + (startx-1) + RectangleVector
    #Row changes
    RectangleVector<- RectangleVector+ rep((0:(rows-1))*ncol(df), each = cols)
  RectangleVector
}

#create matrix, this is an arbitrary example.
dfrows <- 100
dfcols <- 100
df <- matrix(0, nrow = dfrows, ncol = dfcols)

#put in rectangles
df[AddRectangle(df, startx = 3, starty = 3, cols = 10, rows =10)] <-1
df[AddRectangle(df, startx = 40, starty = 50, cols = 30, rows =20)] <-1
df[AddRectangle(df, startx = 45, starty = 10, cols = 30, rows =35)] <-1
df[AddRectangle(df, startx = 80, starty = 80, cols = 10, rows =10)] <-1

image(df,col = grey(seq(0, 1, length = 2)))

set.seed(2017)
#add in noise
noisydf<- df
#Positives
noisydf[runif(dfrows*dfcols)>0.97]<-1
#negatives
noisydf[runif(dfrows*dfcols)>0.97]<-0

#Visualise
image(noisydf,col = grey(seq(0, 1, length = 2)))

This code creates the following image.

Using the pet package I get a very nice picture but don't know what to do with it. What I would like to get our are something like the coordinates of the rectangles. I am not tied to the Hough transform and any successful method is fine.

test <- hough(noisydf)

image(test$hData)

Answer 1

As no-one seems to be answering you, maybe you are open to an alternative solution using ImageMagick which is installed on most Linux distros and is available for macOS and Windows - for free.

So, I am using "Connected Component Analysis" just at the command-line to identify blobs or lumps of connected components with a minimum area. I am specifying 4-connected shapes (rectangles) rather than 8-connected shapes - which allows white pixels at North-East, South-East, South-West and North-West positions to be considered neighbours:

convert rectangles.png -threshold 50%               \
   -define connected-components:verbose=true        \
   -define connected-components:area-threshold=1000 \
   -connected-components 4 -auto-level z.png

Output

Objects (id: bounding-box centroid area mean-color):
  0: 613x460+0+0 287.2,239.0 140894 srgb(255,255,255)
  3: 504x294+69+78 318.1,213.1 115208 srgb(0,0,0)
  168: 153x102+291+237 369.5,286.5 14800 srgb(255,255,255)
  100: 154x59+266+164 344.6,194.5 8334 srgb(255,255,255)
  265: 54x28+84+328 112.5,341.7 1372 srgb(255,255,255)
  41: 49x31+469+110 492.8,123.5 1372 srgb(255,255,255)

Look at the column titles in the first line of output and hopefully you can see it has identified 6 rectangles - the last 4 are white, i.e. rgb(255,255,255) and correspond to your rectangles. Let's look at the one starting 168:.... It is 153 pixels wide by 102 pixels tall and starts 291 pixels from the left and 237 pixels from the top of your image. So, let's colour in those last four:

convert rectangles.png -fill "rgba(255,0,0,0.5)" -draw "rectangle 291,237 444,339" \
   -fill "rgba(0,0,255,0.5)" -draw "rectangle 266,164 420,224" \
   -fill "rgba(255,0,255,0.5" -draw "rectangle 84,328 138,356" \
   -fill "rgba(0,255,255,0.5)" -draw "rectangle 469,110 518,141" result.png

You can do the same sort of thing with OpenCV using cv::connectedComponents.