Is it possible to use Image Preprocessing with image sequences?

Question

I'm trying to use Keras' builtin image preprocessing capabilities to augment images in a sequence. My dataset has the shape (13200, 4, 168, 168, 1) with 13200 sequences, each consisting of 4 168x168px grayscale images.

When trying to run datagen.flow() on my dataset I get:

ValueError: ('Input data in `NumpyArrayIterator` should have rank 4. You passed an array with shape', (13200, 4, 168, 168, 1))

I'm assuming ImageDataGenerator is not able to handle my sequences of 4 images per sample correctly. Is there any way to do this?

Answer 1

Try to define a new generator by:

def sequence_image_generator(x, y, batch_size, generator, seq_len=4):
    new_y = numpy.repeat(y, seq_len, axis = 0)
    helper_flow = generator.flow(x.reshape((x.shape[0] * seq_len,
                                            x.shape[2],
                                            x.shape[3],
                                            x.shape[4])),
                                 new_y,
                                 batch_size=seq_len * batch_size)
    for x_temp, y_temp in helper_flow:
        yield x_temp.reshape((x_temp.shape[0] / seq_len, 
                              seq_len, 
                              x.shape[2],
                              x.shape[3],
                              x.shape[4])), y_temp[::seq_len,:]