How to input the unknown size of time-step to TimeDistributed Keras layer

Question

I have my input state with shape = (84,84,4)

state = Input(shape=(84,84,4), dtype="float")

So I want to pass this to some TimeDistributed layer with time steps size=1..5 (in range of 1 to 5) and I don't know exactly which it equals.

My next layer is something like this:

conv1 = TimeDistributed(Convolution2D(16, 8, 8, subsample=(4, 4), border_mode='valid',
                                      activation='relu', dim_ordering='tf'))(state)

And I've got an error at this layer:

IndexError: tuple index out of range

I just want to pass an unknown time-series size to TimeDistributed and then to LSTM also.

Answer 1

So basically in Keras - you need to provide the sequence length because during computations Keras layers accepts as an input numpy array with a specified shape - what makes compulsory for all inputs (at least in one batch) to have a length fixed. But - you still can deal with varying input size by 0-padding (making all sequence equal size by adding all zero dummy timesteps at the beginning) and then masking what makes your network equivalent to a varying length input network.