Classification of a sequence of images (fixed number)

Question

I successfully trained a CNN for a single image classification, using pre-trained resnet50 from tensorflow_hub.

Now my goal is to give as input to my network a chronological sequence of images (not a video), to classify the behavior of the subject. Each sequence consists of 20 images taken every 100ms.

What is the best kind of NN? Where can I find documentation/examples for problems similar to mine?

Answer 1

Any time there is sequential data some type of Recurrent Neural Network is a great candidate (usually in the form of an LSTM).

Your model may look like a combination of an CNN-LSTM because your pictures have some sort of sequential relationship.

Here is a link to some examples and tutorials. He will set up a CNN in his example but you could probably rig your architecture to use the resNet you have already made. Though your are not dealing with a video your problem shares the same domain.

Here is a paper than uses a NN architecture like the one described above you might find useful.