What happens in Keras when the dataset size is not a multiple of the batch size?

Question

What happens in Keras when the dataset size is not a multiple of the batch size? Does it mean that the last pass of each epoch is going to have smaller batch? Where is the code location to corroborate what happens?

The question is different from this suggested post as the latter one answers to the fact that you can train when the epoch is not a multiple of the batch size, but not to what happens and where is the code so one can get a deeper understanding.

Answer 1

The answer to this question, is easy and quite intuitive in fact. The solution is that the last batch is smaller. I will give an example to illustrate this idea: Let's say that the you have a dataset having 100 samples, and you use a batch size equal to 16. Then you have 6 batches of size 16, which means that you will consume 16*6 = 96 samples, and you have 4 remaining samples, that you didn't see yet in this epoch, so you just create a batch of 4 samples as last batch

Answer 2

First what you refer to is called called the dataset size not epoch size. There are two ways to handle remainder when the dataset size is not divisible by batch size.

• Creating a smaller batch of data (This is the best option most of the time)
• Dropping the remainder of the data (When you need to fix the batch dimension for some reason (e.g a special loss function) and you can only process a full batch of data)

For example in the tf.data API you can pass the argument drop_remainder (Boolean) to decide what you need to happen to that remainder when using tf.data.Dataset.batch().