Batch shape in Keras Custom layer call method

Question

I'm trying to create an implementation of an RBF layer.

Here is the build method:

def build(self, input_shape):

    self.centers = self.add_weight(name='centers',
                                   shape=(self.output_dim, input_shape[1]),
                                   initializer=self.initializer,
                                   trainable=True)
    self.betas = self.add_weight(name='betas',
                                 shape=(self.output_dim,),
                                 initializer=Constant(value=self.init_betas),
                                 trainable=True)

    super(RBFLayer, self).build(input_shape)

Here is the call:

def call(self, x):
    sub = self.centers - x  # centers shape (400, 11970), x shape (100, 11970)
    sqr = sub * sub
    rbf = K.exp(-self.betas * K.sum(sqr, axis=1))
    return rbf  # must have size (100, 400)

Input size is 11970

Layer size is 400

Batch size is 100

My problem is that I expected x variable in the call method to be of shape (None, 11970), so that I could subtract it from self.centers which is (400, 11970) with broadcasting.

But I'm getting x of shape (100, 11970), hence I'm getting a batch at once. And now I need to somehow do 100 subtractions to obtain shape (100, 400, 11970). Then square and sum in the input direction to reduce it to (100, 400) shape.

Could someone advise how to do it?

Answer 1

I think I understood what you want to do. Expand the dimensionality of self.centers and x as follows and then implicit broadcasting will occur:

def call(self, x):
    centers = self.centers[None, :, :]  # Shape=(1, 400, 11970)
    x = x[:, None, :]  # Shape=(100, 1, 11970)
    sub = centers - x  # Shape=(100, 400, 11970)
    sqr = sub * sub
    rbf = K.exp(-self.betas * K.sum(sqr, axis=-1))
    return rbf  # Shape=(100, 400)

Note: not tested.