How to conditionally scale values in Keras Lambda layer?

Question

The input tensor rnn_pv is of shape (?, 48, 1). I want to scale every element in this tensor, so I try to use Lambda layer as below:

rnn_pv_scale = Lambda(lambda x: 1 if x >=1000 else x/1000.0 )(rnn_pv)

But it comes the error:

TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

So what is the proper way to realize this function ?

Answer 1

You can't use Python control flow statements such as if-else statements to perform conditional operations in the definition of a model. Instead you need to use methods defined in Keras backends. Since you are using TensorFlow as the backend you can use tf.where() to achieve that:

import tensorflow as tf

scaled = Lambda(lambda x: tf.where(x >= 1000, tf.ones_like(x), x/1000.))(input_tensor)

Alternatively, to support all the backends, you can create a mask to do this:

from keras import backend as K

def rescale(x):
    mask = K.cast(x >= 1000., dtype=K.floatx())
    return mask + (x/1000.0) * (1-mask)

#...
scaled = Lambda(rescale)(input_tensor)

---

Update: An alternative way to support all the backends is to use K.switch method:

from keras import backend as K

scaled = Lambda(lambda x: K.switch(x >= 1000., K.ones_like(x), x / 1000.))(input_tensor)