"ValueError: Unknown activation: Activation" while trying to load a model with custom objects

Question

I am trying to load a model with two custom objects and am getting this error in the title.

This is where i import/define my functions, and where i allow keras to reference them by name.

from tensorflow.keras.utils import get_custom_objects
from tensorflow.python.keras.layers import LeakyReLU
from tensorflow.keras.layers import Activation
from tensorflow.keras.backend import sigmoid


def swish(x, beta=1):
    return x * sigmoid(beta * x)


get_custom_objects().update({'swish': Activation(swish)})
get_custom_objects().update({'lrelu': LeakyReLU()})

I load the model with this part

from tensorflow.keras.models import load_model
model = load_model('model.h5', custom_objects={'swish': Activation(swish), 'lrelu': LeakyReLU()}, compile=False)

I get the error below:

Traceback (most recent call last):
  File "C:\Users\Ben\PycharmProjects\untitled\trainer.py", line 102, in load_items
    model = load_model(data_loc + 'model.h5', custom_objects={'swish': Activation(swish), 'lrelu': LeakyReLU()}, compile=False)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\saving\save.py", line 146, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\saving\hdf5_format.py", line 168, in load_model_from_hdf5
    custom_objects=custom_objects)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\saving\model_config.py", line 55, in model_from_config
    return deserialize(config, custom_objects=custom_objects)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\layers\serialization.py", line 102, in deserialize
    printable_module_name='layer')
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 191, in deserialize_keras_object
    list(custom_objects.items())))
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\engine\sequential.py", line 369, in from_config
    custom_objects=custom_objects)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\layers\serialization.py", line 102, in deserialize
    printable_module_name='layer')
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 193, in deserialize_keras_object
    return cls.from_config(cls_config)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 594, in from_config
    return cls(**config)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\layers\core.py", line 361, in __init__
    self.activation = activations.get(activation)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\activations.py", line 321, in get
    identifier, printable_module_name='activation')
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 180, in deserialize_keras_object
    config, module_objects, custom_objects, printable_module_name)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 165, in class_and_config_for_serialized_keras_object
    raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)
ValueError: Unknown activation: Activation

Also might be worth noting that i am trying to save and load the model in different projects with different environments. Both are using tf 2.0.0 gpu. The imports should all be the same.

Answer 1

You should not blindly believe every tutorial in the internet. As I said in the comments, the problem is passing an activation function as a Layer (Activation to be precise), which works but it is not correct, as you get problems during model saving/loading:

def swish(x, beta = 1):
    return (x * K.sigmoid(beta * x))

get_custom_objects().update({'swish': Activation(swish)})

model = Sequential()
model.add(Dense(10, input_shape=(1,), activation="swish"))

This code above is NOT the correct way, an activation inside a layer should not be another layer. With this code I get errors during model.save with keras and tf.keras in TensorFlow 1.14. The correct way is to:

def swish(x, beta = 1):
    return (x * K.sigmoid(beta * x))

get_custom_objects().update({'swish': swish})

model = Sequential()
model.add(Dense(10, input_shape=(1,), activation="swish"))

Then you will be able to load and save the model correctly. If you need to add an activation as a layer, you should do:

model.add(Activation("swish"))

Which will also allow model save/load just fine.