Tuning number of hidden layers in Keras

Question

I'm just trying to explore keras and tensorflow with the famous MNIST dataset. I already applied some basic neural networks, but when it comes to tuning some hyperparameters, especially the number of layers, thanks to the sklearn wrapper GridSearchCV, I get the error below:

Parameter values for parameter (hidden_layers) need to be a sequence(but not a string) or np.ndarray.

So you can have a better view I post the main parts of my code.

Data preparation

# Extract label
X_train=train.drop(labels = ["label"],axis = 1,inplace=False)
Y_train=train['label']
del train

# Reshape to fit MLP
X_train = X_train.values.reshape(X_train.shape[0],784).astype('float32')
X_train = X_train / 255

# Label format
from keras.utils import np_utils
Y_train = keras.utils.to_categorical(Y_train, num_classes = 10)
num_classes = Y_train.shape[1]

Keras part

from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import GridSearchCV

# Function with hyperparameters to optimize
def create_model(optimizer='adam', activation = 'sigmoid', hidden_layers=2):
  # Initialize the constructor
    model = Sequential()
      # Add an input layer
    model.add(Dense(32, activation=activation, input_shape=784))

    for i in range(hidden_layers):
        # Add one hidden layer
        model.add(Dense(16, activation=activation))

      # Add an output layer 
    model.add(Dense(num_classes, activation='softmax'))
      #compile model
    model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=
     ['accuracy'])
    return model

# Model which will be the input for the GridSearchCV function
modelCV = KerasClassifier(build_fn=create_model, verbose=0)

GridSearchCV

from keras.activations import relu, sigmoid
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import Dropout
from keras.utils import np_utils

activations = [sigmoid, relu]
param_grid = dict(hidden_layers=3,activation=activations, batch_size = [256], epochs=[30])
grid = GridSearchCV(estimator=modelCV, param_grid=param_grid, scoring='accuracy')
grid_result = grid.fit(X_train, Y_train)

I just want to let you know that the same kind of question has already been asked here Grid Search the number of hidden layers with keras but the answer is not complete at all and I can't add a comment to reply to the answerer.

Thank you!

Answer 1

You should add:

for i in range(int(hidden_layers)):
    # Add one hidden layer
    model.add(Dense(16, activation=activation))

Try to add the values of param_grid as lists :

params_grid={"hidden_layers": [3]}