Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how do I mask the input of lstm in tf.keras

I am building a hybrid model (RNN on top of CNN) and I want to mask the input, the problem is
that mask_zero is not supported by conv layers. I have tried to do masking and pass it to lstm like this:

inputs = tf.keras.layers.Input(shape=(100,))
     mask = tf.keras.layers.Masking().compute_mask(inputs) 
     embedding = tf.keras.layers.Embedding(self.preprocess["max_features"]+1, 300, input_length=100,
                 weights=[self.preprocess["matrix"]], trainable=True)(inputs) 
     lstm = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(200,recurrent_dropout=0.2, dropout=0.2,return_sequences=True))(embedding,mask=mask)
     conv = tf.keras.layers.Conv1D(filters=200, kernel_size=3, padding='same', activation='relu')(lstm)

the 0 ind of matrix is vector of zeros.

I am getting the follwoing error from the lstm layer: IndexError: list assignment index out of range

like image 837
user12425381 Avatar asked Jan 23 '26 09:01

user12425381


1 Answers

Have you tried checking the Docs for Tensorflow? Go to this link I think it will help you.
In the above example, they add mask_zero=True

embedding = layers.Embedding(input_dim=5000, output_dim=16, mask_zero=True)
masked_output = embedding(padded_inputs)

print(masked_output._keras_mask)
like image 197
Ahmad Avatar answered Jan 25 '26 22:01

Ahmad



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!