Variational autoencoder: Does encoder must have the same number of layers as the decoder?

Question

Does encoder must have the same number of layers as the decoder, in Variational Autoencoder? I got a little better result with the encoder (Dense): 54-10-5-3 and Decoder (Dense): 3-5-10-25-35-45-54

Answer 1

You asked an excellent question and the answer is no.

Since Aldream has already pointed a few works that use asymmetric architecture, I would like to point out the theoretical work that is related to your question.

I believe the best explanation is from ICLR 2017 conference paper: "Variational Lossy Autoencoder", Xi Chen and et al.

Section 2 is the must-read section. The author links variational autoencoder to bit-back coding and shows that the average code length (which is equivalent to variational lowerbound) always has extra code length from using approximate posterior distribution. When the decoder of VAE (which is log-datalikelihood p(x|z)) can model the data without the need of q(z|x), VAE will ignore the latent variable!

It means that if we have a very strong decoder (e.g. the same number of layers as the encoder), then there is a risk that VAE might completely ignore learning q(z|x) by simply set q(z|x) to the prior p(z), hence, keeps the KL loss to 0.

For more details, please read the paper above. The connection of VAE to bit-back coding is elegant in my opinion.

Answer 2

No, it is not uncommon to have asymmetric architectures, e.g. [1, 2, 3, etc.].

1. Tang, Shuai, et al. "Exploring Asymmetric Encoder-Decoder Structure for Context-based Sentence Representation Learning." arXiv preprint arXiv:1710.10380 (2017). pdf

2. LiNalisnick, Eric, and Padhraic Smyth. "Stick-breaking variational autoencoders." International Conference on Learning Representations (ICLR). 2017. pdf

3. Nash, Charlie, and Chris KI Williams. "The shape variational autoencoder: A deep generative model of part‐segmented 3D objects." Computer Graphics Forum. Vol. 36. No. 5. 2017. pdf