Graph disconnected when trying to create models in Keras with .get_layer method

Question

In the normal code, I do something like this, and everything works fine:

from keras.layers import Input, Dense
from keras.models import Model
import keras.backend as K
import numpy as np
import tensorflow as tf
from sklearn.datasets import make_blobs

X, y = make_blobs(500,50,2)

def make_network1():
    input_layer = Input((50,))
    layer1 = Dense(100,name='network1_dense1')(input_layer)
    output = Dense(50,name='network1_dense2')(layer1)
    model = Model(input_layer,output)

    return model

def make_network2():
    input_layer = Input((50,))
    layer1 = Dense(100,name='network2_dense1')(input_layer)
    output = Dense(1,name='network2_output')(layer1)
    model = Model(input_layer,output)

    return model

network1 = make_network1()
network2 = make_network2()
output = network2(network1.output)

model = Model(network1.input, output)

Now, I want to experiment with the .get_layer method and .output attribute in Keras by replacing the last line of code with:

model = Model(network1.input, network2.get_layer('network2_output').output)

Then it gives me the following error:

Graph disconnected: cannot obtain value for tensor Tensor("input_4:0", shape=(?, 50), dtype=float32) at layer "input_4". The following previous layers were accessed without issue: []

My Question

However, shouldn't be output and network2.get_layer('network2_output').output the same thing? When I try to print both of them out, it says:

Tensor("model_14/network2_output/BiasAdd:0", shape=(?, 1), dtype=float32)

and

Tensor("network2_output_1/BiasAdd:0", shape=(?, 1), dtype=float32)

And network2 has been connected to the output of network1 already, I don't get why it is disconnected. How to make the code works with the .get_layer and .output methods?

I am using keras==2.24 and tensorflow-gpu==1.5.

Answer 1

After running this line:

output = network2(network1.output)

the network2 model has two computation flows: one is the original one constructed when running make_network2(), and another is the computation flow with network1.output as the input constructed when running the above line. Therefore, it would have two outputs corresponding to each of these two computation flows:

>>> network2.get_output_at(0)
<tf.Tensor 'network2_output_4/BiasAdd:0' shape=(?, 1) dtype=float32>

>>> network2.get_output_at(1)
<tf.Tensor 'model_14/network2_output/BiasAdd:0' shape=(?, 1) dtype=float32>

Therefore, when you want to go from the network1.input to the output of network2 model, you must use the second output which is connected to the network1.input:

model = Model(network1.input, network2.get_output_at(1))

Essentially, network2.get_output_at(1) is equivalent to output obtained in this line: output = network2(network1.output).

Answer 2

shouldn't be output and network2.get_layer('network2_output').output the same thing?

No!, they are not the same thing. Let me explain what is happening here

network1 = make_network1()
network2 = make_network2()
output = network2(network1.output)

First you are creating two model's with one input for each layer and then you are replacing the second model's input with last layers output of the first model. This way you are making inputs of the output variable to be the first model's input. So the network1.inputs and output are connected. But on the following line there is no connection between network1.input and network2.get_layer('network2_output').output

model = Model(network1.input, network2.get_layer('network2_output').output)