What is a "reference edge" in TensorBoard?

Question

The TensorBoard UI shows something called a "Reference edge" as distinct from a data flow (Tensor) edge:

What distinguishes the former from the latter?

The documentation says that "the outgoing operation node can mutate the incoming tensor", but indicates different symbols that don't match the UI, so it's hard to tell what's meant by "incoming" and "outgoing":

For example how does this definition apply to

cs = tf.constant([1,2,3], name='const_share')
vs = tf.Variable([1,2,3], name='var_share')
tf.add(cs, vs, name='opVS1')
tf.add(vs, cs, name='opVS2')

or to

tf.add([4],[3], name='opA')

In both cases it just seems that the reference edge indicates that the value its tail fills the Tensor represented by the edge.

Answer 1

I do not see any reference edges on your graph, but here is how you can easily get one:

import tensorflow as tf
a = tf.Variable(1, name="a")
b = a.assign_add(2)

with tf.Session() as sess:
    tf.summary.FileWriter('logs', sess.graph)
    sess.run(tf.global_variables_initializer())

When you will click on a and remove it from the main graph you will see a reference edge:

In TF there are no bidirectional edges during the runtime. But there are some operations (like tf.assign_add) which return the same value as one of the input by modifying it:

ref: A mutable Tensor

So TF adds a reference edge where the operation reads the previous value, does something and rewrite it with the new value. The analogy is most probably with pointers/references. So the doc kind of makes sense:

A reference edge showing that the outgoing operation node can mutate the incoming tensor.