How to create/initialize a Variable with Tensorflow 1.0 Java API

Question

I'm trying to port this line of Python code:

my_var = tf.Variable(3, name="input_a")

to Java. I was able to do this with tf.constant in this way:

graph.opBuilder("Const", name)
        .setAttr("dtype", tensorVal.dataType())
        .setAttr("value", tensorVal).build()
        .output(0);

and I tried a similar approach with Variables:

graph.opBuilder("Variable", name)
        .setAttr("dtype", tensorVal.dataType())
        .setAttr("shape", shape)
        .build()
        .output(0);

but I get this error:

Exception in thread "main" java.lang.IllegalStateException: Attempting to use uninitialized value input_a
[[Node: input_a/_2 = _Send[T=DT_INT32, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_5_input_a", _device="/job:localhost/replica:0/task:0/cpu:0"](input_a)]]

I suppose I need to set a special attribute with the value or I need to initialize it later. But I cannot find the way.

I plan to do the same for most of the other tf methods (here my current effort). So I'd like to understand how to come up with the answer by myself. For example by looking at this Python sources:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/variable_scope.py https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/state_ops.py

I suspect I should assign the "initializer" attribute but there is no Initializer interface in the java API or initialize methods. Is not yet implemented? I'm new to tensorflow and to Python.

Answer 1

Having the same need as you, I used the assign node of tensorflow to assign the value to my variable. So first you need to define your node the way you did and then you need to add this node with the corresponding value. Then I refer to this new assigned node later in my graph so it does not raise the error java.lang.IllegalStateException: Attempting to use uninitialized value.

I expanded the Graph feature with a GraphBuilder class and added this required classes:

class GraphBuilder(g: Graph ) {
  def variable(name: String, dataType: DataType, shape: Shape): Output = {
    g.opBuilder("Variable", name)
      .setAttr("dtype", dataType)
      .setAttr("shape", shape)
      .build()
      .output(0)
  }

  def assign(value: Output, variable: Output): Output = {
      graph.opBuilder("Assign", "Assign/" + variable.op().name()).addInput(variable).addInput(value).build().output(0)
  }
}

val WValue = Array.fill(numFeatures)(Array.fill(hiddenDim)(0.0))
val W = builder.variable("W", DataType.DOUBLE, Shape.make(numFeatures, hiddenDim))
val W_init = builder.assign(builder.constant("Wval", WValue), W)

The assign nodes will assign your variables with pre-set value at each forward pass so it's not suited for training either. But anyway, from this post it seems that you need to add dependencies as by default the JAVA API does not provide the training nodes: https://github.com/tensorflow/tensorflow/issues/5518.