How to pass multiple 2d array as input to tensorflow serving api?

Question

I have a tensorflow model which takes two 2d arrays as input.

This is how I trained the model.

x.shape == (100, 250)
y.shape == (100, 10)

model.fit([x,y], y_train)

Now I'm using tensorflow serving API for deploying into production. Now when I tried to make a api request for predictions I'm getting an error

"{'error': 'instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map'}"

data1 = json.dumps({"signature_name": "serving_default",
                   "instances": [x.tolist(), y.tolist()]})
# both x and y are numpy 2d array 

json_response = requests.post('http://44.287.13.8:9000/v1/models/context_model/versions/1:predict',
                               data=data1, headers=headers)

pred = json.loads(json_response.text)

print(pred)
{'error': 'instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map'}

Answer 1

When you have multiple inputs, you need to know the names of inputs from GraphDef and pass a list of dict to tensorflow serving

First, make sure that you have correct input names by running this command in terminal

saved_model_cli show --dir /home/your_user_name/path/to/your/saved_model/1 --all

In my case, I get the following as part of the output. What is important is that the names of the inputs are input_1 and input_2, again in my case.

signature_def['serving_default']: The given SavedModel SignatureDef contains the following input(s): inputs['input_1'] tensor_info: dtype: DT_FLOAT shape: (-1, 2) name: serving_default_input_1:0 inputs['input_2'] tensor_info: dtype: DT_FLOAT shape: (-1, 2) name: serving_default_input_2:0

Also, as you can see, each input takes 2 fields, respectively.

Finally, I can send an http request for a prediction with input_1 of [1.,0.] and input_2 of [0.36708057, 0.66139287]

import requests
import json

data = json.dumps({"signature_name":"serving_default","instances":[{'input_1':[1., 0.],'input_2':[0.36708057, 0.66139287]}]})
headers = {"content-type":"application/json"}
json_response = requests.post('http://localhost:8501/v1/models/test:predict',data=data,headers=headers)
print(json.loads(json_response.text))

The output was {'predictions':[[0.315578]]}