Tensorflow Serving of model with 3 versions present on disk but only latest is available

Question

I'm running tensorflow_model_server (version 1.8.0 installed using apt-get) with the --model_config_file option.

My config file is something along the lines of:

model_config_list: {
  config: {
    name: "MyModelName",
    base_path: "<path to model>/MyModelName"
    model_platform: "tensorflow"
  }
}

In the the MyModelName directory there are 3 versions of the model (directories 1, 2 and 3).

When I start the model server, I can see that version 3 is made available and I can access it via a serving client by not specifying the version (so the latest is assumed) or specifically asking for version 3.

If I try and specifically ask for version 2 of the model the request fails with an error message "Servable not found for request: Specific(MyModelName, 2)".

Is it possible through the tensorflow_model_server command line options or content of my model config file to have all versions of the model present on disk available to be used?

Answer 1

I was trying to solve the same problem. If you see the ModelConfig definition, there is a field called model_version_policy of type FileSystemStoragePathSourceConfig.ServableVersionPolicy. So ideally if you set this field in your config like so:

model_config_list: {
  config: {
    name: "MyModelName",
    base_path: "<path to model>/MyModelName"
    model_platform: "tensorflow",
    model_version_policy: {all: {}}
  }
}

then you should be able to change the version policy to load all available versions.

See this github issue for more info.