semitech:weaviate extensions-storage connection refused while deploying docker container

Question

I am trying to deploy weaviate docker container using following docker-compose stack. cotainner is deployed but i am getting log message connection refused for storage engine like below. help !

{"action":"extensions_retrieve_all","error":"Get http://weaviate:8080/v1/modules/text2vec-contextionary/extensions-storage/: dial tcp 172.19.0.2:8080: connect: connection refused","level":"error","msg":"","time":"2021-09-07T06:42:31Z"}

version: '2'
services:
  weaviate:
    image: semitechnologies/weaviate:1.2.1
    ports:
    - 8080:8080
    restart: on-failure:0
    environment:
      CONTEXTIONARY_URL: contextionary:9999
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-contextionary'
      ENABLE_MODULES: text2vec-contextionary
  contextionary:
    environment:
      OCCURRENCE_WEIGHT_LINEAR_FACTOR: 0.75
      EXTENSIONS_STORAGE_MODE: weaviate
      EXTENSIONS_STORAGE_ORIGIN: http://weaviate:8080
      NEIGHBOR_OCCURRENCE_IGNORE_PERCENTILE: 5
      ENABLE_COMPOUND_SPLITTING: 'false'
    image: semitechnologies/contextionary:en0.16.0-v1.0.2

Answer 1

Short answer

This is just a temporary timing issue during startup and nothing to worry about, as long as the error doesn't repeat once all components are ready.

PS: The Weaviate version you are using (v1.2.0) is quite old, at the time of writing this message the most recent version is v1.7.0. You can use the Weaviate Customizer to generate your desired config with the newest versions.

Long answer

The Weaviate setup you are using consists of two components, the Weaviate Core Database, and the contextionary inference container which is provided by the text2vec-contextionary module.

The weaviate container is stateful (as it is the database), the module containers are typically stateless to allow for easier scaling. However, the text2vec-contextionary module has a feature where it can be extended with new concepts and those need to be stored somewhere. If modules require third-party storage, they can use the internal module APIs persistence functions. The text2vec-contextionary module makes use of that functionality and exposes its stored extensions to the inference container on this internal API (GET /v1/modules/text2vec-contextionary/extensions-storage). This allows the inference container to use Weaviate as a persistent database by accessing it on that URL.

What you are seeing during startup is that both containers start up independently. If the contextionary inference container is slightly faster than the Weaviate Core database, it immediately tries to contact Weaviate which is not yet ready for the first three seconds.

If you look at the following logs (which are taken from the docker-compose.yml you posted above), you can see two things:

1. The first red arrow represents the point in time where contextionary container starts trying to access the weaviate container
2. The second red arrow represents the point in time when the weaviate container has fully started up. Note that the error no longer occurs at that point.

All that means is that 3 requests to sync state from the persistent database into the contextionary container have failed, but all subsequent ones (as you no longer see errors after the second red arrow) have succeeded and everything is now fully usable.