Redirect host inside docker-compose network to localhost outside docker

Question

I have a set of containers I run using docker-compose. The network is the default network docker-compose creates, and the containers talk to each other using the hostnames docker-compose sets up automatically. So if I define a service named "my_service" in docker-compose, I access this container as the host "my_service".

In some cases I'd like to disable a specific container and redirect all requests to it to the actual localhost, not a docker container. So instead of the host "my_service" being routed to the container with the same name, I'd like to route this to the actual localhost, where I'm running the same service e.g. in an IDE with a debugger attached or something similar.

I'm not sure how to best achieve this, whether I can modify the network itself to do this or whether I have to proxy these requests in some way. Is there a way to do this that ideally only requires some changes in the docker-compose.yml and no changes to my containers itself?

I'm using docker-compose on Linux (Ubuntu 20.04) or WSL2 on Windows 10.

Answer 1

tl;dr

You would need 3 things to make it work:

1. A predefined external network for access to the host via a fixed IP
2. hostname entry for each participating service
3. An additional Docker service as router in order to redirect requests to the host. We will also use the aliases entry to allow other services to reach this router service via a predefined hostname.

How to

Note: Tested for Linux. Not sure, if it works for Windows.

1. Create the external network with a gateway. E.g. via the custom IP 172.30.0.1 defined beloew, each docker service should be able to access the host.

docker network create \
    --driver=bridge \
    --subnet=172.30.0.0/16 \
    --gateway=172.30.0.1 \
    host-net

2. Add custom hostname for each participating service in your docker-compose.yml, e.g.

services:
    service-1:
        hostname: service-1.in.docker

    service-2:
        hostname: service-2.in.docker

So, service-2 can now access to service-1 via the domain service-1.in.docker and it should work vice versa.

3. Create an additional service that utilizes the well-known multipurpose relay tool socat to act like a router, e.g.

services:
    router:
        image: alpine/socat
        command: TCP-LISTEN:5900,fork TCP:172.30.0.1:15900

Here based on TCP, it listens on localhost at port 5900 and redirects the incoming requests to the destination 172.30.0.1 (the gateway -> host machine) at port 15900.

Example

version: "3.9"

services:
  router:
    image: alpine/socat
    entrypoint: >
      sh -c "
        socat TCP-LISTEN:5900,fork TCP:172.30.0.1:15900 &
        socat TCP-LISTEN:5901,fork TCP:172.30.0.1:15901 &
        wait
      "
    networks:
      host-net:
      bridged:
        aliases:
          - service-1.in.docker

  service-1:
#   hostname: service-1.in.docker
    image: hashicorp/http-echo
    command: -listen=:5900 -text="hello world"
    networks:
      bridged:

  service-2:
    hostname: service-2.in.docker
    image: curlimages/curl:7.75.0 
    command: ["sh", "-c", "while true; do curl service-1.in.docker:5900 && sleep 5; done"]
    networks:
      bridged:

networks:
  host-net:
    external: true
  bridged:

Notes:

1. Here the router service includes two important things: host-net as network, and aliases entry for the network bridged, in which other services are also involved.

2. The router service works as follows:

(request) -> [localhost:5900] -> (redirect) -> [172.30.0.1:15900]

(request) -> [localhost:5901] -> (redirect) -> [172.30.0.1:15901]

3. Since we commented the line hostname: service-1.in.docker, the curl requests made by service-2 (curl service-1.in.docker:5900) will go to the router service (due to the alias service-1.in.docker). Subsequently, these requests are forwarded to the host machine on port 15901.

4. If we comment out the line hostname: service-1.in.docker, service-1 would respond to service2 with "hello world".

If you have the possibility to dynamically change the hostnames used by the individual services, such as env vars or properties files etc., you can make use of them, so you don't have to comment in and out every time:

services:
  router:
    image: alpine/socat
    networks:
      bridged:
        aliases:
          - service-1.in.router

  service-2:
    hostname: service-2.in.docker
    command: ["sh", '-c', 'while true; do curl ${DEST_SERVICE_HOST_ADDR_SET_BY_ENV_VAR} && sleep 5; done']

DEST_SERVICE_HOST_ADDR_SET_BY_ENV_VAR can be provided with service-1.in.router or service-1.in.docker as per need.

Footnote:

You can run a test server on your host maschine to test this, e.g.:

python3 -m http.server 15901