Python requests in Docker Compose containers

Question

Problem

• I have a 2-container docker-compose.yml file.
• One of the containers is a small FastAPI app.
• The other is just trying to hit the API using Python's requests package.

I can access the app container from outside with the exact same code as is in the Python package trying to hit it, and it works, but it will not work within the package.

docker-compose.yml

version: "3.8"
services:
  read-api:
    build:
      context: ./read-api
    depends_on:
      - "toy-api"
    networks:
      - ds-net
  toy-api:
    build:
      context: ./api
    networks:
      - ds-net
    ports:
      - "80:80"
networks:
  ds-net:

Relevant requests code

from requests import Session

def post_to_api(session, raw_input, path):
    print(f"The script is sending: {raw_input}")
    print(f"The script is sending it to: {path}")
    response = session.post(path, json={"payload": raw_input})
    print(f"The script received: {response.text}")

def get_from_api(session, path):
    print(f"The datalake script is trying to GET from: {path}")
    response = session.get(path)
    print(f"The datalake script received: {response.text}")

session = Session()
session.trust_env = False ### I got that from here: https://stackoverflow.com/a/50326101/534238
get_from_api(session, path="http://localhost/test")
post_to_api(session, "this is a test", path="http://localhost/raw")

Running It REPL-Style

If I create an interactive session and run those exact commands above in the requests code portion, it works:

>>> get_from_api(session, path="http://localhost/test")
The script is trying to GET from: http://localhost/test
The script received: {"payload":"Yes, you reached here..."}
>>> post_to_api(session, "this is a test", path="http://localhost/raw")
The script is sending: this is a test
The script is sending it to: http://localhost/raw
The script received: {"payload":"received `raw_input`: this is a test"}

To be clear: the API code is still being run as a container, and that container was still created with the docker-compose.yml file. (In other words, the API container is working properly, when accessed from the host.)

Running Within Container

Doing the same thing within the container, I get the following (fairly long) errors:

read-api_1    | The script is trying to GET from: http://localhost/test
read-api_1    | Traceback (most recent call last):
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
read-api_1    |     conn = connection.create_connection(
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
read-api_1    |     raise err
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
read-api_1    |     sock.connect(sa)
read-api_1    | ConnectionRefusedError: [Errno 111] Connection refused
read-api_1    | 
read-api_1    | During handling of the above exception, another exception occurred:
.
.
.
read-api_1    | Traceback (most recent call last):
read-api_1    |   File "access_api.py", line 99, in <module>
read-api_1    |     get_from_api(session, path="http://localhost/test")
read-api_1    |   File "access_datalake.py", line 86, in get_from_api
read-api_1    |     response = session.get(path)
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
read-api_1    |     return self.request('GET', url, **kwargs)
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
read-api_1    |     resp = self.send(prep, **send_kwargs)
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
read-api_1    |     r = adapter.send(request, **kwargs)
read-api_1    |   File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
read-api_1    |     raise ConnectionError(e, request=request)
read-api_1    | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa9c69b3a0>: Failed to establish a new connection: [Errno 111] Connection refused'))
ai_poc_work_read-api_1 exited with code 1

Attempts to Solve

I thought it was with how the host identified itself within the container group, or whether that origin could be accessed, so I have already tried to change the following, with no success:

• Instead of using localhost as the host, I used read-api.
  • Actually, I started with read-api, and had no luck, but once using localhost, I could at least use REPL on the host machine, as shown above.
  • I also tried 0.0.0.0, no luck. (I did not expect that to fix it.)
• I have changed what CORS ORIGINS are allowed in the API, including all of the possible paths for the container that is trying to read, and just using "*" to flag all CORS origins. No luck.

---

What am I doing wrong? It seems the problem must be with the containers, or maybe how requests interacts with containers, but I cannot figure out what.

Here are some relevant GitHub issues or SO answers I found, but none solved it:

• GitHub issue: Docker Compose problems with requests
• GitHub issue: Solving high latency requests in Docker containers
• SO problem: containers communicating with requests

Answer 1

Within the Docker network, applications must be accessed with the service names defined in the docker-compose.yml.

If you're trying to access the toy-api service, use

get_from_api(session, path="http://toy-api/test")

You can access the application via http://localhost/test on your host machine because Docker exposes the application to the host machine. However, loosely speaking, within the Docker network, localhost does not refer to the host's localhost but only to the container's localhost. And in the case of the read-api service, there is no application listening to http://localhost/test.