Django StreamingHttpResponse for text is getting buffered

Question

I am having trouble with streaming a simple text response with django. it's a very straight-forward demo.

python3.11.4
django4.2.5

python serve a text generator with delay

@csrf_exempt
def testPost(request):
    if request.method == 'POST':
        print('--- post')
        def gen1():
            for i in range(10):
                time.sleep(0.5)
                print(f'----- {i}')
                yield str(i)
        return StreamingHttpResponse(gen1(), content_type="text/plain", buffered=False)

js tries to get response bit by bit

function testPost(messages, success, error) {
  const xhr = new XMLHttpRequest();
  xhr.open("POST", apiUrl);
  xhr.responseType = "text";
  xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
  xhr.onprogress = (event) => {
    const partialResponse = xhr.responseText.substring(
      xhr.loadedBytes || 0,
      event.loaded
    );
    xhr.loadedBytes = event.loaded;
    success(partialResponse); // Handle the partial response
  };
  xhr.onload = () => {
    console.log('---loaded');
    success(xhr.responseText); // Handle the final response
  };
  xhr.onerror = error;
  const requestData = new URLSearchParams({
    messages: JSON.stringify(messages),
  }).toString();

  xhr.send(requestData);
}

// Usage
chatPost(messages, (partialResponse) => {
  console.log(partialResponse); // Handle the partial response
}, (error) => {
  console.error(error);
});

here's the output:

server goes first

2023-10-01 07:21:02,946 INFO     Listening on TCP address 127.0.0.1:8001
--- post
----- 0
----- 1
----- 2
----- 3
----- 4
----- 5
----- 6
----- 7
----- 8
----- 9
127.0.0.1:35074 - - [01/Oct/2023:07:21:11] "POST /api/test/" 200 10

frontend result comes after all logging in server

0123456789
--loaded
0123456789

note that I am running this with daphne -p 8001 app.asgi:application on a server with multiple vCPUs with asgi, and redis daphne postgres are setup defautly.

here's my daphne conf

[fcgi-program:asgi]
# TCP socket used by Nginx backend upstream
# socket=tcp://localhost:8001
socket=tcp://0.0.0.0:8001

# Directory where your site's project files are located
directory=/var/www/app/app
# activate virtual environment
environment=PATH="/var/www/app/bin:%(ENV_PATH)s"

# Each process needs to have a separate socket file, so we use process_num
# Make sure to update "mysite.asgi" to match your project name
command=daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers app.asgi:application

# Number of processes to startup, roughly the number of CPUs you have
numprocs=3

# Give each process a unique name so they can be told apart
process_name=asgi%(process_num)d

# Automatically start and recover processes
autostart=true
autorestart=true

# Choose where you want your log to go
stdout_logfile=/var/www/app/app/asgi.log
redirect_stderr=true

nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 768;
    # multi_accept on;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    upstream channels-backend {
        server localhost:8001;  # Change Daphne's port to 8001
    }
    

    server {
        listen          80; # 8000 must visit with a port
        server_name     example.com www.example.com;
        charset         utf-8;
        return 301 https://$host$request_uri; # Redirect all http to https connections
    }

    server {
        listen 443 ssl http2;
        server_name example.com www.example.com; # Primary and www domain

        ssl_certificate /etc/nginx/cert/www.example.com.pem;
        ssl_certificate_key /etc/nginx/cert/www.example.com.key;

        # include snippets/ssl-params.conf;  # Your SSL parameters file

        # Your server configuration
        location /static {
            alias       /var/www/exp/exp/static;
        }

        location /media {
            alias       /var/www/exp/exp/media;
        }

        # For all other requests, pass to Daphne
        location / {
            try_files $uri @proxy_to_app;
        }

        location @proxy_to_app {
            proxy_pass http://channels-backend;

            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";

            proxy_buffering off;
            proxy_redirect off;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host $server_name;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

    ##
    # Basic Settings
    ##

    tcp_nopush on;
    tcp_nodelay on;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
    ssl_ecdh_curve secp384r1;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 valid=300s;
    resolver_timeout 5s;
    # add_header X-Frame-Options DENY;
    # add_header X-Content-Type-Options nosniff;
    # ssl_dhparam /etc/ssl/certs/dhparam.pem;
    # openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
    
    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip off;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
    include servers/*;
}

---

something is buffering the response, I have excluded:

• maybe due to server delay?

nope, tested with iteration of 30 and delay of 1s, pinged delayed about 80ms, not network delay for sure.

• config nginx proxy buffering already turned off

proxy_buffering off;

• nginx gzip already closed

gzip off;

• response buffered attr already set

return StreamingHttpResponse(gen1(), content_type="text/plain", buffered=False)

• all middleware disabled, going through the test, failed. it would also wait until full response is parsed

MIDDLEWARE = [
    'corsheaders.middleware.CorsMiddleware',

    # 'django.middleware.security.SecurityMiddleware',
    # 'django.contrib.sessions.middleware.SessionMiddleware',
    # 'django.middleware.common.CommonMiddleware',
    # 'django.middleware.csrf.CsrfViewMiddleware',
    # 'django.contrib.auth.middleware.AuthenticationMiddleware',
    # 'django.contrib.messages.middleware.MessageMiddleware',
    # 'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

---

this have been troubling me all day, will be more than grateful if u can give me a hand.

Answer 1

I've encountered a similar issue and delved into it for a few hours. It seems that we need to add additional attributes to the response in the Django app to allow streaming with NGINX.

Initially, the code looked like this:

Before

return StreamingHttpResponse(
            streaming_content=stream, content_type="text/event-stream"
        ) # Not working

After some investigation, I found a solution that worked for me. We need to focus on two specific lines:

response["Cache-Control"] = "no-cache"  # Prevent client cache
response["X-Accel-Buffering"] = "no"  # Allow streaming over NGINX server

After

        response = StreamingHttpResponse(
            streaming_content=stream, content_type="text/event-stream"
        )
        response["Cache-Control"] = "no-cache"  # prevent client cache
        response["X-Accel-Buffering"] = "no"  # Allow Stream over NGINX server
        return response

So, applying this to your code:

@csrf_exempt
def testPost(request):
    if request.method == 'POST':
        print('--- post')
        def gen1():
            for i in range(10):
                time.sleep(0.5)
                print(f'----- {i}')
                yield str(i)

        response = StreamingHttpResponse(gen1(), content_type="text/plain", buffered=False)
        response['Cache-Control'] = 'no-cache'
        response['X-Accel-Buffering'] = 'no'
        return response

This adjustment should resolve the issue.

Reference: https://github.com/SteveLTN/https-portal/issues/354