http streaming in both directions

Question

Is the following scenario valid for HTTP?

1. Client sends HTTP header to Server
2. Server receives header and sends HTTP response header
3. Client streams HTTP body (chunked transfer)
4. Server receives request body chunks and sends HTTP response body chunks (again chunked transfer)

I tried implementing this using HttpWebRequest and Asp.Net Web Api but got this error on the client

An unhandled exception of type 'System.NotSupportedException' occurred in System.dll

Additional information: The stream does not support concurrent IO read or write operations

Client

static void Main(string[] args)
{
      HttpWebRequest request = WebRequest.Create("http://localhost.fiddler:16462/") as HttpWebRequest;
      request.SendChunked = true;
      request.ContentType = "application/octet-stream";
      request.Method = "POST";
      request.AllowWriteStreamBuffering = false;
      request.AllowReadStreamBuffering = false;
      request.Timeout = 3600000;

      var requestStream = request.GetRequestStream();
      var responseStream = request.GetResponse().GetResponseStream();
      int bufLength = 10 * 1024;

      byte[] requestBuffer = new byte[bufLength];
      byte[] responseBuffer = new byte[bufLength];

      for (int i = 0; i < 1024; ++i)
      {
          requestStream.Write(requestBuffer, 0, bufLength);
          responseStream.Read(responseBuffer, 0, bufLength);
      }

      requestStream.Close();
      responseStream.Close();
}

I verified that TcpClient does support simultaneous request response streaming. However, it would be nice to see HttpWebRequest also support this.

using System;
using System.Net.Sockets;
using System.Text;
using System.Threading;

namespace TcpClientTest
{
    class Program
    {
        static void Main(string[] args)
        {
            TcpClient client = new TcpClient("localhost", 16462);
            var stream = client.GetStream();

            byte[] buffer = Encoding.UTF8.GetBytes("POST http://localhost:16462/ HTTP/1.1\r\n");
            stream.Write(buffer, 0, buffer.Length);

            buffer = Encoding.UTF8.GetBytes("Content-Type: application/octet-stream\r\n");
            stream.Write(buffer, 0, buffer.Length);

            buffer = Encoding.UTF8.GetBytes("Host: localhost:16462\r\n");
            stream.Write(buffer, 0, buffer.Length);

            buffer = Encoding.UTF8.GetBytes("Transfer-Encoding: chunked\r\n\r\n");
            stream.Write(buffer, 0, buffer.Length);

            int chunkLen = 128 * 1024;
            string chunkSizeStr = chunkLen.ToString("X");
            byte[] chunkSizeBytes = Encoding.UTF8.GetBytes(chunkSizeStr + "\r\n");
            buffer = new byte[chunkLen];
            for (int i = 0; i < chunkLen; ++i)
            {
                buffer[i] = (byte)'a';
            }

            // Start reader thread
            var reader = new Thread(() =>
            {
                byte[] response = new byte[128 * 1024];
                int bytesRead = 0;
                while ((bytesRead = stream.Read(response, 0, response.Length)) > 0)
                {
                    Console.WriteLine("Read {0} bytes", bytesRead);
                }
            });

            reader.Start();

            // Streaming chunks
            for (int i = 0; i < 1024 * 1024; ++i)
            {
                stream.Write(chunkSizeBytes, 0, chunkSizeBytes.Length);
                stream.Write(buffer, 0, buffer.Length);
                stream.Write(Encoding.UTF8.GetBytes("\r\n"), 0, 2);
            }

            buffer = Encoding.UTF8.GetBytes("0\r\n\r\n");
            stream.Write(buffer, 0, buffer.Length);
            reader.Join();
        }
    }
}

Answer 1

That doesn't seem to be valid HTTP behaviour. You could make a custom "HTTP-like" client and server that start sending a response before the client finishes the request, but that seems invalid according to the HTTP spec:

6 Response

  After receiving and interpreting a request message, a server responds
  with an HTTP response message.

Web sockets might be your thing though! They allow for bidirectional streams on top of HTTP. https://en.wikipedia.org/wiki/WebSocket