How to prevent XMLHttpRequest from buffering the entire response

Question

I'm trying to use an ajax call to a streaming endpoint on my server. The connection should be able to accept pushed data from the server forever, but the XMLHttpRequest appears to buffer the entirety of the response. I just want the browser client to receive each chunk of data once and move on to the next.

Is there any way to prevent this behavior?

UPDATE: Seems like Firefox has the ability to support this by setting XMLHttpRequest.responseType equal to "moz-chunked-text" or "moz-chunked-arraybuffer". No support in other browsers though. Likely not the best approach anyway.

WebKit equivalent to Firefox's "moz-chunked-arraybuffer" xhr responseType

Answer 1

Check out this wiki http://en.wikipedia.org/wiki/Push_technology

I believe what you're thinking of is long polling. Where a server sets it's output to chunked (see How to make PHP generate Chunked response for a php example)

Once you have a server sending a chunked response, you could use something like this https://github.com/flowersinthesand/portal to continuously read the stream.

An alternative if you can't alter your server transfer encoding, or if you have to stick with ajax, is to have your client poll the server for changes. Something like (using jQuery to shorten this)

setInterval(function(){
   // last_updated lets you compare what's changed since the last call so you can stream updates. Or you could have the server respond with a certain amount of bytes, and then have this param remember where they are in the stream and send the next X bytes
   $.get("/my/endpoint?last_updated=" + (new Date().getTime()), function(d){
      console.log("Got response");
   });
}, 5000); // Refresh every 5 seconds

Personally, I've had a lot of luck using Socket.io It's node.js based but it will take care of the how so each client will get the best performance possible. (Internally it tries to use websockets and falls back on flash sockets, and then polling so you get the fancy speed for new'ish browsers while still supporting everyone)