Playing incoming ArrayBuffer audio binary data from binaryjs server simultaneously

Question

Good day! I'm into video chat streaming this morning and I've bumped into a problem with the incoming ArrayBuffer which contains binary data of an audio.

Here is the code I found for playing binary audio data (Uint8Array):

function playByteArray(byteArray) {
    var arrayBuffer = new ArrayBuffer(byteArray.length);
    var bufferView = new Uint8Array(arrayBuffer);
    for (i = 0; i < byteArray.length; i++) {
        bufferView[i] = byteArray[i];
    }
    context.decodeAudioData(arrayBuffer, function(buffer) {
        buf = buffer;
        play();
    });
}

// Play the loaded file
function play() {
    // Create a source node from the buffer
    var source = context.createBufferSource();
    source.buffer = buf;
    // Connect to the final output node (the speakers)
    source.connect(context.destination);
    // Play immediately
    source.start(0);
}

Now below, I've used MediaStreamRecorder from https://github.com/streamproc/MediaStreamRecorder to record the stream from getUserMedia. This code will continuously send the recorded binary data to the server.

if (navigator.getUserMedia) {
        navigator.getUserMedia({audio: true, video: true}, function(stream) {
            video.src = (window.URL || window.webkitURL).createObjectURL(stream); //get this for video strewam url
          video.muted = true;
            multiStreamRecorder = new MultiStreamRecorder(stream);
            multiStreamRecorder.canvas = {
              width: video.width,
              height: video.height
            };
            multiStreamRecorder.video = video;

          multiStreamRecorder.ondataavailable = function(blobs) {
            var audioReader = new FileReader();
              audioReader.addEventListener("loadend", function() {
                var arrBuf = audioReader.result;
                var binary = new Uint8Array(arrBuf);
                streamToServ.write(binary);
                // streamToServ is the binaryjs client
              });
              audioReader.readAsArrayBuffer(blobs.audio);
            };
          multiStreamRecorder.start(1);
        }, onVideoFail);
    } else {
        alert ('failed');
    }

Convert the blobs produced (audio and video) to binary and send it to binaryjs which will be played on another client with this:

client.on('stream', function (stream, meta) {
    stream.on('data', function(data) {
        playByteArray(new Uint8Array(data));
    });
});

I had no problems with transferring the binary data but the problem is there is a hiccup sound in the playback significantly on every binary data that was played. Is there something wrong on how I play the incoming ArrayBuffers? I'm also thinking of asking streamproc about this.

Thanks in advance!

Cheers.

Answer 1

I found a solution to this problem by making an audio buffer queueing. Most of the code is from here:

Choppy/inaudible playback with chunked audio through Web Audio API

Thanks.