Is it possible to do voice pitch shifting in Twilio group video?

Question

We have built a web application. The application's core is to arrange the meetings/sessions on the web. So User A(Meeting co-ordinator) will arrange a meeting/session and all other participants B, C, D and etc will be joining in the meeting/session. So I have used Twilio group video call to achieve it.

I have the below use case. We want to do the voice pitch shifting of the User A's(Meeting co-ordinator) voice. So all other participants will be receiving the pitch-shifted voice in group video. We have analyzed the AWS Polly in Twilio but it doesn’t match with our use case.

So please advice is there any services in Twilio to achieve this scenario.
(or) will it be possible to interrupt Twilio group call and pass the pitch-shifted voice to other participants?

Sample Code Used

initAudio();

function initAudio() {

analyser1 = audioContext.createAnalyser();
analyser1.fftSize = 1024;
analyser2 = audioContext.createAnalyser();
analyser2.fftSize = 1024;

if (!navigator.getUserMedia)
    navigator.getUserMedia = navigator.webkitGetUserMedia || navigator.mozGetUserMedia;

if (!navigator.getUserMedia)
    return(alert("Error: getUserMedia not supported!"));

navigator.getUserMedia({ audio: true }, function(stream){
    gotStream(stream);
}, function(){ console.log('Error getting Microphone stream'); });

if ((typeof MediaStreamTrack === 'undefined')||(!MediaStreamTrack.getSources)){
    console.log("This browser does not support MediaStreamTrack, so doesn't support selecting sources.\n\nTry Chrome Canary.");
} else {
    MediaStreamTrack.getSources(gotSources);
}
}
function gotStream (stream) {
audioInput = audioContext.createMediaStreamSource(stream);
outputMix = audioContext.createGain();
dryGain = audioContext.createGain();
wetGain = audioContext.createGain();
effectInput = audioContext.createGain();
audioInput.connect(dryGain);
audioInput.connect(effectInput);
dryGain.connect(outputMix);
wetGain.connect(outputMix);
audioOutput = audioContext.createMediaStreamDestination();
outputMix.connect(audioOutput);
outputMix.connect(analyser2);
crossfade(1.0);
changeEffect();
}
    function crossfade (value) {
        var gain1 = Math.cos(value * 0.5 * Math.PI);
        var gain2 = Math.cos((1.0 - value) * 0.5 * Math.PI);

    dryGain.gain.value = gain1;
    wetGain.gain.value = gain2;
}

function createPitchShifter () {
    effect = new Jungle( audioContext );
    effect.output.connect( wetGain );
    effect.setPitchOffset(1);
    return effect.input;
}

function changeEffect () {
    if (currentEffectNode)
        currentEffectNode.disconnect();
if (effectInput)
    effectInput.disconnect();

var effect = 'pitch';

switch (effect) {
    case 'pitch':
        currentEffectNode = createPitchShifter();
        break;
}

audioInput.connect(currentEffectNode);
}

Facing the error while adding the Localaudiotrack to a room

var mediaStream = new Twilio.Video.LocalAudioTrack(audioOutput.stream);

room.localParticipant.publishTrack(mediaStream, {
    name: 'adminaudio'
});

ERROR: Uncaught (in promise) TypeError: Failed to execute 'addTrack' on 'MediaStream': parameter 1 is not of type 'MediaStreamTrack'.

Answer 1

Twilio developer evangelist here.

There is nothing within Twilio itself that pitch shifts voices.

If you are building this in a browser, then you could use the Web Audio API to take the input from the user's microphone and pitch shift it, then provide the resultant audio stream to the Video API instead of the original mic stream.