Menu
I'm trying to decode H264 frames, sent by my backend, like this:
/*
packet = {
type: "frame",
keyframe: <Boolean>,
pts: <BigInt>,
data: <ArrayBuffer/Uint8Array>
}
*/
const chunk = new EncodedVideoChunk({
type: packet.keyframe === false ? 'delta' : 'key',
timestamp: 0,
data: data
});
console.debug("CHUNK");
this.decoder.decode(chunk);
The decoder looks like this:
this.decoder = new VideoDecoder({
output: (frame) => {
console.debug("DECODE")
<...>
error: (error) => {
console.error(error);
}
});
The problem I have is that my log DECODE is never printed, whereas CHUNK is, but at the same time, there are also no errors thrown.
If anyone has an idea on what else I could try, I'd be very grateful.
629
asked Jan 24 '26 23:01
"I'm trying to decode H264 frames, sent by my backend, like this..."
The decoder setup depends on your input H.264 data:
Assuming it provides data in AnnexB format whose codec name is known (eg: "avc1.42C01E").
You can find the part after "avc1." by extracting the first 3 byte values in your SPS and converting them each into a hex string.
AnnexB means...
00 00 00 01.Example H.264 keyframe bytes (from AnnexB file):
00 00 00 01 67 42 C0 1E DD EC 04 40 00 00 03 00 40 00 00 0F 03 C5 8B E0 00 00 00 01 68 CE 0F 2C 80 00 00 01 65 88 84 04 BC 46 28 00 0C 7D 47 00 01 7D 78 E0 00 22 3D 80
Below is the minimum required code to decode the above H.264 keyframe bytes.
Future frames will only need to update new frame data using:
myOutputFrameData.data = new Uint8Array( some frame bytes )
Also update the frame "type" which can be "key" (IDR & I frames) or "frame" (P & B frames):
myOutputFrameData.type = "key"
Then decode new frame data with:
const chunk_frame = new EncodedVideoChunk( myOutputFrameData );
decoder.decode( chunk_frame );
Example code for decoding some H.264 frame...
<!DOCTYPE html>
<html>
<body>
<!-- Canvas to display a decoded frame -->
<canvas id="myCanvas" width="320" height="240">
</canvas>
<br>A decoded frame (eg: yellow fill)
<script>
//# for displaying decoded frame in Canvas
const myCanvas = document.getElementById("myCanvas");
const ctx = myCanvas.getContext("2d");
//# STRING for codec format...
str_codec_format = "annexb";
//# STRING for codec name... is: "avc1." + 0xAA + 0xBB + 0xCC
var str_codec_name = "avc1.42C01E";
///////////////////////////////////////////
//# AnnexB needs SPS + PPS at front then followed by keyframe..
let myFrameData_Key = new Uint8Array(
[
//# SPS (0x67 ... etc )
0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0xC0, 0x1E, 0xDD, 0xEC, 0x04, 0x40, 0x00, 0x00, 0x03, 0x00, 0x40, 0x00, 0x00, 0x0F, 0x03, 0xC5, 0x8B, 0xE0,
//# PPS (0x68 ... etc )
0x00, 0x00, 0x00, 0x01, 0x68, 0xCE, 0x0F, 0x2C, 0x80,
//# Keyframe data (0x65 ... etc )
0x00, 0x00, 0x00, 0x01, 0x65, 0x88, 0x84, 0x04, 0xBC, 0x46, 0x28, 0x00, 0x0C, 0x7D, 0x47, 0x00, 0x01, 0x7D, 0x78, 0xE0, 0x00, 0x22, 0x3D, 0x80
]
);
//# setup decoder initialize...
let decoder_init = {
output: ( videoFrame ) => { handleFrame( videoFrame ); },
error: ( evt ) => { console.log( evt.message ); },
};
//# setup decoder config...
let decoder_config = {};
decoder_config.codec = str_codec_name;
decoder_config.avc = { format: str_codec_format }
decoder_config.hardwareAcceleration = "prefer-hardware";
//# start Video decoder...
const decoder = new VideoDecoder( decoder_init );
decoder.configure( decoder_config );
//# Object to send to Video decoder (holds frame data + frame settings)
let myOutputFrameData = { };
myOutputFrameData.type = "key";
myOutputFrameData.timestamp = 0;
//# Add frames bytes into Object sent to the VideoDecoder
myOutputFrameData.data = new Uint8Array( myFrameData_Key );
/////# Try to decode a keyframe
//# convert Object into EncodedVideoChunk...
const chunk_frame = new EncodedVideoChunk( myOutputFrameData );
//# Decoder only accepts Objects from EncodedVideoChunk...
decoder.decode( chunk_frame );
async function handleFrame( input )
{
alert("Decode success : Got Frame = " + input );
//# draw frame inside a Canvas object
ctx.drawImage(input, 0, 0, myCanvas.width, myCanvas.height);
//# close frame and flush decoder (send to display)
input.close(); //decoder.flush();
await decoder.flush();
}
</script>
</body>
</html>
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
