In which format should I pass image from Rust to JS?

Question

I'm trying to pass image from JS to Rust, process it and return back from Rust to JS. Please, help me to understand in which format should I return image from Rust.

My JS code look like this:

import { Image } from "image_processing";

const input = document.getElementById('image-input')
const image_canvas = document.getElementById('image-canvas')

input.addEventListener('change', async (event) => {
    const buffer = await event.target.files[0].arrayBuffer();
    const view = new Uint8Array(buffer);
    const image = Image.new(view);
    console.log('xxx1', view);

    const imageBlob = image.get_image_blob();
    console.log('xxx2', imageBlob);

    const blob = new Blob(imageBlob, { type: 'image/jpeg'});
    console.log('xxx3', blob);


    image_canvas.width = image.get_width();
    image_canvas.height = image.get_height();
    const imageBitmap = await createImageBitmap(blob, 0, 0, image_canvas.width, image_canvas.height);
    const context = image_canvas.getContext('2d');
    context.drawImage(imageBitmap, 0, 0);
});

And Rust Image.new and Image.get_image_blob functions looks like this:

#[wasm_bindgen]
impl Image {
    pub fn new(image_blob: Vec<u8>) -> Image {
        let reader = Reader::new(io::Cursor::new(image_blob))
            .with_guessed_format()
            .expect("Cursor io never fails");

        let extension = match reader.format() {
            Some(extension) => extension,
            None => panic!("Can't guess image extension")
        };

        let width;
        let height;
        let image = match reader.decode() {
            Ok(image) => {
                let (w, h) = image.dimensions();
                
                width = w;
                height = h;
  
                image
            },
            Err(error) => {
                panic!("Can't get image: {}", error)
            }
        };

        Image {
            image,
            width,
            height,
            extension,
        }
    }
    
    .
    .
    .

    pub fn get_image_blob(&self) -> Vec<u8> {
        match self.extension {
            x if x == ImageFormat::Png => self.image.to_rgba8().to_vec(),
            _ => self.image.to_rgb8().to_vec()
        }
    }
}

When I'm trying to process jpeg image in browser console I get this:

It's look like my returning blob from get_image_blob is incorrect. Before I process my image with Rust it has right file signature for jpeg: [255, 216, 255, ...]. But after processing it became [162, 162, 152, ...]. I assume that I should try changing to_rgb8 method on something else, but I can't understand why. Can you help me and explain what I'm doing wrong?

EDIT: Turns out that signature changes when I call decode method, but I still don't know why.

Answer 1

The decode method take a JPEG file and returns the raw pixels as an uncompressed block of memory. In other words, the [162, 162, 152...] means that the top right pixel has value red=162, green=162, and blue=152. You can make image from raw byte data by using ImageData or by encoding that data with help of image::codecs::jpeg::JpegEncoder.encode or save_buffer_with_format methods. For more information you can read this issue.