Binary Media Type Support in Serverless Framework

Question

Can anyone help me with understanding how the binary media type support for serverless actually works?

I am using the following in the provider section of my yaml file:

binaryMediaTypes:
      - '*/*'

For each defined function that returns a binary payload I have the following:

contentHandling: CONVERT_TO_BINARY

I think this is all that needs to be done but it doesnt seem to work, my csv file gets downloaded but the content is still base 64 encoded and in the settings for my API gateway the binary media types is empty. If however, I go to the API GW settings and add a " Binary Media Types" for “text/csv” manually and redeploy the API in the console when I then download my CSV it looks ok.

I thought the serverless framework should be setting the values I define in the config file into the API GW " Binary Media Types" setting but it doesnt seem to do anything.

When I was using a couple of plugins combined to do this everything worked ok but it is my understanding that this should just work without any plugins now.

I have a similar issue with the api gateway x ray tracing option which doesnt work when added to the serverless file.

Can anyone help me with this? Failing this not working I will resort to setting it in terraform instead.

Answer 1

For future reference if anyone is looking for an explanation on the process of binary media type support for serverless:

I personally have failed to find good internal serverless packages that support binary. This is now all handled directly within serverless/APIG.

To answer the main question, I'll answer this in phases:

(1) Your serverless package

The biggest pitfall is that default Accept headers are set to */*, which is why you see a lot of documentation surrounding setting your Binary Media Type to */* within APIG. Do not do this, this means that every request you send will be encoded, which will mess up your cors approach and give you a false response. Add this to your YAML and within you API in API gateway -> <your api> -> settings -> Binary media type you will see your MIME type added once deployed.

provider:
  apiGateway:
    binaryMediaTypes:
      - 'application/vnd.mapbox-vector-tile'

This is all you need to add to your serverless file.

(2) Your functions, NODEJS Typescript:

import serverless from 'serverless-http';
import cors from 'cors';
import express from 'express';
import api from './api';

const app = express();


app.use(cors());
app.use('/', api);

export const api: serverless.Handler = serverless(app, {
  binary: ['application/vnd.mapbox-vector-tile'],
});

Note here I am using the serverless-http package, the reason this is important is because it allows you to set the accepted MIME types.

import express, { Request, Response, NextFunction } from 'express';
import axios from 'axios';

const router = express.Router();

router.get('/ping', (req: Request, res: Response) => {
  res.send('Pong');
});

router.get('/layers/:gsdata/:z/:x/:y', async (req: Request, res: Response, next: NextFunction) => {
  const {
    z, x, y, gsdata,
  } = req.params;

  try {
    const response = await axios(
      { method: 'get', url, responseType: 'stream' },
    );

    res.set({
      'Content-Type': 'application/vnd.mapbox-vector-tile',
      'Access-Control-Allow-Origin': '*',
      'Access-Control-Allow-Methods': 'GET, POST, OPTIONS, PUT, PATCH, DELETE',
    });

    response.data.pipe(res);
  } catch (err) {
    console.error(err);
    next(err);
  }
});

export default router;

As you can see here we need to set the Content type and make sure we are returning a stream and piping that into our response. This basically tells lambda we want this to be encoded into a base64.

Why base64? -

Base64 encoding schemes are commonly used when there is a need to encode binary data that needs to be stored and transferred over media that are designed to deal with ASCII. This is to ensure that the data remain intact without modification during transport

If this is not done you will end up losing packets across APIG. Hence why your data may not work if sent as a raw stream.

(3) Your client Accept header

If you do not add an accept header to your request, from the client, you will get a base64 back, if you add an accept header of application/vnd.mapbox-vector-tile this will tell APIG to decode the format on response

await fetch('<yourAPICall>', { headers: { Accept: 'application/vnd.mapbox-vector-tile' } })

Replace all cases of application/vnd.mapbox-vector-tile with your accepted binary media type and hey presto 🎉 🎉 🎉

---

Flow for Serverless, APIG binary media type