How to dump compressed strings in json?

Question

I've a string that I'm compressing using zlib, storing it in a dictionary and creating a md5 hash of the dictionary. But I'm getting the error:

UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte

The code is:

data['extras'] = zlib.compress("My string".encode("utf-8"))  //The string is very large that\'s why it\'s needed to be compressed to save up memory 
checkup['hash'] = hashlib.md5(json.dumps(dict(data), sort_keys=True)).hexdigest()

The dictionary is something like:

{'extras':'x\x9cK\x04\x00\x00b\x00b'}

Can anyone tell me how I can I dump this dictionary/string in JSON ?

The string is a long json. Something like:

{
    "listing": {
            "policies": null,
            "policy_explanation": "Some Text",
            "policy_name": "Flexi3",
            "updated": "7 weeks ago",
            "city": "Bengaluru",
            "country": "India",
             .
             .
             .   
}

Answer 1

You could first base64 encode it to get this to work. It will add some size to the string but probably less than you saved by compressing it first:

data['extras'] = base64.b64encode(zlib.compress("My string".encode("utf-8")))