How to remove \n, \r, \t in a json

Question

I'm creating a json file using jq. This is the output:

{
   "temperature":"21", 
   "humidity":"12.3", 
   "message":"Today ID 342 is running"
}
{
   "temperature":"13", 
   "humidity":"40.1", 
   "message":"Today ID 98 is running"
}

If i try to open this file using Python, it gives me errors unless i remove manually newlines and tabs like this:

{"temperature":"21","humidity":"12.3","message":"Today ID 342 is running"}
{"temperature":"13","humidity":"40.1","message":"Today ID 98 is running"}

I tried to use the -j option in jq, but nothing changed. Any suggestions? Also a solution which uses other programs is fine (sed etc). Thanks!!

Answer 1

tabs, newlines or spaces within a json dict or list are absolutely ok.

The file is not a valid json document because it contains many json documents (dictionaries in this case) separated by newlines. The result of this is not a valid json document and can't be parsed by a strict json parser. At least not by the one which comes with Python's json library.

If you accept to pre-process the file with jq, you could put those objects into a list with the -s option:

jq -s . input.json > output.json
cat output.json
[
  {
    "temperature": "21",
    "humidity": "12.3",
    "message": "Today ID 342 is running"
  },
  {
    "temperature": "13",
    "humidity": "40.1",
    "message": "Today ID 98 is running"
  }
]

Then use json.load in Python:

import json

with open('output.json') as file_desc:
    measurements = json.load(file_desc)

---

Pure python solutions can be found here: How I can I lazily read multiple JSON values from a file/stream in Python?