how to for loop a jq array result in shell script

Question

I have a json data as below

{
  "id": {
    "bioguide": "E000295",
    "thomas": "02283",
    "govtrack": 412667,
    "opensecrets": "N00035483",
    "lis": "S376"
  },
  "bio": {
    "gender": "F",
    "birthday": "1970-07-01"
  },
  "tooldatareports": [
    {
      "name": "A",
      "tooldata": [
        {
          "toolid": 12345,
          "data": [
            {
              "time": "2021-01-01",
              "value": 1
            },
            {
              "time": "2021-01-02",
              "value": 10
            },
            {
              "time": "2021-01-03",
              "value": 5
            }
          ]
        },
        {
          "toolid": 12346,
          "data": [
            {
              "time": "2021-01-01",
              "value": 10
            },
            {
              "time": "2021-01-02",
              "value": 100
            },
            {
              "time": "2021-01-03",
              "value": 50
            }
          ]
        }
      ]
    }
  ]
}

now I can use below command line to get a list containing two dict, each dict have a key "data", value is a list,

cat data.json |jq -n --stream '[fromstream(inputs | (.[0] | index("data")) as $ix | select($ix) | .[0] |= .[$ix:])]'

I want to print each dict with a loop in shell script

My except in each loop have printed a dict, total have 2 dict

But it seems like a string

below is my shell script

array=$(cat ernst.json | jq -n --stream '[fromstream(inputs | (.[0] | index("data")) as $ix | select($ix) | .[0] |= .[$ix:])]')

for d in $array
do
    echo $d
done

Does anyone have any idea?

Answer 1

$ jq -cn --stream '[fromstream(inputs | (.[0] | index("data")) as $ix | select($ix) | .[0] |= .[$ix:])][]' data.json | while read d; do echo "item: $d"; done
item: {"data":[{"time":"2021-01-01","value":1},{"time":"2021-01-02","value":10},{"time":"2021-01-03","value":5}]}
item: {"data":[{"time":"2021-01-01","value":10},{"time":"2021-01-02","value":100},{"time":"2021-01-03","value":50}]}

Note that you can get very similar output with simpler jq:

jq -c '.tooldatareports[].tooldata[].data' data.json  | while read d; do echo "item: $d"; done
item: [{"time":"2021-01-01","value":1},{"time":"2021-01-02","value":10},{"time":"2021-01-03","value":5}]
item: [{"time":"2021-01-01","value":10},{"time":"2021-01-02","value":100},{"time":"2021-01-03","value":50}]

And you can get identical output (this seems likely to be unnecessary) with:

$ jq -c '.tooldatareports[].tooldata[].data | {"data": .}' data.json  | while read d; do echo "item: $d"; done
item: {"data":[{"time":"2021-01-01","value":1},{"time":"2021-01-02","value":10},{"time":"2021-01-03","value":5}]}
item: {"data":[{"time":"2021-01-01","value":10},{"time":"2021-01-02","value":100},{"time":"2021-01-03","value":50}]}

Answer 2

Your output is a json list, so you won't get away with a direct bash loop over it. The bash loop won't know what to do with the square brackets and will use each space and newline as a separator. Use jq again to get separate objects from your json list. That is done with jq -c '.[]':

array=$(cat data.json |jq -n --stream '[fromstream(inputs | (.[0] | index("data")) as $ix | select($ix) | .[0] |= .[$ix:])]' | jq -c '.[]')

Now you have two objects in two separate lines and no spaces (-c = compact output) and you can loop over them in bash:

for d in $array
do
    echo "$d"
    # whatever else you need to do with them
done

{"data":[{"time":"2021-01-01","value":1},{"time":"2021-01-02","value":10},{"time":"2021-01-03","value":5}]}
{"data":[{"time":"2021-01-01","value":10},{"time":"2021-01-02","value":100},{"time":"2021-01-03","value":50}]}