Python - Convert JSON key/values into key/value where value is an array

Question

I have a JSON file with numerous entries like this:

    {
    "area1": "California",
    "area2": "Sierra Eastside",
    "area3": "Bishop Area",
    "area4": "Volcanic Tablelands (Happy/Sad Boulders)",
    "area5": "Fish Slough Boulders",
    "grade": "V6 ",        
    "route": "The Orgasm",
    "type1": "Boulder",
    "type2": "NONE",
    "type3": "NONE",
    "type4": "NONE",
},

I want to take the area and type entries and turn them into arrays:

   {
    "area": ["California","Sierra Eastside","Bishop Area","Volcanic Tablelands (Happy/Sad Boulders)","Fish Slough Boulders"]
    "grade": "V6 ",        
    "route": "The Orgasm",
    "type": ["Boulder","NONE","NONE","NONE"]
},

I have this code which almost works:

json_data=open('../json/routes_test.json')
datas = json.load(json_data)
datas_arrays = []
area_keys = ['area1','area2','area3','area4','area5']
type_keys = ['type1','type2','type3','type4']

for data in datas:
    areaArray = []
    typeArray = []
    deleteArray = []
    for k, v in data.iteritems():
        for area_key in area_keys:
            if (k == area_key):
                areaArray.append(v)
                deleteArray.append(k)
        for type_key in type_keys:
            if (k == type_key):
                typeArray.append(v)
                deleteArray.append(k)
    for k in deleteArray:
        del data[k]
    data['area'] = areaArray
    data['type'] = typeArray
    datas_arrays.append(data)
    print datas_arrays
    print "********"

out = json.dumps(datas_arrays, sort_keys=True,indent=4, separators=(',', ': '))
print out
f_out= open('../json/toues_test_intoarrays.json', 'w')    
f_out.write(out)
f_out.close()   

The problem is that the area array is all out of order and the type array is backwards, which I can't have. I find it strange that one is unordered and one is ordered but backwards. To me it seems like the iteration should assure they're placed in order.

Answer 1

Python dictionaries have an arbitrary ordering, they are not sorted. You want to use your prebuilt lists of keys instead:

with open('../json/routes_test.json') as json_data:
    datas = json.load(json_data)
    area_keys = ['area1','area2','area3','area4','area5']
    type_keys = ['type1','type2','type3','type4']

    for data in datas:
        data['area'] = [data[k] for k in area_keys]
        data['type'] = [data[k] for k in type_keys]
        for k in area_keys + type_keys:
            del data[k]

out = json.dumps(datas, sort_keys=True, indent=4, separators=(',', ': '))
print out
with open('../json/toues_test_intoarrays.json', 'w') as f_out:
    f_out.write(out)

which changes the dictionaries in-place.

You could even determine the area and type keys from each entry:

    for data in datas:
        keys = sorted(data.keys())

        area_keys = [k for k in keys if k.startswith('area')]
        data['area'] = [data[k] for k in area_keys]

        type_keys = [k for k in keys if k.startswith('type')]
        data['type'] = [data[k] for k in type_keys]

        for k in area_keys + type_keys:
            del data[k]

and omit the list literals with the 'area1', 'area2' etc. hardcoded lists altogether.

Answer 2

Iterate the keys in order.

for k, v in sorted(data.iteritems()):

This will fail once you get past 9, but it will do for now.