Python to create dict keys path similarly to mkdir -p

Question

Let's say we have a dict parsed from json and we read values from it from the keys in the form of key path path-to.my.keys

my_dict['path-to']['my']['keys']

In file system we have mkdir -p to create such path if it not exists.

In python, do we have such similar syntax/function to create key path for dict aka default empty dict for missing keys? My google search results not very helpful.

Answer 1

TLDR

You can use dict.setdefault or collections.defaultdict.

def make_path(d: dict, *paths: str) -> None:
    for key in paths:
        d = d.setdefault(key, {})

make_path(my_dict, 'path-to', 'my', 'keys')
assert my_dict['path-to']['my']['keys'] is not None

Full details

Solution 1. dict.setdefault:

my_dict.setdefault('path-to', {}).setdefault('my', {}).setdefault('keys', {})

Pros:

• my_dict is normal dict
• making dict happens only explicitly
• No restrict of depth

Cons:

• You should call setdefault method every use cases.

---

Solution 2. collections.defaultdict:

from collections import defaultdict

my_dict = defaultdict(lambda: defaultdict(lambda: defaultdict(dict)))
my_dict['path-to']['my']['keys']

Pros:

• You don't need to call checking existence at all.

Cons:

• Making dictionary happens implicitly.
• my_dict is not pure dict.
• You have depth limit by definition of my_dict.

---

Solution 3. advanced from solution 1: Make your own function

def make_path(my_dict: dict, *paths: str) -> dict:
    while paths:
        key, *paths = paths
        my_dict = my_dict.setdefault(key, {})
    return my_dict


test = {'path-to': {'test': 1}}
print(test)

make_path(test, 'path-to', 'my', 'keys')['test2'] = 4
print(test)

print(make_path(test))  # It's okay even no paths passed

output:

{'path-to': {'test': 1}}
{'path-to': {'test': 1, 'my': {'keys': {'test2': 4}}}}
{'path-to': {'test': 1, 'my': {'keys': {'test2': 4}}}}

---

Solution 4. advanced from solution 2: Make your own class

class MyDefaultDict(dict):
    def __missing__(self, key):
        self[key] = MyDefaultDict()
        return self[key]


my_dict = MyDefaultDict()
print(my_dict)
my_dict['path-to']['my']['keys'] = 'hello'
print(my_dict)

output:

{}
{'path-to': {'my': {'keys': 'hello'}}}

---

Conclusion

I think that solution 3 is most similar to your need, but you can use any other options if it fits to your case.

---

Append

How about in Solution 4 we have dict :d already parsed from a json? Your solution starts from MyDefaultDict() type not from what returned from jsons.loads()

If you can edit json.loads part, then try:

import json


class MyDefaultDict(dict):
    def __missing__(self, key):
        self[key] = MyDefaultDict()
        return self[key]


data = '{"path-to": {"my": {"keys": "hello"}}}'
my_dict = json.loads(data, object_pairs_hook=MyDefaultDict)
print(type(my_dict))

output:

<class '__main__.MyDefaultDict'>

Answer 2

There's the recursive defaultdict trick that allows you to set values at random paths down a nested structure without explicitly creating the path:

import json
from collections import defaultdict

nested = lambda: defaultdict(nested)

d = nested()
d['path']['to']['nested']['key'] = 'value'

print(json.dumps(d))
# {"path": {"to": {"nested": {"key": "value"}}}}

Non-existing keys will return empty defaultdicts.