Menu
I'm trying to de-normalize my Django Postgres database that backs a JSON API by storing certain objects in JSONFields:
https://docs.djangoproject.com/en/1.11/ref/contrib/postgres/fields/#jsonfield
I extended the django.contrib.postgres.fields.JSONField field to serialize data into Python objects automatically. I'm doing this in order to encapsulate my logic around the objects and to enforce the structure of the objects stored in the JSONField. I'm following the Django documentation on custom model fields here:
https://docs.djangoproject.com/en/1.11/howto/custom-model-fields/
I'm able to store my objects in the custom JSONField and can retrieve them as native Python objects, however, I've broken the admin console. When I try to view one of my objects, i get this error:
TypeError: <core.fields.PostalAddress object at 0x7fdcfaade4e0> is not JSON serializable
I assume that the problem is that built-in json.dumps function doesn't play nicely with random objects, so I hope that there is some method I can override in my PostalAddress class to get it to play nice.
Here is my code, although this is a simple case of an address, I want to apply this pattern to more complicated and useful custom JSONField use cases, so I would like to see the serialized JSON in the admin console and be able to edit the text and save it.
fields.py
from django.contrib.postgres.fields import JSONField
class PostalAddress(object):
def __init__(self, street='', city='', state='', postal_code=''):
self.street = street
self.city = city
self.state = state
self.postal_code = postal_code
def deserialize_address(address_dict):
return PostalAddress(**address_dict)
def get_empty_address():
return PostalAddress()
class PostalAddressField(JSONField):
description = "A postal address"
def __init__(self, *args, **kwargs):
super(PostalAddressField, self).__init__(*args, **kwargs)
def get_prep_value(self, value):
if value is None:
return value
if isinstance(value, PostalAddress):
return json.dumps(value.__dict__)
return value
def from_db_value(self, value, expression, connection, context):
if value is None:
return value
return deserialize_address(value)
def to_python(self, value):
if isinstance(value, PostalAddress):
return value
if value is None:
return value
return deserialize_address(value)
models.py
from django.db import models
from .fields import PostalAddressField, get_empty_address
class Person(models.Model):
shipping_address = PostalAddressField(default = get_empty_address)
Full Trace:
Request Method: GET
Request URL:...
Django Version: 1.11.6
Python Version: 3.5.2
Installed Applications:
['grappelli',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.postgres',
'rest_framework',
'rest_framework.authtoken',...
]
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Template error:
In template /home/ubuntu/venv/lib/python3.5/site-packages/grappelli/templates/admin/includes/fieldset.html, error at line 24
<core.fields.PostalAddress object at 0x7f2c7fc085f8> is not JSON serializable 14 : {% if field.is_checkbox %}
15 : <div class="c-1"> </div>
16 : <div class="c-2">
17 : {{ field.field }}{{ field.label_tag|prettylabel }}
18 : {% else %}
19 : <div class="c-1">{{ field.label_tag|prettylabel }}</div>
20 : <div class="c-2">
21 : {% if field.is_readonly %}
22 : <div class="grp-readonly">{{ field.contents }}</div>
23 : {% else %}
24 : {{ field.field }}
25 : {% endif %}
26 : {% endif %}
27 : {% if line.fields|length_is:'1' %}{{ line.errors }}{% endif %}
28 : {% if not line.fields|length_is:'1' and not field.is_readonly %}{{ field.field.errors }}{% endif %}
29 : {% if field.field.help_text %}
30 : <p class="grp-help">{{ field.field.help_text|safe }}</p>
31 : {% endif %}
32 : </div>
33 : </div>
34 : {% endfor %}
Traceback:
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/core/handlers/base.py" in _get_response
217. response = self.process_exception_by_middleware(e, request)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/core/handlers/base.py" in _get_response
215. response = response.render()
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/response.py" in render
107. self.content = self.rendered_content
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/response.py" in rendered_content
84. content = template.render(context, self._request)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/backends/django.py" in render
66. return self.template.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
207. return self._render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in _render
199. return self.nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/loader_tags.py" in render
177. return compiled_parent._render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in _render
199. return self.nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/loader_tags.py" in render
177. return compiled_parent._render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in _render
199. return self.nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/loader_tags.py" in render
72. result = block.nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/loader_tags.py" in render
72. result = block.nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/defaulttags.py" in render
216. nodelist.append(node.render_annotated(context))
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/loader_tags.py" in render
216. return template.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
209. return self._render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in _render
199. return self.nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/defaulttags.py" in render
411. return strip_spaces_between_tags(self.nodelist.render(context).strip())
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/defaulttags.py" in render
216. nodelist.append(node.render_annotated(context))
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/defaulttags.py" in render
216. nodelist.append(node.render_annotated(context))
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/defaulttags.py" in render
322. return nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/defaulttags.py" in render
322. return nodelist.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
990. bit = node.render_annotated(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_annotated
957. return self.render(context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render
1046. return render_value_in_context(output, context)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/template/base.py" in render_value_in_context
1024. value = force_text(value)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/utils/encoding.py" in force_text
76. s = six.text_type(s)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/utils/html.py" in <lambda>
385. klass.__str__ = lambda self: mark_safe(klass_str(self))
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/forms/boundfield.py" in __str__
40. return self.as_widget() + self.as_hidden(only_initial=True)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/forms/boundfield.py" in as_widget
125. value=self.value(),
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/forms/boundfield.py" in value
162. return self.field.prepare_value(data)
File "/home/ubuntu/venv/lib/python3.5/site-packages/django/contrib/postgres/forms/jsonb.py" in prepare_value
55. return json.dumps(value)
File "/usr/lib/python3.5/json/__init__.py" in dumps
230. return _default_encoder.encode(obj)
File "/usr/lib/python3.5/json/encoder.py" in encode
198. chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.5/json/encoder.py" in iterencode
256. return _iterencode(o, 0)
File "/usr/lib/python3.5/json/encoder.py" in default
179. raise TypeError(repr(o) + " is not JSON serializable")
Exception Type: TypeError at /admin/...
Exception Value: <core.fields.PostalAddress object at 0x7f2c7fc085f8> is not JSON serializable
742
The issue you're having is that Django uses it's JSONField form field to try and deserialize the object in the admin - this fails because it just uses json.dumps() which cannot handle your PostalAddress object.
You have overridden the model field, but you will also need to override the form field used in the admin. The documentation describes how to specify a custom form field for a model field.
Something like this:
from django.contrib.postgres.forms import JSONField
# Define a new form field
class PostalAddressJSONField(JSONField):
def prepare_value(self, value):
# Here, deserialize the object in a way that works.
# I've copied what you've done in your model field.
return json.dumps(value.__dict__)
Then specify this new form field in your PostalAddressField:
class PostalAddressField(JSONField):
def formfield(self, **kwargs):
defaults = {'form_class': PostalAddressJSONField}
defaults.update(kwargs)
return super().formfield(**defaults)
The ModelAdmin form should now use this custom form field, and be able to deserialize it correctly.
87
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
