Howto distinguish between not-provided and null when deserializing JSON

Question

We send JSON strings from our front end as input to our java code. The Java side turns that into beans using Gson. Now my front end guy approached me with these requirements:

• sometimes he wants to pass a new value, that the backend simply writes into the database
• sometimes he wants to pass no value, which tells the backend to not do anything about this value
• sometimes he wants to pass a null, which tells the backend to reset to some "default value" (known to the backend, but the front end doesn't care about it)
• it should also work with strings, numbers, boolean, ...

We came up with this idea:

import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;
import java.lang.reflect.Type;
import java.util.Objects;
import org.junit.Test;
import com.google.gson.*;

class ResetableValue<T> {
    private static enum Content {
        VALUE, RESET, NOT_PROVIDED
    };
    private final T value;
    private final Content content;

    public ResetableValue(T value) {
        this(value, Content.VALUE);
    }
    private ResetableValue(T value, Content content) {
        this.value = value;
        this.content = content;
    }
    static <T> ResetableValue<T> asReset() {
        return new ResetableValue<>(null, Content.RESET);
    }
    static <T> ResetableValue<T> asNotProvided() {
        return new ResetableValue<>(null, Content.NOT_PROVIDED);
    }
    T getValue() {
        if (content != Content.VALUE) {
            throw new IllegalStateException("can't provide value for " + content);
        }
        return value;
    }
    boolean isReset() {
        return content == Content.RESET;
    }
    boolean isNotProvided() {
        return content == Content.NOT_PROVIDED;
    }
    @Override
    public String toString() {
        if (content == Content.VALUE) {
            return Objects.toString(value);
        }
        return content.toString();
    }
}

class ResetableValueDeserializer implements JsonDeserializer<ResetableValue<String>> {
    public ResetableValue<String> deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
        throws JsonParseException {
        return new ResetableValue<String>(json.getAsJsonPrimitive().getAsString());
    }
}
class ExampleBean {
    private ResetableValue<String> property = ResetableValue.asNotProvided();
    public ResetableValue<String> getProperty() {
        if (property == null) {
            return ResetableValue.asReset();
        }
        return property;
    }
    @Override
    public String toString() {
        return "property: " + Objects.toString(property);
    }
}

public class GsonStuffTest {
    Gson gson = new GsonBuilder().registerTypeAdapter(ResetableValue.class, new ResetableValueDeserializer()).create();

    @Test
    public void testValue() {
        String serializedContent = "{\"property\":\"foo\"}";
        ExampleBean bean = gson.fromJson(serializedContent, ExampleBean.class);
        assertThat(bean.getProperty().getValue(), is("foo"));
    }
    @Test
    public void testIsNotProvided() {
        String serializedContent = "{}";
        ExampleBean bean = gson.fromJson(serializedContent, ExampleBean.class);
        assertThat(bean.getProperty().isNotProvided(), is(true));
    }
    @Test
    public void testIsReset() {
        String serializedContent = "{\"property\":null}";
        ExampleBean bean = gson.fromJson(serializedContent, ExampleBean.class);
        assertThat(bean.getProperty().isReset(), is(true));
    }
}

Please note: the idea is of course to have multiple different fields of that type ResetableValue in a bean. And then one field might care a value, one is omitted, another is set to to null.

Question(s):

• The above example "works" - but I really dislike the fact that I have to handle the "reset" case within the getProperty() method of my bean. As that means: it is not enough to have a custom deserializer, I also need to put that special check into any getter method. So: are there more elegant solutions to this? Is there a way to have Gson distinguish between "a property is not showing up" vs. "a property is set to null"?
• The above example claims to be generic; but obviously the deserializer code only works for string properties. Is there a way to make this "really generic"?

I guess a different way to express my question is: is there something like "Optionals" support when deserializing JSON into beans using Gson?

Answer 1

The above example "works" - but I really dislike the fact that I have to handle the "reset" case within the getProperty() method of my bean. As that means: it is not enough to have a custom deserializer, I also need to put that special check into any getter method. So: are there more elegant solutions to this? Is there a way to have Gson distinguish between "a property is not showing up" vs. "a property is set to null"?

Sort of. Your getProperty seems to have a redundant check: it should never check for null and just return the property field in any case assuming that Gson managed to instantiate it.

The above example claims to be generic; but obviously the deserializer code only works for string properties. Is there a way to make this "really generic"?

Yes, via type adapter factories and type adapters (regarding the latter: JsonSerializer and JsonDeserializer classes use JSON trees consuming more memory, but type adapters are stream-fashioned and consume much less).

Let's consider you have a generic tri-state value holder like the following one. I would also hide away the constructor to make it more fluent and encapsulate the way it's instantiated (or not instantiated).

final class Value<T> {

    private static final Value<?> noValue = new Value<>(State.NO_VALUE, null);
    private static final Value<?> undefined = new Value<>(State.UNDEFINED, null);

    private enum State {

        VALUE,
        NO_VALUE,
        UNDEFINED

    }

    private final State state;
    private final T value;

    private Value(final State state, final T value) {
        this.value = value;
        this.state = state;
    }

    static <T> Value<T> value(final T value) {
        if ( value == null ) {
            return noValue();
        }
        return new Value<>(State.VALUE, value);
    }

    static <T> Value<T> noValue() {
        @SuppressWarnings("unchecked")
        final Value<T> value = (Value<T>) noValue;
        return value;
    }

    static <T> Value<T> undefined() {
        @SuppressWarnings("unchecked")
        final Value<T> value = (Value<T>) undefined;
        return value;
    }

    T getValue()
            throws IllegalStateException {
        if ( state != State.VALUE ) {
            throw new IllegalStateException("Can't provide value for " + state);
        }
        return value;
    }

    boolean isValue() {
        return state == State.VALUE;
    }

    boolean isNoValue() {
        return state == State.NO_VALUE;
    }

    boolean isUndefined() {
        return state == State.UNDEFINED;
    }

    @Override
    public String toString() {
        if ( state != State.VALUE ) {
            return state.toString();
        }
        return Objects.toString(value);
    }

}

Next, define a simple data bag to hold the values. Note that you must either declare them as undefined() in order to preserve the null-object semantics, or assign it a null so Gson would take care of it below (you choose).

final class DataBag {

    final Value<Integer> integer = null;/* = undefined()*/
    final Value<String> string = null;/* = undefined()*/

    private DataBag() {
    }

}

Some reflection utilities code here to analyze type parameterization and build sub-to-super class hierarchy iterators (I don't know how to create a Java 8 stream from scratch yet):

final class Reflection {

    private Reflection() {
    }

    static Type getTypeParameter0(final Type type) {
        if ( !(type instanceof ParameterizedType) ) {
            return Object.class;
        }
        final ParameterizedType parameterizedType = (ParameterizedType) type;
        return parameterizedType.getActualTypeArguments()[0];
    }

    static Iterable<Class<?>> subToSuperClass(final Class<?> subClass) {
        return subToSuperClass(Object.class, subClass);
    }

    static <SUP, SUB extends SUP> Iterable<Class<?>> subToSuperClass(final Class<SUP> superClass, final Class<SUB> subClass) {
        if ( !superClass.isAssignableFrom(subClass) ) {
            throw new IllegalArgumentException(superClass + " is not assignable from " + subClass);
        }
        return () -> new Iterator<Class<?>>() {
            private Class<?> current = subClass;

            @Override
            public boolean hasNext() {
                return current != null;
            }

            @Override
            public Class<?> next() {
                if ( current == null ) {
                    throw new NoSuchElementException();
                }
                final Class<?> result = current;
                current = result != superClass ? current.getSuperclass() : null;
                return result;
            }
        };
    }


}

ValueTypeAdapterFactory

ValueTypeAdapterFactory is responsible for any generic values by delegating (de)serialization process to downstream type adapters.

final class ValueTypeAdapterFactory
        implements TypeAdapterFactory {

    private static final TypeAdapterFactory valueTypeAdapterFactory = new ValueTypeAdapterFactory();

    private ValueTypeAdapterFactory() {
    }

    static TypeAdapterFactory getValueTypeAdapterFactory() {
        return valueTypeAdapterFactory;
    }

    @Override
    public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
        if ( !Value.class.isAssignableFrom(typeToken.getRawType()) ) {
            return null;
        }
        final Type valueTypeParameter = getTypeParameter0(typeToken.getType());
        // Some boring Java unchecked stuff here...
        @SuppressWarnings("unchecked")
        final TypeAdapter<Object> innerTypeAdapter = (TypeAdapter<Object>) gson.getDelegateAdapter(this, TypeToken.get(valueTypeParameter));
        final TypeAdapter<Value<Object>> outerTypeAdapter = new ValueTypeAdapter<>(innerTypeAdapter);
        @SuppressWarnings("unchecked")
        final TypeAdapter<T> typeAdapter = (TypeAdapter<T>) outerTypeAdapter;
        return typeAdapter;
    }

    private static final class ValueTypeAdapter<T>
            extends TypeAdapter<Value<T>> {

        private final TypeAdapter<T> innerTypeAdapter;

        private ValueTypeAdapter(final TypeAdapter<T> innerTypeAdapter) {
            this.innerTypeAdapter = innerTypeAdapter;
        }

        @Override
        public void write(final JsonWriter out, final Value<T> value)
                throws IOException {
            if ( value.isValue() ) {
                final T innerValue = value.getValue();
                innerTypeAdapter.write(out, innerValue);
                return;
            }
            // Considering no-value is undefined in order not to produce illegal JSON documents (dangling property names, etc)
            if ( value.isNoValue() || value.isUndefined() ) {
                innerTypeAdapter.write(out, null);
                return;
            }
            throw new AssertionError();
        }

        @Override
        public Value<T> read(final JsonReader in)
                throws IOException {
            final JsonToken token = in.peek();
            if ( token == NULL ) {
                in.nextNull();
                return noValue();
            }
            return value(innerTypeAdapter.read(in));
        }

    }

}

PostValueTypeAdapterFactory

PostValueTypeAdapterFactory is responsible for "adjusting" POJOs that have null-initialized Value fields by using reflection. By not registering this factory all Value fields must be initialized with undefined() manually. Any sequential data structures like iterables/collections/lists|sets, maps or arrays are not implemented here for simplicity.

final class PostValueTypeAdapterFactory
        implements TypeAdapterFactory {

    private static final TypeAdapterFactory postValueTypeAdapterFactory = new PostValueTypeAdapterFactory();

    private PostValueTypeAdapterFactory() {
    }

    static TypeAdapterFactory getPostValueTypeAdapterFactory() {
        return postValueTypeAdapterFactory;
    }

    @Override
    public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
        final List<Field> valueFields = collectValueFields(typeToken.getRawType());
        if ( valueFields.isEmpty() ) {
            return null;
        }
        final TypeAdapter<T> delegateTypeAdapter = gson.getDelegateAdapter(this, typeToken);
        return new PostValueTypeAdapter<>(delegateTypeAdapter, valueFields);
    }

    // Just scan class the whole type hierarchy (except java.lang.Object) to find any occurrences of Value<T> fields
    private static List<Field> collectValueFields(final Class<?> type) {
        return StreamSupport.stream(subToSuperClass(type).spliterator(), false)
                .filter(clazz -> clazz != Object.class)
                .flatMap(clazz -> Stream.of(clazz.getDeclaredFields()))
                .filter(field -> field.getType() == Value.class)
                .peek(field -> field.setAccessible(true))
                .collect(toImmutableList());
    }

    private static final class PostValueTypeAdapter<T>
            extends TypeAdapter<T> {

        private final TypeAdapter<T> delegateTypeAdapter;
        private final List<Field> valueFields;

        private PostValueTypeAdapter(final TypeAdapter<T> delegateTypeAdapter, final List<Field> valueFields) {
            this.delegateTypeAdapter = delegateTypeAdapter;
            this.valueFields = valueFields;
        }

        @Override
        public void write(final JsonWriter out, final T value)
                throws IOException {
            delegateTypeAdapter.write(out, value);
        }

        @Override
        public T read(final JsonReader in)
                throws IOException {
            try {
                final T value = delegateTypeAdapter.read(in);
                for ( final Field valueField : valueFields ) {
                    // A Value<T> field is null? Make it undefined
                    if ( valueField.get(value) == null ) {
                        valueField.set(value, undefined());
                    }
                }
                return value;
            } catch ( final IllegalAccessException ex ) {
                throw new IOException(ex);
            }
        }

    }

}

JUnit test:

public final class GsonStuffTest {

    private static final Gson gson = new GsonBuilder()
            .registerTypeAdapterFactory(getValueTypeAdapterFactory())
            .registerTypeAdapterFactory(getPostValueTypeAdapterFactory())
            .create();

    @Test
    public void testIsValue() {
        final DataBag dataBag = parseDataBag("{\"integer\":100,\"string\":\"foo\"}");
        assertThat(dataBag.integer.isValue(), is(true));
        assertThat(dataBag.integer.getValue(), is(100));
        assertThat(dataBag.string.isValue(), is(true));
        assertThat(dataBag.string.getValue(), is("foo"));
    }

    @Test
    public void testIsNoValue() {
        final DataBag dataBag = parseDataBag("{\"integer\":null,\"string\":null}");
        assertThat(dataBag.integer.isNoValue(), is(true));
        assertThat(dataBag.string.isNoValue(), is(true));
    }

    @Test
    public void testIsUndefined() {
        final DataBag dataBag = parseDataBag("{}");
        assertThat(dataBag.integer.isUndefined(), is(true));
        assertThat(dataBag.string.isUndefined(), is(true));
    }

    private static DataBag parseDataBag(final String json) {
        return gson.fromJson(json, DataBag.class);
    }

}