Using Redis as a key-value store

Question

I'm testing if Redis can be a good solution for a key-value store. In my case, users of tenants can save in our system key-value pairs. The data look like the following :

{
"user" : "x1",
"tenant" : "y1",
"key": "key1",
"value": "value1",
"last_updated": "xxx-xx-xxx"
"another-metadata-field": "ddd"
}

My main goal is to :

1. Query the value of key1 based on user & tenant fast (select value from table where user=x and tenant=y and key=z).
2. Store key-value pairs of user & tenants fast.
3. Retrieve all the key-value pairs and metadata (last_updated..) of specific user/tenant (can be slow ~ 2s) (select * from table where user=x and tenant=y or select * from table where tenant=z)

I can do it with RDBMS but I guess it will be slower than using an in-memory db (which is persistent).

With Redis, I though about saving the data in the following manner (with native Redis types):

127.0.0.1:6379> sadd tenant1 user1
(integer) 1
127.0.0.1:6379> sadd tenant1 user2
(integer) 1
127.0.0.1:6379> sadd tenant1_user1 key1
(integer) 1
127.0.0.1:6379> sadd tenant1_user1 key2
(integer) 1
127.0.0.1:6379> hset "tenant1:user1:key1" value 50
(integer) 1
127.0.0.1:6379> hset "tenant1:user1:key1" last_updated "xx-xx-xxxx"
(integer) 1
127.0.0.1:6379> hget "tenant1:user1:key1" value
"50"

With this approach, I can retrieve a specific key fast but in order to retrieve all the key-values of specific user/tenant I'll need to do something like the following :

// Get all key-values of specific user and tenant
smembers tenant1_user1
1) "key1"
2) "key2"

hgetall "tenant1:user1:key1"
1) "value"
2) "50"
3) "last_updated"
4) "xx-xx-xxxx"

// Get all key-value pairs of tenant
127.0.0.1:6379> smembers tenant1
1) "user2"
2) "user1"
127.0.0.1:6379> smembers tenant1_user1
1) "key1"
2) "key2"
127.0.0.1:6379> hgetall "tenant1:user1:key1"
1) "value"
2) "50"
3) "last_updated"
4) "xx-xx-xxxx"

I was wondering :

1. Is this the redis-way for implementation in this use case ?

2. if the value is json, saving json as string seems like a bad decision in aspect of memory utilization. Does redis-json can be a good fit here ?

Answer 1

it looks to me like you're on the right track. With your approach of using hashes and sets you could made a couple of performance improvements relatively easily...

HSET is variadic, so you can set multiple properties at once, saving you a network round trip:

HSET "tenant1:user1:key1" value 50 last_updated "xx-xx-xxxx"

You can make your lookup that uses the sets and hashes together work as an atomic thing that executes on the Redis server using Lua scripting (https://redis.io/docs/interact/programmability/eval-intro/). Consider functions if you are using Redis 7 (https://redis.io/docs/interact/programmability/functions-intro/). This allows you to do looping and branch/conditional logic on the server.

If you have access to Redis Stack (https://redis.io/docs/getting-started/install-stack/) -- note that this is licensed differently but is still free for most use cases (https://redis.io/docs/about/license/) then you can use the search capability instead. You'd continue to store your data in Hashes (or use JSON documents in Redis Stack as it adds JSON as a native data type). Then use the FT.CREATE command to create an index over your data set, describing the fields you want to index. Redis will then track changes in the data for you and update the index automatically without your application having to maintain extra data structures such as your sets.

You can then perform SQL like queries using the FT.SEARCH command. This is quite a big topic, there are courses at Redis University covering these concepts if you decide to go the Redis Stack route:

• https://university.redis.com/courses/ru204/
• https://university.redis.com/courses/ru203/

You'd also asked:

if the value is json, saving json as string seems like a bad decision in aspect of memory utilization. Does redis-json can be a good fit here ?

Serializing JSON into a Redis String (e.g. to store it as a field in a hash) is expensive in the sense that you need to send all of the JSON to and from Redis to read any of it, and that you can't atomically update it. If you have access to Redis Stack, using the JSON data type allows you to significantly reduce unnecessary data movement between the client and server, and perform all sorts of operations on JSON documents in Redis atomically.