Syncs of RDBMS to Redis

Question

I have RDS that serves as the source of truth. A challenge that I have is to have this database partially synced to Redis to make it available for a server app to use. This would be a one way sync always going in one direction, but I can't wrap my head around what tools do I use to make these syncs happen preferably in an optimized way. In other words, rather than loading the entire data set it would be great if deltas are synced only.

I hope someone can give some insight on how this can be done. Thank you!

Answer 1

Most RDBMS provide a way to subscribe to the transaction allowing you to put in place a "change data capture" event streams.

In this case you can subscribe to the databases events, and put the change or updated record inside Redis.

You can for example use Debezium to capture the event, as you can see Debezium community has connectors for various datasources.

---------          ------------         --------- 
| RDBMS |========>>| Debezium | ======> | Redis |
---------          ------------         ---------

This demonstration (mostly Java) shows this (a little richer since it is using Redis Streams and an intermediate state), the event is capture by this method so capture Insert/Update/Delete in MySQL and sending the information to Redis.

Another option, that does not match you need but interesting, is to do a "write behind" Cache. In this case you update the Cache in Redis and Redis push the update in RDBMS using Gears.

You can find more information about this "write behind" with Redis Gears in this GitHub repo.

---------          ---------         --------- 
| Redis |========>>| Gears | ======> | RDBMS |
---------          ---------         ---------