StackExchange.Redis: Does a transaction hit the server multiple times?

Question

When I execute a transaction (MULTI/EXEC) via SE.Redis, does it hit the server multiple times? For example,

        ITransaction tran = Database.CreateTransaction();
        tran.AddCondition(Condition.HashExists(cacheKey, oldKey));

        HashEntry hashEntry = GetHashEntry(newKeyValuePair);

        Task fieldDeleteTask = tran.HashDeleteAsync(cacheKey, oldKey);
        Task hashSetTask = tran.HashSetAsync(cacheKey, new[] { hashEntry });

        if (await tran.ExecuteAsync())
        {
            await fieldDeleteTask;
            await hashSetTask;
        }

Here I am executing two tasks in the transaction. Does this mean I hit the server 4 times? 1 for MULTI, 1 for delete, 1 for set, 1 for exec? Or is SE.Redis smart enough to buffer the tasks in local memory and send everything in one shot when we call ExecuteAsync?

Answer 1

It has to send multiple commands, but it doesn't pay latency costs per command; specifically, when you call Execute[Async] (and not before) it issues a pipeline (all together, not waiting for replies) of:

WATCH cacheKey                  // observes any competing changes to cacheKey
HEXIST cacheKey oldKey          // see if the existing field exists
MULTI                           // starts the transacted commands
HDEL cacheKey oldKey            // delete the existing field
HSET cachKey newField newValue  // assign the new field

then it pays latency costs to get the result from the HEXIST, because only when that is known can it decide whether to proceed with the transaction (issuing EXEC and checking the result - which can be negative if the WATCH detects a conflict), or whether to throw everything away (DISCARD).

So; either way 6 commands are going to be issued, but in terms of latency: you're paying for 2 round trips due to the need for a decision point before the final EXEC/DISCARD. In many cases, though, this can itself be further masked by the reality that the result of HEXIST could already be on the way back to you before we've even got as far as checking, especially if you have any non-trivial bandwidth, for example a large newValue.

---

However! As a general rule: anything you can do with redis MULTI/EXEC: can be done faster, more reliably, and with fewer bugs, by using a Lua script instead. It looks like what we're actually trying to do here is:

for the hash cacheKey, if (and only if) the field oldField exists: remove oldField and set newField to newValue

We can do this very simply in Lua, because Lua scripts are executed at the server from start to finish without interruption from competing connections. This means that we don't need to worry about things like atomicity i.e. other connections changing data that we're making decisions with. So:

var success = (bool)await db.ScriptEvaluateAsync(@"
if redis.call('hdel', KEYS[1], ARGV[1]) == 1 then
    redis.call('hset', KEYS[1], ARGV[2], ARGV[3])
    return true
else
    return false
end
", new RedisKey[] { cacheKey }, new RedisValue[] { oldField, newField, newValue });

The verbatim string literal here is our Lua script, noting that we don't need to do a separate HEXISTS/HDEL any more - we can make our decision based on the result of the HDEL. Behind the scenes, the library performs SCRIPT LOAD operations as needed, so: if you are doing this lots of times, it doesn't need to send the script itself over the network more than once.

From the perspective of the client: you are now only paying a single latency fee, and we're not sending the same things repeatedly (the original code sent cacheKey four times, and oldKey twice).

---

(a note on the choice of KEYS vs ARGV: the distinction between keys and values is important for routing purposes, in particular on sharded environments such as redis-cluster; sharding is done based on the key, and the only key here is cacheKey; the field identifiers in hashes do not impact sharding, so for the purpose of routing they are values, not keys - and as such, you should convey them via ARGV, not KEYS; this won't impact you on redis-server, but on redis-cluster this difference is very important, as if you get it wrong: the server will most-likely reject your script, thinking that you are attempting a cross-slot operation; multi-key commands on redis-cluster are only supported when all the keys are on the same slot, usually achieved via "hash tags")