In this article by Vinoo Das author of the book Learning Redis, we will see how Redis as a NOSQL data store, provides a loose sense of transaction. As in a traditional RDBMS, the transaction starts with a BEGIN and ends with either COMMIT or ROLLBACK. All these RDBMS servers are multithreaded, so when a thread locks a resource, it cannot be manipulated by another thread unless and until the lock is released. Redis by default has MULTI to start and EXEC to execute the commands. In case of a transaction, the first command is always MULTI, and after that all the commands are stored, and when EXEC command is received, all the stored commands are executed in sequence. So inside the hood, once Redis receives the EXEC command, all the commands are executed as a single isolated operation. Following are the commands that can be used in Redis for transaction:

• MULTI: This marks the start of a transaction block
• EXEC: This executes all the commands in the pipeline after MULTI
• WATCH: This watches the keys for conditional execution of a transaction
• UNWATCH: This removes the WATCH keys of a transaction
• DISCARD: This flushes all the previously queued commands in the pipeline

(For more resources related to this topic, see here.)

The following figure represents how transaction in Redis works:

Transaction in Redis

Pipeline versus transaction

As we have seen for many generic terms in pipeline the commands are grouped and executed, and the responses are queued in a block and sent. But in transaction, until the EXEC command is received, all the commands received after MULTI are queued and then executed. To understand that, it is important to take a case where we have a multithreaded environment and see the outcome.

In the first case, we take two threads firing pipelined commands at Redis. In this sample, the first thread fires a pipelined command, which is going to change the value of a key multiple number of times, and the second thread will try to read the value of that key. Following is the class which is going to fire the two threads at Redis: MultiThreadedPipelineCommandTest.java:

package org.learningRedis.chapter.four.pipelineandtx;
public class MultiThreadedPipelineCommandTest {
public static void main(String[] args) throws InterruptedException {
   Thread pipelineClient = new Thread(new PipelineCommand());
   Thread singleCommandClient = new Thread(new SingleCommand());
   pipelineClient.start();
   Thread.currentThread().sleep(50);
   singleCommandClient.start();
}
}

The code for the client which is going to fire the pipeline commands is as follows:

package org.learningRedis.chapter.four.pipelineandtx;
import java.util.Set;
import Redis.clients.jedis.Jedis;
import Redis.clients.jedis.Pipeline;
public class PipelineCommand implements Runnable{
Jedis jedis = ConnectionManager.get();
@Override
public void run() {
     long start = System.currentTimeMillis();
     Pipeline commandpipe = jedis.pipelined();
     for(int nv=0;nv<300000;nv++){
       commandpipe.sadd("keys-1", "name"+nv);
     }
     commandpipe.sync();
     Set<String> data= jedis.smembers("keys-1");
     System.out.println("The return value of nv1 after pipeline [ " + data.size() + " ]");
   System.out.println("The time taken for executing client(Thread-1) "+ (System.currentTimeMillis()-start));
   ConnectionManager.set(jedis);
}
}

The code for the client which is going to read the value of the key when pipeline is executed is as follows:

package org.learningRedis.chapter.four.pipelineandtx;
import java.util.Set;
import Redis.clients.jedis.Jedis;
public class SingleCommand implements Runnable {
Jedis jedis = ConnectionManager.get();
@Override
public void run() {
   Set<String> data= jedis.smembers("keys-1");
   System.out.println("The return value of nv1 is [ " + data.size() + " ]");
   ConnectionManager.set(jedis);
}
}

The result will vary as per machine configuration but by changing the thread sleep time and running the program couple of times, the result will be similar to the one shown as follows:

The return value of nv1 is [ 3508 ]
The return value of nv1 after pipeline [ 300000 ]
The time taken for executing client(Thread-1) 3718

Please fire FLUSHDB command every time you run the test, otherwise you end up seeing the value of the previous test run, that is 300,000

Now we will run the sample in a transaction mode, where the command pipeline will be preceded by MULTI keyword and succeeded by EXEC command. This client is similar to the previous sample where two clients in separate threads will fire commands to a single key on Redis.

The following program is a test client that gives two threads one with commands in transaction mode and the second thread will try to read and modify the same resource:

package org.learningRedis.chapter.four.pipelineandtx;
public class MultiThreadedTransactionCommandTest {
public static void main(String[] args) throws InterruptedException {
   Thread transactionClient = new Thread(new TransactionCommand());
   Thread singleCommandClient = new Thread(new SingleCommand());
   transactionClient.start();
   Thread.currentThread().sleep(30);
   singleCommandClient.start();
}
}

This program will try to modify the resource and read the resource while the transaction is going on:

package org.learningRedis.chapter.four.pipelineandtx;
import java.util.Set;
import Redis.clients.jedis.Jedis;
public class SingleCommand implements Runnable {
Jedis jedis = ConnectionManager.get();
@Override
public void run() {
   Set<String> data= jedis.smembers("keys-1");
   System.out.println("The return value of nv1 is [ " + data.size() + " ]");
   ConnectionManager.set(jedis);
}
}

This program will start with MULTI command, try to modify the resource, end it with EXEC command, and later read the value of the resource:

package org.learningRedis.chapter.four.pipelineandtx;
import java.util.Set;
import Redis.clients.jedis.Jedis;
import Redis.clients.jedis.Transaction;
import chapter.four.pubsub.ConnectionManager;
public class TransactionCommand implements Runnable {
Jedis jedis = ConnectionManager.get();
@Override
public void run() {
     long start = System.currentTimeMillis();
     Transaction transactionableCommands = jedis.multi();
     for(int nv=0;nv<300000;nv++){
       transactionableCommands.sadd("keys-1", "name"+nv);
     }
     transactionableCommands.exec();
     Set<String> data= jedis.smembers("keys-1");
     System.out.println("The return value nv1 after tx [ " + data.size() + " ]");
   System.out.println("The time taken for executing client(Thread-1) "+ (System.currentTimeMillis()-start));
   ConnectionManager.set(jedis);
}
}

The result of the preceding program will vary as per machine configuration but by changing the thread sleep time and running the program couple of times, the result will be similar to the one shown as follows:

The return code is [ 1 ]
The return value of nv1 is [ null ]
The return value nv1 after tx [ 300000 ]
The time taken for executing client(Thread-1) 7078

Fire the FLUSHDB command every time you run the test. The idea is that the program should not pick up a value obtained because of a previous run of the program. The proof that the single command program is able to write to the key is if we see the following line: The return code is [1].

Let's analyze the result. In case of pipeline, a single command reads the value and the pipeline command sets a new value to that key as evident in the following result:

The return value of nv1 is [ 3508 ]

Now compare this with what happened in case of transaction when a single command tried to read the value but it was blocked because of the transaction. Hence the value will be NULL or 300,000.

The return value of nv1 after tx [0] or
The return value of nv1 after tx [300000]

So the difference in output can be attributed to the fact that in a transaction, if we have started a MULTI command, and are still in the process of queueing commands (that is, we haven't given the server the EXEC request yet), then any other client can still come in and make a request, and the response would be sent to the other client. Once the client gives the EXEC command, then all other clients are blocked while all of the queued transaction commands are executed.

Pipeline and transaction

To have a better understanding, let's analyze what happened in case of pipeline. When two different connections made requests to the Redis for the same resource, we saw a result where client-2 picked up the value while client-1 was still executing:

Pipeline in Redis in a multi connection environment

What it tells us is that requests from the first connection which is pipeline command is stacked as one command in its execution stack, and the command from the other connection is kept in its own stack specific to that connection. The Redis execution thread time slices between these two executions stacks, and that is why client-2 was able to print a value when the client-1 was still executing.

Let's analyze what happened in case of transaction here. Again the two commands (transaction commands and GET commands) were kept in their own execution stacks, but when the Redis execution thread gave time to the GET command, and it went to read the value, seeing the lock it was not allowed to read the value and was blocked. The Redis execution thread again went back to executing the transaction commands, and again it came back to GET command where it was again blocked. This process kept happening until the transaction command released the lock on the resource and then the GET command was able to get the value. If by any chance, the GET command was able to reach the resource before the transaction lock, it got a null value.

Please bear in mind that Redis does not block execution to other clients while queuing transaction commands but blocks only during executing them.

Transaction in Redis multi connection environment

This exercise gave us an insight into what happens in the case of pipeline and transaction.

Summary

In this article, we saw in brief how to use Redis, not simply as a datastore, but also as pipeline the commands which is so much more like bulk processing. Apart from that, we covered areas such as transaction, messaging, and scripting. We also saw how to combine messaging and scripting, and create reliable messaging in Redis. This capability of Redis makes it different from some of the other datastore solutions.

Resources for Article:

Further resources on this subject:

• Implementing persistence in Redis (Intermediate) [article]
• Using Socket.IO and Express together [article]
• Exploring streams [article]