Message broker – Async communication between services

In the previous topic, we talked about synchronous communication between microservices using binary and alternatives to REST. This topic will deal with the communication between microservices using message broker, that is, a messaging system with a physical element, a communication layer, and a message bus.

With messaging systems, it is impossible to reproduce the Death Star. The design of the Death Star in a more robust application would be something like the following:

The diagram of a messaging system is totally different, similar to the one shown in the following:

The message bus can be used for both synchronous and asynchronous communication, but certainly, the major point of emphasis of the message bus is in asynchronous communication.

You may wonder, if the messaging diagram is simpler and you can use this type of tool for synchronous communication, why not use this messaging for all types of communication between microservices?

The answer to this question is quite simple. A message bus is a physical component within the stack of microservices. It needs to be scaled just like any other physical component-based data storage and cache. This means that with a high-volume message, the synchronous mode of communication could be committed to an unwanted delay in the responses of the processes.

It is critical to the engineering team to understand where to correctly apply each tool without compromising the stack because of an apparent ease.

Within the various message brokers, there are some that stand out more, such as:

• ActiveMQ
• RabbitMQ
• Kafka

Let us understand the functioning of each of them a little better.

ActiveMQ

ActiveMQ is extremely traditional and very experienced. For years it was like the standard message bus in the Java world. There is no doubting the maturity and robustness of ActiveMQ.

It supports the programming languages used in the market. The ActiveMQ problem is related to the most common communication protocol, STOMP. Most mature libraries use ActiveMQ STOMP, which is not one of the models of sending more message performers. The ActiveMQ has been working on OpenWire for a solution in place of STOMP, but so far it is only available for Java, C, C++, and C#.

The ActiveMQ is very easy to implement, has been undergoing constant evolution, and has good documentation. If our application, the news portal, was developed on the Java platform, or any other that supports OpenWire, ActiveMQ is a case to be considered carefully.

RabbitMQ

RabbitMQ uses AMQP, by default, as the communication protocol, which makes this a very performative tool for the delivery of messages. RabbitMQ documentation is incredible and has native support for the languages most used in the market programming.

Among the many message brokers, RabbitMQ stands out because of its practicality of implementation, flexibility to be coupled with other tools, and its simple and intuitive API.

The following code shows how simple it is to create a system of Hello World in Python with RabbitMQ:

# import the tool communicate with RabbitMQ 
    import pika 
    # create the connection 
    connection = pika.BlockingConnection( 
      pika.ConnectionParameters(host='localhost')) 
    # get a channel from RabbitMQ 
    channel = connection.channel() 
    # declare the queue 
    channel.queue_declare(queue='hello') 
    # publish the message 
    channel.basic_publish(exchange='', 
                      routing_key='hello', 
                      body='Hello World!') 
    print(" [x] Sent 'Hello World!'") 
    # close the connection 
    connection.close() 

With the preceding example, we took the official RabbitMQ site, we are responsible for sending the message Hello World queue for the hello. The following is the code that gets the message:

    # import the tool communicate with RabbitMQ 
    import pika 
  
    # create the connection 
    connection = pika.BlockingConnection( 
    pika.ConnectionParameters(host='localhost')) 
 
    # get a channel from RabbitMQ 
    channel = connection.channel() 
 
    # declare the queue 
    channel.queue_declare(queue='hello') 
  
    # create a method where we'll work with the message received 
    def callback(ch, method, properties, body): 
      print(" [x] Received %r" % body) 
 
    # consume the message from the queue passing to the method 
          created above 
    channel.basic_consume(callback, 
                          queue='hello', 
                          no_ack=True) 
 
    print(' [*] Waiting for messages. To exit press CTRL+C') 
 
    # keep alive the consumer until interrupt the process 
    channel.start_consuming() 
 

The code is very simple and readable. Another feature of RabbitMQ is the practical tool for scalability. There are performance tests that indicate the ability of RabbitMQ in supporting 20,000 messages per second for each node.

Kafka

Kafka is not the simplest message broker on the market, especially with regards to the understanding of its inner workings. However, it is by far the most scalable and is message broker performative for the delivery of messages.

In the past, unlike ActiveMQ and RabbitMQ, Kafka was not sending transactional messages, which could be a problem in applications where losing any kind of information was a high cost. But in the most recent versions of Kafka this problem has been solved, and currently, there are transactional shipping options.

In Kafka, the numbers are really impressive. Some benchmarks indicate that Kafka supports, without difficulty, over 100,000 messages per second for each node.

Another strong point of Kafka is the ability to integrate with various other tools like Apache Spark. Kafka has good documentation. However, Kafka does not have a wide range of supported programming languages.

For cases where performance needs to reach high levels, Kafka is more than a suitable option.

Note

For our news portal, we adopt RabbitMQ due to compatibility and the good performance that the tool has, compatible with the current situation of our application.