Introduction to Message Brokers
Message brokers are programs that enables services to communicate with each other and exchange information. If you already know about APIs, you may think that message brokers are same as APIs. But it’s not, the message broker usually does this by implementing a queue, that the different services can write to or read from. So, this will allow these services to asynchronously communicate with each other, even if they were written in different languages or implemented on different platforms.
So, these message brokers behave as a bridge between applications, allowing senders to publish messages without knowing where the receivers are or how many of them there are.
Use cases of Message Brokers
As you already guessed, message brokers are widely used in software development. They’re useful whenever reliable multi-services communication, secure message delivery, or asynchronous features are required. Now, I will show you several use cases of message brokers.
- If you have to develop a payment processing system, it’s important that payments are sent only once. Handling these transactions using message brokers ensures that payment information will neither be lost or duplicated and provides proof of receipt.
- Assume like there is a task, that send data to several applications and have to avoid direct use of their API. In such case, message brokers can save your day.
- Also, message brokers come in handy when you want to control data feeds. For example, the number of registrations in any system.
Basic components of Message Broker
There are 4 components in message brokers that you need to know.
- Producer — This is the endpoint which sends any kind of data, that is stored inside the message broker to distribute.
- Consumer — This is the endpoint which asks from the message broker for messages and consume it.
- Queue — This is a data type which the message broker use to store messages inside, with the logic of FIFO(First in First out). Queues stores the messages until a consuming service processes them. These queues also allow asynchronous type of programming, since the queues are the ones responsible for delivering the messages, the sender can continue performing different tasks.
- Exchanger — This is known as a logical configuration or even entity, on top of the queues, which tells the message broker to create some sort of a group, which a consumers can listen to the receiving messages or producers can write to send messages.
Types of Message Brokers
There are two basic forms of communications with a Message Broker:
- Point-to-Point (Queues)
- Publish and Subscribe (Topics)
Point-to-Point (Queues)
In this pattern, the communication happens between only one producer and one consumer. So basically, it’s a one-to-one relation set between the producer and the consumer of the message. Each message is sent and consumed only once. This style of messaging generally uses a queue to store messages sent by the producer until they are received by the consumer. We can use this pattern for example, when some action needs to be performed only once.
Publish and Subscribe (Topics)
Publish and Subscribe pattern is slightly different from the previous one. In this pattern, the producer is known as a publisher and the consumer is known as a subscriber. The publisher of the message doesn’t know anything about subscribers and the message is being sent to the topic. So, one or many publishers can publish on the same topic, and a message from on or many publishers can be received by many subscribers. We can use this pattern for implementing notifications mechanism or distributing independent tasks.
Examples of message brokers
Today there are many message brokers out there, but the most popular ones are RabbitMQ, Apache Kafka, and Redis. So, let’s go through these message brokers briefly.
Apache Kafka
This message broker was originally developed by LinkedIn to handle high throughput and low latency processing. These required a massive load of data for a long period of time and that’s what Kafka is good at. It’s heavily used in real-time streaming data analysis applications because of its integration with Apache Storm and Spark. If you compared Kafka to other message brokers by their performance, Kafka is usually on the top.
Kafka supports all major languages, including Python, Java, C/C++, .NET, PHP, Ruby, JavaScript, Go, Swift and more. Also, it is a perfect choice for big data use cases since it can achieve high throughput with limited resources.
In case if you want to know how popular Kafka is, all the tech giants including Netflix, eBay, Uber, PayPal, and Pinterest are using Kafka as their message broker.
RabbitMQ
RabbitMQ is the most widely deployed and popular open-source message broker. It was originally released in 2007 and is one of the first common message brokers to be created. RabbitMQ delivers messages through both point-to-point and pub-sub methods by implementing Advanced Message Queuing Protocols (AMQP). It’s designed to support complex routing logic.
This message broker comes with a plugin named as, “Management plugin” and it helps users to operate RabbitMQ using it through a graphical user interface, and facilities like viewing different statistics related to messaging. Also, it can keep an overview of all of the operations and data happening in queues.
RabbitMQ also supports all major languages, including Python, Java, .NET, PHP, Ruby, JavaScript, Go, Swift, and more.
Redis
Compared to other message brokers, Redis is bit different due to it is an in-memory data structure store. Because of that it can be used as either a high-performance key-value store or as a message broker. But the thing is messages are not guaranteed to be durable there.
There is another difference in Radis, that it has no persistency but rather throws its memory into a disk or db. However, it’s a good thing for real-time data processing. Originally, Redis was not one-to-one and one-to-many. But since Redis 5.0 introduced, it supports the one-to-many option.
So, this is the end of the article and I hope you enjoyed it. Happy Coding👨💻.
References
Samofal, V., 2019. Introduction to Message Brokers. Part 1: Apache Kafka vs. RabbitMQ — DZone Big Data. [online] dzone.com. Available at: <https://dzone.com/articles/introduction-to-message-brokers-part-1-apache-kafk> [Accessed 14 May 2021].
Łuczak, B., 2021. Message broker — complete know-how, use cases and a step-by-step guide | TSH.io. [online] The Software House. Available at: <https://tsh.io/blog/message-broker/> [Accessed 15 May 2021].
