Message Queue Products - Compared

Posted 12. February 2021. 8 min read.
#MQ#integration#data#message queue#RabbitMQ#Apache Kafka#Google Cloud Pub/Sub

Message Queue Products

The majority of enterprise applications are built on cloud architecture. The applications are often built in small blocks and then aggregated into a large block using message queuing. Instead of developing one large application, it's beneficial to build smaller parts and communicate them asynchronously using message queues. This method simplifies coding while improving scalability, reliability, and performance.

Message queues automate communication-related tasks and coordinate distributed applications. They are used to aggregate and distribute workloads when building large scale applications. Also, they enable process-related communication between IT systems by processing, storing, and deleting actions.

Messages can consist of alerts, replies, or requests. The queuing platform stores messages before forwarding them to the intended recipients.

Message Queue Architecture

Message queues provide an asynchronous communication protocol. Its architecture consists of destination or server services known as consumers and client applications known as producers.

The process begins when a producer creates messages and sends them to the queue. The consumer retrieves the message from the queue and processes them. If the consumer doesn't retrieve the message, it remains in the queue.

Ideally, message queues operate similarly to traditional postal services.

Best Message Queue Products

There is a wide range of message queue products in the market that help in building applications. They include;

Google Cloud Pub/Sub

The Google Cloud Pub/Sub is a fully managed messaging service that allows communications between applications. It holds data for up to 7 days and guarantees the delivery of messages. Pub/Sub servers run in Google regions throughout the world.

What Protocol Does Google Cloud Pub/Sub Support?

This messaging queue product features both gRPC and the standard HTTP/REST APIs. These APIs give developers the option to write and use their own client codes.

Its protocol is an active/outbound protocol that collects Google Cloud Platform logs. Google Cloud Pub/Sub integrates with tools such as;

  • Google IoT Core
  • Google Cloud Source Repositories
  • Firebase
  • OSS integrations

Synchronous vs. Asynchronous Support

This queue provides support for asynchronous processes.

What Standard Do They Conform To?

Google Cloud Pub/Sub uses a publish/subscribe mechanism for communications. Its design supports both push-and-pull message delivery models.

For the push-based delivery, it initiates requests to the subscriber application to push messages. In the pull-based delivery, the subscriber application sends a request to the server to retrieve messages.

Pros

  • Highly scalable
  • Easy-to-use APIs
  • Simple UI

Cons

  • Poor documentation
  • Uses outdated technologies with limited extensions

IBM MQ

IBM MQ is a versatile queuing product with a wide range of capabilities. It transports any data, enabling businesses to build reusable and flexible architectures. This messaging queue product works with a wide range of applications and computing platforms for security-rich message delivery.

IBM MQ offers control over the flow of messages as well as communications visibility. Its development tools support business growth and extensibility.

With IBM MQ, applications have the choice of programming languages and interfaces.

What Protocol Does IBM MQ Support?

Protocols in IBM MQ Radar provide data collection using various connection options. These connections can passively receive data or pull the data back into the QRadar.

The connection options that receive data include SNMP, HTTP Receiver, and Syslog. SCP, SFTP, FTP, and JDBC pull data into the event pipeline.

APIs directly supported by IBM include:

  • XMS for C/C++ and .NET[2]
  • Java Message Service (JMS)
  • IBM MQ for Java, C++, COBOL, and C
  • .NET
  • Representational State Transfer

Synchronous vs. Asynchronous Support

IBM MQ provides asynchronous messaging. All messages are held in the queue in case of an application or network disruption.

Pros of IBM MQ

IBM MQ offers a wide range of capabilities that safely move communications between applications. Some of the perks of IBM MQ include;

Stability. For long, this message queuing product has proved secure and stable. It doesn't encounter incidents of hacking or dropping.

High Performance. Besides, it accommodates huge volumes of transactions hence increasing the organization's transaction capacity. It's programmer-friendly and provides JMS and MQ-specific access to essential functions.

Reliable. As long as the queue managers are set up correctly and the client code is properly written, IBM MQ handles whatever you throw at it.

Cons of IBM MQ

Does Not Send Notifications In Case of Failures. This queuing provider lacks a reporting feature. It does not send alerts in case of failures.

Lacks Good GUI for Linux. While it's only present in the windows interface, Linux users have to develop scripts and execute commands.

Learning Curve. To use IBM MQ effectively, you need to learn how it operates. Besides, it requires a sort of high-end expertise.

RabbitMQ

Rabbit MG is a popular message queuing platform that supports a wide range of messaging protocols and developer platforms. It's a messaging broker that runs on all major operating systems.

With RabbitMQ, you can define queues and push messages. It offers a point of interface or contact between the producer and consumer.

What Protocol Does Rabbitmq Support?

Rabbit MQ has a natively built-in Advanced Message Queuing Protocol (AMQP) that's easy to integrate and implement. With the newer versions of AMQP, RabbitMQ has extended its support. STOMP is a messaging protocol based on text and is the only protocol that can be used over telnet, a client-server protocol.

RabbitMQ plugin supports all versions of STOMP. MQTT protocol receives support for lightweight subscription messaging and messaging semantics from RabbitMQ. AMQP 1.0 is one of the complex protocols supported by RabbitMQ plugins. For HTTP, RabbitMQ can transmit messages even when it is not a messaging protocol.

Synchronous vs. Asynchronous Support

RabbitMQ supports asynchronous messaging, which means the applications do not have to wait for a real-time response.

Pros of Rabbitmq

Great UI. Rabbit MQ has an easy-to-use interface that allows you to control and monitor every aspect. It collects data and metrics and exposes them in the UI for alerting, visualization, and storage.

Flexible Routing. Before arriving at queues, messages are routed through exchanges. It has built-in exchange types that facilitate routing. In case of complex routing, developers can write their own exchanges as plugins or bind together exchanges.

Multiple Protocols. RabbitMQ supports a wide range of protocols, either through the use of plugins or directly. Some of the supported protocols include MQTT, STOMP, AMQP 0-9-1, and HTTP.

Cons

Lack of Observability. If you want to see the messages in the queue, you would have to re-queue them. Taking messages out to re-queue can lead to loss and break the order of messages.

Apache Kafka

Apache Kafka is an open-source message queuing platform built purposely for pipelining, distributed streaming, and fast replay of data feeds. Kafka can subscribe to, publish, and process data streams in real-time. It moves massive amounts of data from any point to wherever it's needed at a go.

Apache Kafka supports use cases where scalability and throughput are crucial. By minimizing point-to-point integrations, it reduces latency. Data reaches users fast enough to improve efficiency, especially in use cases that demand real-time data.

What Protocol Does Apache Kafka Support?

Apache Kafka uses binary protocol over TCP. However, it does not support messaging protocols. It doesn't support non-based TCP protocols such as JMS or MQTT.

Synchronous vs. Asynchronous Support

Apache Kafka offers inherently asynchronous integration to make development more agile. It streams data from development teams and populates it in a data store such that the data can be shared between multiple applications.

Pros of Apache Kafka

Scalability. Kafka is designed to allow you to handle huge volumes of data. It can upscale or downscale at any time.

High Performance. Even with extreme loads of data, Kafka maintains the same level of high performance. It can perform millions of writes per second.

Cons

Doesn't hold historical data. While Kafka is great for streaming, it doesn't store historical data. It's only ideal where data gets consumed very quickly.

Limited Flexibility. Kafka doesn't allow you to filter data in messages hence limiting its flexibility.

Comparison Between Message Queue Products

Having compared the products above, it's essential to have a side-by-side comparison. Here are detailed comparisons touching on messaging patterns, supported languages, and multiple protocols;

Multiple Protocols Supported Languages Message Patterns
Google Cloud Pub/Sub FTP, TCP/IP, UDP Java, PHP Python, and C# Push/Sub
IBM MQ SOAP, AMQP VB.NET, Java, C++, ESQL Point-to-point Pub/Sub
Rabbit MQ STOMP, MQTT, HTTP, AMQP Supports Java, Javascript, Python, C++, C, PHP, Haskell, Rust Message Queue, RPC like Req-Rep, ROUTING, Pub/Sub; but not at the same.
Apache Kafka Does not supportmessage protocols Supports Ruby, Node.js, C#, PHP, Python Pub/Sub

The Bottom Line

While many different options are widely available, make sure you take the time to examine the options and take careful stock of what each messaging queue product does or does not offer. This guide presents a detailed overview that can help guide your approach to choosing the product that best serves your needs.

© 2021 Derek Muensterman
Made with
☕ 💖 🍕 ☕ 💖