The Windows Azure AppFabric Service Bus provides connectivity for disparate components within a distributed application. In its current incarnation, which at the time of this writing is the Service Bus September 2011 Release, the Service Bus actually offers three distinct connectivity technologies: the Service Bus Relay, Service Bus Message Buffers, and the new Service Bus Brokered Messaging. The Service Bus Relay enables clients and services to communicate independently of network address translation (NAT) and firewall obstacles via a cloud-hosted relay service. The Service Bus Message Buffers enable transient, in-memory, queue-like connectivity for components. Message Buffers have been deprecated in favor of the more robust messaging feature set found in Service Bus Brokered Messaging. In a nutshell, Service Bus Brokered Messaging provides a messaging infrastructure that enables communication via durable queues. In this article, we will focus on the core concepts of Service Bus Brokered Messaging.
Introducing Queues
At the heart of Service Bus Brokered Messaging lies the queue. A queue provides a first-in-first-out (FIFO) structure for transmitting messages between one (or more) message senders and one or more message receivers. Figure 1 shows a single sender transmitting a number of messages to a single queue. Each message is retrieved at most once, by a single receiver -- shown as Client A getting Msg 1, Client B getting Msg 2, and so on. The messages themselves contain both headers (also called message properties) and a body, where the body is simply a UTF-8 encoded byte array.
Figure 1: A Service Bus queue supporting multiple receivers
This first pattern of retrieval is known as Receive and Delete, because the message is retrieved and deleted from the queue in a single atomic operation. The Service Bus also supports at-least-once message-delivery semantics via the Peek/Lock and Complete retrieval pattern. In this approach, a client that receives a message must explicitly complete the message before it is actually deleted from the queue. If the client does not do so within the configurable time window, the message reappears in the queue for another client to receive it. This helps guard against message loss in scenarios where a client retrieves a message but crashes while processing the message.
Touching on Topics
The feature-rich cousin of queues is the topic. Topics receive messages from senders just as queues do; where topics differ is in how clients receive messages from them. Instead of receiving messages directly from the topic, clients receive messages from something akin to a virtual queue (or -- for readers with database-inclined minds -- a "view" on top of the topic) known as a subscription. Just as for a queue, a single subscription can have multiple clients retrieving messages from it, as shown in Figure 2. In addition, subscriptions support both Retrieve and Delete as well as the Peek/Lock and Complete retrieval patterns.
Figure 2: A Service Bus topic supporting multiple subscriptions
The message processing is generally still FIFO, except that each subscription defines filters that control what messages actually appear via the subscription. A filter defines a condition expression using an SQL-92 syntax (e.g., "MyProperty LIKE 'Z%'") that can examine the values of message headers in deciding whether or not to include the message in the subscription.
A filter can be configured to accept all messages, no messages at all, or only messages that match the condition expression. By default, each subscription receives every message sent to the topic unless a filter that says otherwise is configured. In Figure 2, we see that Client A receives only Msg 2 because Subscription A contains a filter that excluded the other messages. Note that in the absence of a filter, a single message enqueued to a topic is multicast to every subscription -- in other words, a single message can be received multiple times. In Figure 2, this is why both Subscription A and Subscription B receive Msg 2.
In addition to supporting filters, a subscription can also define actions that add, remove, or modify the value of message headers. In the object model, a filter and action are defined within a rule. In this way, a rule can be defined that is used just to filter out messages (by defining only a filter), or that is used to selectively alter message headers (by defining both a filter and an action). A single subscription can have multiple rules associated with it.
There is also a special type of filter that examines the value of the CorrelationId message property for an exact match; this is called a correlation filter. This filter is better optimized for matching performance than the SQL expression-based filters and is the recommended approach if your conditions involve only a simple equality test.
Additional Features
Service Bus Brokered messaging offers an assortment of rich features that may help you in many messaging scenarios beyond queues, topics, subscriptions, and rules. Each of these scenarios warrants an article on its own, so I will introduce each one briefly and encourage you to read more about them online by following the links in the Additional Resources box at the end of this article.
- Sessions: Messages sent to queues and topics can be marked as part of a session, which clients can then retrieve in a contiguous sequence. This enables additional features such as support for sending messages larger than the allowed message size, by chunking messages and sending them within the same session. Sessions also enable receivers to respond to specific senders in a Request/Reply messaging pattern.
- Transactions: The Complete (of the Peek/Lock and Complete retrieval pattern) and Send operations can participate within a local transaction -- ensuring, for example, that a message received is deleted from a queue only after a related message has successfully been sent to another queue. A failure to either Complete or to Send means the received message will reappear in the original queue and the new message is never sent.
- Duplicate detection: This feature ensures that messages having a duplicate MessageId property (of a previously sent message) are not enqueued to a queue or topic within a configurable window of time.
- Batch operations: Clients can opt to pre-fetch a group of messages (instead of just the next message) with every receive request, with retrieval semantics identical to Peek/Lock, in order to maximize the throughput of every receive request. Senders, when using the async versions of the Service Bus Brokered Messaging APIs, can batch messages up to a small amount of time and/or size in the hopes of sending multiple messages in a single request. This also applies to Complete requests.
- Deferred message processing: This feature enables receivers to "shelve" a message and retrieve it later by using a unique ID.
- Dead lettering: The Service Bus can move messages to a system-provided dead-letter sub-queue when the messages have reached their max time to live or max number of retrievals.
Robust Azure Messaging
Having read this overview, you should now have an idea of how to use Azure's Service Bus Brokered Messaging in a variety of scenarios. From its support for publish-subscribe scenarios enabled by topics and subscriptions to the temporal decoupling and load leveling between senders and receivers inherently enabled by queues and topics with subscriptions, Service Bus Brokered Messaging offers a robust messaging solution for connectivity between your application's components.
