Representational State Transfer (REST)–enabled HTTP programming platforms broaden the client base that can be used to interact with services beyond those that are purely based on the Microsoft .NET Framework or that support SOAP. The Azure AppFabric Service Bus Durable Message Buffers that are broadened in this way provide platform-level support for reliable and replicated message queuing.
In this article, I examine the updated version of the Durable Message Buffers feature as it applies to the Windows Azure AppFabric Community Technology Preview (CTP) February 2011 release. I will show the feature's capabilities and how to take advantage of them from REST-based .NET clients in a fashion that translates well to other REST-enabled HTTP programming platforms such as Silverlight, Flash, Ruby, and even web pages.
To take advantage of the new release, you need to access the Labs portal. Also, you need to download Windows Azure AppFabric SDK v2.0, samples. Optionally, you can download the offline CHM Help file. You might also consider using the RestMessageBufferClient class provided within the samples as a starting point for your own .NET helper classes because the SDK libraries currently do not include helper classes for a message buffer.
AppFabric Durable Message Buffers
The AppFabric Durable Message Buffers component gives you a message queuing service that uses replicated storage and provides an internal failover feature for increased reliability. That the component is exposed to a REST operation means that it is available to a large swath of clients.
Generally, the service can be divided into two major feature sets: management and runtime. The Management feature set lets you create, delete, get a description of, and list buffers. The Runtime feature set lets you send, retrieve, and delete messages. These features are accessed and secured separately by making requests to the following Uniform Resource Identifiers (URIs):
- Management Address: http(s)://{serviceNameSpace}-mgmt.servicebus.appfabriclabs.com
- Runtime Address: http(s)://{serviceNameSpace}.servicebus.appfabriclabs.com
When you first provision your account, you select the service namespace to use, and that single namespace is used by both management and runtime operations. Interaction for both management and runtime centers on REST, where requests and responses comply with the HTTP/1.1 spec. Figure 1 shows a high-level flow of messages across the Durable Message Buffer.
Figure 1: AppFabric Durable Message Buffers
In the current CTP, the maximum message size is 256KB. Each buffer can store up to 100MB of message payload data, and there is a limit of 10 buffers per account.
Using the Durable Message Buffers
To use the service, you need to create a namespace through the management portal. With a namespace in place, you can then programmatically manage buffers and interact with them.
I will examine the steps in detail to set up a namespace, create a buffer, send messages to the buffer, retrieve messages from the buffer, and delete the buffer. I will also briefly cover related functionality for how to get a buffer's description, list all buffers created under an account, perform atomic read/delete operations, and unlock messages that were examined but were not processed.
Namespaces, Service Accounts, and Tokens
To start, you need to create a namespace that scopes the buffers you will create. To do this, log on to the Windows Azure AppFabric Labs Management Portal, and click New Namespace. The Create a new Service Namespace dialog box appears. You will need to select an Azure subscription and provide a globally unique service namespace, region, and connection pack size. For this CTP, the region and connection pack size options are fixed, which Figure 2 shows.
Figure 2: Creating a namespace
With a namespace in place, you need to look up the account username and password that you use to access the management and runtime features. Access to a namespace is controlled by the AppFabric Access Control Service (ACS). But when you create a namespace, a default account is already created for you to use. In the AppFabric Labs Management Portal, select the namespace, and examine the Properties tab on the right side for Default Issuer (this is the username) and for Default Key (this is the password). For an example of the Properties tab, see Figure 3.
Figure 3: Namespace properties
Or, you can create your own service accounts for management and runtime. See the Creating Custom Service Accounts section for more information about how to do this. With this information in hand, you are now ready to start coding!
Before you can perform any management or runtime operation, you must authenticate with the ACS and get a token for the management or runtime operation, based on the operation you want to perform. This amounts to making a POST request to
https://{serviceNameSpace}-sb.accesscontrol.appfabriclabs.com, by using the username and password that you previously obtained from the namespace and a scope value that is determined by whether you are authenticating to perform a management or runtime operation. Here's the format of the scope:
- Management address: http://{serviceNameSpace}-mgmt.servicebus.appfabriclabs.com
- Runtime address: http://{serviceNameSpace}.servicebus.appfabriclabs.com
Figure 4 shows a complete example of how to obtain either a management or runtime token from ACS.
Create Your First Message Buffer
Now that you have a management authorization token, you can create a new buffer. To do this, make a POST request, as the table in Figure 5 shows, to the management URL with a special payload called a message buffer description.
Figure 5: Creating a message buffer
This creates a new buffer as defined by the provided message buffer description AtomPub document. If the buffer already exists, the response status will be 409 - Conflict.
The message buffer description is an AtomPub document that takes the form as shown in Figure 6 and that basically provides the buffer's name, URL, authorization policy, transport protection policy, and message lock duration.
The authorization policy can be set to NotRequired, Required (this is the default), RequiredToReceive, or RequiredToSend. The policy determines whether an ACS token is required in the Authorization header for runtime operations. When the token is required, the request must use Secure Sockets Layer (SSL). The transport protection policy supports the AllPaths value or the None value (this is the default) to determine whether SSL is always required or if it is optional. Finally, the lock duration property is set to the time, in seconds, that a message lock can be held before it is automatically returned to the queue and made available to other clients. This value is capped at five minutes. On success, the response body to this POST contains a complete message buffer description, and the server fills in the various defaults, as Figure 7 shows. Figure 8 shows how to create a message buffer in .NET.
Sending Messages
Now that a buffer is available, you can send messages to it by a POST operation, as the table in Figure 9 shows.
Figure 9: Sending a message
This enqueues a message into the specified buffer. It's important to be aware that, while the maximum buffer size is 100MB, there is no message quantity limit, although messages themselves are limited to 256KB. Messages are rejected if they will cause the operation to exceed either of these quotas.
For the request, the Content-type header must have the value of text/xml;charset=utf-8. Figure 10 shows how to send a string payload to the message buffer. The most important thing is to SOAP-encode the payload by using a MessageEncoder.
Retrieving Messages
Now that you have a message in the buffer, how can you access it? There are two approaches: You can atomically read and delete a message from the buffer, or you can use a peek/lock approach, which places a temporary lock on the message. For our purposes, let's examine the peek/lock approach.
To perform a peek/lock, you perform a POST. This retrieves the next message from the queue and places a lock on it for the duration that was specified during buffer creation. During this period, no other readers can see the message. After the message has been successfully consumed, the client should delete the locked message. If the client does not delete the locked message within the specified period or explicitly unlock it, the message is again visible in the queue, as the table in Figure 11 shows.
Figure 11: Peek/lock message retrieval
In the request, the timeout is an XML duration for which the request will wait for an available message. The timeout can range from 0 seconds (which returns a result immediately if no messages are available) to 120 seconds. Figure 12 shows how to perform a peek/lock operation and capture the message location and the lock-id and lock-location header values. These header values are needed to either unlock or delete a message. See how the scheme is changed from sb:// to http:// for the message location and lock location URIs. (This is the format that any successive delete operation expects.)
Deleting a Locked Message
After successfully processing a locked message, the caller needs to delete the locked message, or it will be returned to the queue and made available to other clients. This deletion is performed by a DELETE operation that specifies the message location, lock-id, and lock location (information that the peek/lock operation acquired) of the locked message, which the table in Figure 13 shows.
Figure 13: Deleting a locked message
A sample implementation to delete a locked message is shown in Figure 14.
Unlocking Messages and Atomic Read/Delete Operations
A locked message may be returned to the queue by using the Unlock operation. In addition, there is an atomic Read/Delete operation that returns the value of a message and deletes it from the queue. To perform both of these tasks, you make a DELETE request, as structured in the table in Figure 15.
Figure 15: Unlock and atomic read/delete operations
In Figure 15, the code to perform an unlock operation is similar to the code shown to delete a locked message. And, the atomic read/delete operation is similar to the code shown to retrieve a peek-locked message.
Deleting and Retrieving Buffers
When you will no longer use a buffer (remember that the CTP has a quota of 10 buffers per account), you can delete it. To do this, invoke a DELETE operation that names the buffer, as shown in the table in Figure 16.
Figure 16: Deleting a buffer
Figure 17 shows the complete code for deleting a buffer.
In addition to creating and deleting buffers, the management features also let you retrieve a list of all buffers under a namespace and the message buffer descriptions. You might use these operations to check whether a buffer is created before you try to write to it. See the table in Figure 18 for the request and response details.
Creating Custom Service Accounts
Access to management and runtime functionality is secured through the ACS, which, given a particular account, returns one or more action claims that have the values of Send, Manage, or Listen, according to the rules defined for the account. By default, the owner account receives all three claims.
If you want to alter these claims, create a new runtime or management service account. You can do this from the ACS labs management portal. To access the ACS portal from Azure AppFabric Labs Management Portal, select a namespace, and click the Access Control Service button. To create a new Runtime Service Account within the ACS portal, click Service Identities, then click Add. You must provide a name for the identity and then add a password credential.
Next, configure which operations the accounts are allowed to perform (read, post messages, and manage buffers). To do this, you can add a custom rule group that defines one rule for each operation that a user can perform. And in each rule, you match the input claim value to the username, then you output a claim of the type net.windows.servicebus.action that has a value of Listen, Send, or Manage.
Finally, to associate this rule group with the management or runtime scope, select the Relying Party Applications link, then select either the Service Bus or the ServiceBusManagement application. To create the association within either of these applications, click to select the check box next to the rule group name you just defined, then click Save.
How to REST Easy
Durable Message Buffers give you a portable way to reliably queue work between different clients and services, helping you to REST easy! You can find more information about this topic in the MSDN references API Reference (Durable Message Buffers) and AppFabric Access Control.
Zoiner Tejada ( [email protected]) is passionate about workflow and the future of implementing connected systems with them. He's the president of TejadaNET, is recognized as a Microsoft Industry Influencer, and is an advisor to Microsoft's Connected Systems Division. Zoiner has a degree in computer science from Stanford University and blogs at TheWorkflowElement.com.
