Friday, August 8, 2008

AMQP enthusiasm?

All existing messaging solutions (WebSphereMQ, JMS, ...) use proprietary protocols. This is not a problem within a single organization. But between organizations, standard protocols are needed. Therefore, the B2B world uses protocols such as AS2, RNIF (RosettaNet) or good old (S)FTP(S).

AMQP is an initiative to bring a standard binary wire protocol to the messaging world. Just like POP3+SMTP allows you to retrieve and send emails using whatever email server, AMQP will allow any AMQP client to receive and send messages via any AMQP compliant server.

But when I read the spec, AMQP is focusing on the client-server protocol, contrary to SMTP that is (also) used for communication between mail servers. The AMQP spec states that a bridge should be used for server-2-server communication, but doesn't provide any details. As such, AMQP is focusing on messaging within the corporate firewall.

AMQP can be used for unbalanced B2B scenario's, where one side runs the AMQP broker. This is a setup similar to one big company or intermediary running an (S)FTP(S) server and smaller organizations putting and retrieving files from it. But for good decoupling, server-to-server communication is preferred. The server at the sending side will take care of delivering the message to the server at the opposite side. Like e.g. AS2 does: once an organization has an AS2 server in place, it becomes equal to all its AS2 counterparts.

With all this in mind, I was a bit puzzled by Paul Fremantle's enthusiasm about AMQP. In particular because he is the WS-RM spec lead.

WS-ReliableMessaging should have brought reliable async messaging to the WS-world. But it didn't. The WS-RM spec doesn't mention message persistence and so (most) vendors have an in-memory implementation, which is not reliable.

I still remember going through the book "Programming Indigo" and learning about the ReliableSessionEnabled binding property. What a disappointment to learn that for real reliability, one had to use the MsmqIntegration Binding and thus the proprietary MSMQ transport layer.

5 comments:

Matthew said...

Guy,

For context, I chair the AMQP User SIG and we appreciate your comments and insight.

With regards to the current spec, 0-10, you are correct that it does not address broker to broker federation. For many of us involved, this is an essential aspect of the protocol that has to be in place before it is a viable Internet scale solution.

But we have to deliver in steps. Please see to the AMQP User Requirements for a more complete picture of the scope of the protocol and the specification roadmap.

http://jira.amqp.org/confluence/display/AMQP
/AMQP+1-0+Business+Requirements

A refinement of these requirements under review can be found at:

http://jira.amqp.org/confluence/display/AMQP
/AMQP+1.0+Business+Requirements+-+Revised

Your comments and feedback are greatly appreciated.

It is worth noting that both openAMQ(zeroMQ) and RabbitMQ have federation solutions functioning but as a specification team we have not turned our attention to standardizing the protocol.

Our aims is to delight, not disappoint - explicitly not with vendor lock in strategies.

Step by step we are getting there, especially through community review and support such as yours.

Matthew Arrott

Paul Fremantle said...

Guy

I've posted a response here:
http://pzf.fremantle.org/2008/08/amqp-beyond-firewall.html

alexis said...

Guy

Thank-you for your feedback on AMQP. I'm with RabbitMQ and would like to add to what Matthew and Paul said.

First of all, we have had a lot of interest in RabbitMQ and AMQP from all sorts of end users and customers. I mention this because then, the obvious question becomes: why do people care? I think there are three reasons: new use cases, integration, and cost benefits of using a protocol. While global addressing is in scope for AMQP as Matthew explains, it is not needed for any of those cases.

In terms of use cases you mention that WS-* does not provide an open reliable queue, and this also impacts things like .NET WCF which btw works with RabbitMQ. But there are many other cases as well for example providing messaging 'that just works' and is stable, and scalable, for emerging platforms such as Ruby. A graphic is here.

Giving people a messaging platform that is both open and actually works across multiple languages and cases, then delivers better integration. As Paul Fremantle pointed out on his blog, there are many companies who just need integration and lots of it, in house.

There is also a case to be made for the benefits of open protocols (compared to open APIs) in delivering true interoperability without costly customisation. Jeff Gould talked about this in his article. Protocols allow users to cope with many more integration cases (e.g. where timeout is involved) which again turns out to be very useful because it lets you manage more cases with one technology which is much cheaper as you scale up to many different scenarios.

On the B2B front, while federation would be very useful, we have also seen customers who want client-server B2B. For example exchanges, aggregators and utilities, who want the 'cheapest way to connect business messaging interoperably'. The key need here seems to be to connect to another party without requiring them to use the same product or same version of a product.

Last but not least we have had a stab at federation by integrating AMQP with XMPP. This has gotten a fair amount of interest from commentators (google for it...) but the original integration is described here.

Anyway - I hope this is helpful..

alexis

Pieter Hintjens said...

AMQP does in fact support very nice server-to-server messaging, based on the notion of chaining together exchanges in a client-server pattern. It requires no special protocol support.

OpenAMQ does this, and it's fully interoperable with other AMQP implementations. See the documentation of federation on http://www.openamq.org/doc:user-3-advanced.

The key is that the exchange-queue-binding semantic can be stretched across networks by making the server act as a client to other servers.

Federation is not properly described in the AMQP specifications because the editors are not really aware of how it works. But OpenAMQ has done this since its first release.

alexis said...

To add to what Pieter said, RabbitMQ does some server-server communication using the relay queue pattern.

We have also experimented with combining XMPP federation and addressing with AMQP routing and delivery. Feel free to contact us for details.