Good to see Ben Mann has a new blog. You may know Ben as the Product Manager for WebSphere MQ but his blog is all about the new Managed File Transfer product being developed in Hursley Labs. For years people have been asking about moving files over WMQ and many have built applications to do just that. But it is not as simple a task as it sounds. If you put one file per message there is no reassembly required but you are limited to 100mb files. On the other hand you can choose to segment the file into multiple messages but then you have to reassemble the segments and they may arrive out of order or some may not arrive at all. Either way there are questions of how to deal with all the OS-level issues such as file permissions, path traversal, whether to delete the source file or overwrite the target, etc. Agents on either end have to deal with these issues and then somehow float exceptions from the remote node back up to the local end user in a time frame and format that .
I’ve seen dozens of different implementations of file transfer over WebSphere MQ over the years. No two have been alike and none were fully realized in their feature set, their user interface or their instrumentation. Each implementation I’ve seen so far has been constrained to the very narrow requirements of a specific business or a specific project within that business. But for the most part the implementations that I’ve seen have either been quick-and-dirty prototypes that got thrown into production and never completed, or they have locked their developers into the (usually unfunded) long-term task of maintaining the application and extending it throughout the enterprise. My exposure tends to be skewed toward the ones with problems of course, and by no means am I trying to say that nobody out there has done a good job at this. But many of the folks I’ve talked to have come to the conclusion in hindsight that they would have been better off buying than building.
So I’m excited to see what the folks in Hursley will come up with. The new product that Ben is working and blogging on is called WebSphere MQ File Transfer Edition and it is scheduled for release in fourth quarter 2008. It will be extensible and scriptable, it will allow files of any arbitrary size and it will be secure and auditable. The UI will be based on Eclipse so I expect it will look and feel much like the tooling for WebSphere MQ or WebSphere Message Broker. If you have any interest in such a product, drop by Ben’s blog and give him a shout out.