No waiting in these queues. IBM MQ V9 and the MQ Appliance M2001 delivers fast, reliable and secure message queuing

June 29, 2016

wile_e_coyote

Recent weeks have been pretty busy on this blog, reflecting just how busy the MQ development team has been in bringing out new and updated offerings in MQ V9 and the MQ Appliance M2001 here and here. And of course in our cloud messaging options.

As both of these have been fairly full of new content I thought I would do just a short update to focus on a couple of key benefits which are specifically measurable in these 2 refreshed offerings. After all, a lot of the new and improved features can sometimes be hard to quantify in terms of the benefits they provide, but in each offering this time there are some easy to define benefits.

As you may have seen in my most recent update, the MQ Appliance M2001 added large capacity SSD storage which enables much faster throughput for persistent messages. These are the messages that get written to storage to ensure they are still available in the case of failure before the message has been successfully deliver to all consumers. At high rates of message throughput, there can be a lot of contention for access to storage with traditional hard drives. With the new MQ Appliance M2001, this potential bottleneck has been removed. You can now read the latest MQ Appliance M2001 performance report here which shows that the performance in those scenarios which saw large volumes of persistent messages sees improvement of up to 3.5 times the previous message rate.

Clearly this represents a significant improvement and given that persistent messages are used in those business critical situations where IBM MQ delivers so much value, it is a hugely important benefit.

 

In MQ V9 there were a number of enhancements but the one I specifically want to call out is, as part of the MQ Advanced package, the enhancement to MQ Advanced Message Security (MQ AMS). The change here was to add a new mode of operation – Confidentiality. This new mode changed the way in which the encryption operations are performed on the message contents (MQ AMS offers policy based encrypted message contents which ensures data at rest is protected in case of a security breach). The goal of this change was to continue to offer a strong level of security for the message contents without too big of an impact on the performance and throughput from the effects of the encryption used.

Now instead of new asymmetric keys being generated for every exchange, the feature can be configured to allow for reusable symmetric keys to be used after the initial generation of an asymmetric key. This still provides a very high level of security, but depending on the reuse count before a new asymmetric key is generated, can drastically cut the performance overhead. The benefits can see more than an order of magnitude increase in throughput. You can see a quick snap shot of some of the early results in Jon Rumsey’s blog here – which includes a small table showing performance improvements exceeding 10x gains. With everyone concerned about security these days, the ability to better protect your information and customer data with little performance impact has to be a good thing.

 

So what are you waiting for? With secure, reliable enterprise messaging for on-premise deployments, cloud deployments or physical appliances, there is no waiting with IBM MQ V9 or IBM MQ Appliance M2001.

no-waiting

[An interesting history of Wile E. Coyote here]

Flash aaaahh – saviour of the universe: IBM MQ Appliance M2001

June 10, 2016

flash_gordon_facebook_cover_by_audrey41lorgeoux-d538dgo

Anyone who is a fan of cheesy sci-fi movies, or soundtracks by Queen will have the words “Flash, Flash I love you but we only have 14 hours to save the earth” running through their head, along with the line in the song that goes “Flash aaaaah, saviour of the universe”. And of course he did save the universe from Ming the Merciless.

But what if I told you Flash could also save your business? Not Flash Gordon of course, but flash storage, in the form of the SSDs that are now a part of the IBM MQ Appliance M2001 which is now generally available (June 10th 2016). We did cover this in an earlier blogpost, but I thought I would take advantage of our initial shipment date to cover just how critical the IBM MQ Appliance, backed by state of the art 3.2TB SSDs can be to your business.

MQ Appliance M2001

Each MQ Appliance M2001 model has 2 of the 3.2TB SSDs in a RAID 1 configuration. This means that every persistent message and all log data is written not just once to the SSD storage, but twice giving you complete redundancy of data. And a key part of the MQ Appliance functionality is the High Availability configuration – essentially nothing more than a simple menu option when creating a queue manager – allowing you to have the MQ queue manager on one MQ Appliance synchronously replicated to another MQ Appliance. This means that any message written to the SSDs on one MQ Appliance is not just copied to the second pair of SSDs but is written under the same unit of work that writes the messages on the first MQ Appliance. This therefore means you have 4 copies of the message stored for both reliability and availability.

Another part of the MQ Appliance update was the ability to do not just synchronous replication for High Availability but also asynchronous replication for Disaster Recovery to another MQ Appliance. Therefore you can point the MQ queue manager at another, typically off-site MQ Appliance and the same message will replicate there, ensuring there are another 2 copies of the message, and providing your business with a highly resilient messaging system designed to ensure optimum reliability and availability of messages.

After all, think about how important your messages are to your business. In effect, they are your business. Your messages are your business transactions, your new orders, your customer address details, your stock levels and distribution information. Lose your messages and you lose everything.

With the latest SSD technology inside the MQ Appliance you are calling on Flash to save your business – and with the MQ Appliance M2001, Flash saves the day again.

 

[Flash Gordon image title image above is from  http://orig07.deviantart.net/fc1d/f/2012/163/5/d/flash_gordon_facebook_cover_by_audrey41lorgeoux-d538dgo.jpg]

MQ in the Cloud – Your messaging ‘silver lining’

May 16, 2016

MQ clouds puttenham

As the senior product manager for a product like IBM MQ, I don’t just spend my day writing blogs – but frequently get questions from both colleagues and our many customers. And recently, one of the most common questions I get is whether MQ runs in the cloud.

The answer is “Yes” – that was easy wasn’t it.

However maybe there is some more information to share, to help describe the journey to cloud and to exploit the benefits provided by IBM MQ at every step, whether you are looking for enterprise grade business critical messaging in your private cloud infrastructure, in a public cloud (hosted or not) or a hybrid cloud spanning the combination of these.

 

Mostly the cloud environment that IBM customers I have been talking to are thinking about is best described as “Hybrid”. Almost all of our customers are starting to explore some aspects of the benefits of cloud – and what it means to them. But deployment, especially of business critical applications, is likely to happen in stages. Today many customers run virtually all application workload on-premise, but typically this will be in virtual machine environments. There is a shift to deploy selected workload in the cloud. Perhaps this might start with engagement or marketing applications, but these applications and the associated workload doesn’t run in isolation on the cloud but in conjunction with the rest of the enterprise running on-premise, or connecting to partners in the wider business ecosystem. Then as businesses shift some of their critical back office applications to cloud deployment options, the hybrid nature of infrastructure will increase.

Fundamental to the success of this change is the availability of reliable and secure connectivity to allow the safe and scalable exchange of information between applications independent of whether they are running together in the cloud, on-premise or any other combination.

How does IBM MQ work in this type of deployment? Well, as has been proven for more than 20 years, IBM MQ provides a way to exchange data in the form of messages between applications, systems and services and to do so reliably, securely, rapidly and simply. Messages are moved through MQ Queue Managers that can be deployed locally to the application – wherever that application may be, or remote from the application but accessed by MQ Clients bound to the application.

In pretty much any type of hybrid environment, MQ continues to be a critically useful middleware tool. Either the application running in the cloud environment can make use of the MQ Client to connect to a MQ Queue Manager running elsewhere (such as on-premise) or the MQ Queue Manager can itself be deployed in the cloud environment along with the application. And you might use your own tools to deploy MQ, or you could use tools such as Chef to deploy MQ. Other options for deployment include a MQ plug-in for IBM UrbanCode Deploy.

MQ offers support for running on IBM SoftLayer, Microsoft Azure, Amazon Web Services, and OpenStack cloud. It leverages a Bring Your Own Software License to make it simple for customers to choose where to deploy the IBM MQ license entitlements they may already have. For deployments of IBM MQ on SoftLayer you might choose to use the MQ Advanced pattern designed for IBM PureSystems which can run on the SoftLayer environment.

In addition to this deployment style, IBM recently confirmed MQ is supported to run in Docker environments which further extends where MQ might be deployed to meet customer needs, such as hosted PaaS environments like IBM Bluemix.

Docker_(container_engine)_logo

So basically for every cloud environment, or virtualised environment, or container, your business can continue to take advantage of the benefits of IBM MQ – whether by running MQ Queue Managers in the cloud environment, or continuing to run them on-premise (perhaps as the physical MQ Appliance), or in any hybrid combination.

So what are you waiting for? For MQ on cloud – the answer is yes.

*UPDATED to add link to the AWS example – see above*

IBM MQ V9 – A fast, secure, reliable and more agile MQ

April 19, 2016

edwin-moses-getty_2129850b

Some of you reading this blog may recall the great athlete Ed Moses – who had a record 122 race winning streak in just about the hardest event – the 400M Hurdles. You need to be strong, fast, and agile just to compete, and to keep winning you need to be reliable. Well, this is how we view IBM MQ, especially with the latest release – IBM MQ V9. You may have seen a recent blogpost on here that had a Statement of Direction talking about a new way of delivering IBM MQ – one that provided a Long Term Support release, and a Continuous Delivery release. The aim of this model is to give customers more choice to select either highly stable releases with just fixes, or releases that benefitted from additional function in the fixpacks.

TRY IT: Click here to get a free trial of MQ

UPDATE: There is a FAQ on the new support model. Read it here.

On April 19th, IBM announced MQ V9 which is the first release that moves to this new more agile delivery model. As such at the initial release it delivers a small set of additional capabilities that will be available to all customers. Then subsequent mod-level updates will deliver even more updates to customers choosing the continuous delivery stream, but all customers moving to V9 will get the benefit of the new capabilities being delivered in this release.

As with previous releases of IBM MQ, customers have a lot of choice in where and how they may want to deploy this version. IBM supports deployment of MQ – and MQ Advanced pretty much on every commercial IT environment where business critical applications may be exchanging data reliably, securely, and at scale. This could be on-premise, deployed in cloud environments like IBM Softlayer, Microsoft Azure or Amazon AWS. IBM also supports virtualization with many customers deploying in VM images, and also in Docker containers, which can be deployed anywhere, including in IBM’s Bluemix platform. This flexibility enables customers to make use of enterprise messaging to support deployments on-premise, on cloud or in hybrid environments.

So what are the key new features of MQ V9 being delivered in this release? Well there are a number of them that are called out in the announcement letters – so you can read the MQ V9 distributed announcement letter here. And the MQ V9 z/OS MLC announcement letter here. And you can read the MQ V9 One Time Charge announcement letter here. But below I will call out a few of the features that I think will be most important to customers.

One of the features likely to be most interesting is a change to the MQ Client Channel Definition Table (CCDT), which is needed by the MQ Client application to provide the channel definitions needed to connect to the MQ Queue Manager. This file is created automatically and prior to MQ V9 needed to be distributed to the client application prior to use. The big change from this new release is that the CCDT can be a web addressable file instead of needing to be distributed out to every client, and to then need to do that with every change. By having a web addressable CCDT accessed by URI, then there are much lower administration needs, and also the MQ infrastructure can be much more dynamic as changes can be made centrally and take effect quickly and without application disruption.

 

The second big change to the new release of MQ is in MQ Advanced Message Security (MQ AMS). This feature, which is a priced extension to MQ (available either separately or as a part of MQ Advanced) provides policy based encryption at rest of the MQ message contents. By using this capability, businesses can be assured that their message contents can only be unencrypted and read by the targeted application destination, and there is no risk of exposure should any security breach take place which provides access to the system or storage where the MQ Queue Manager holds its queues. This privacy and integrity has been assured by the generation of asymmetric keys for every exchange between client and queue manager, which provides an extremely high level of security, but can introduce a high overhead in terms of the processor cost of the asymmetric key generation.

MQ AMS performance

With MQ V9, a new mode of operation is added to MQ AMS, called ‘Confidentiality’. In this mode there is an initial asymmetric key exchange then subsequent exchanges can reuse (to an extent that can be configured) a symmetric key. This still provides a high level of security and protection for the message content, but with a dramatically lower level of overhead in terms of encryption workload cost. IBM expects that due to the increasing importance of security and protecting systems and data from breaches, that this new feature of MQ AMS will help more customers protect their message contents and therefore their business and customer data. IBM expects to produce performance data for the new AMS configuration around the time that MQ V9 is generally available. But the early testing shows considerable improvement.

 

A further change for MQ AMS is the support of non-IBM JREs for use with MQ AMS. Previously applications written in Java that relied on a non-IBM JRE wouldn’t work with MQ AMS. In MQ V9 this has now changed so that suitable non-IBM JREs can be used, as well as IBM JREs, extending the ability of more customers to use MQ AMS.

 

There are a number of other new functions and capabilities available in MQ V9, such as updates to MQ Managed File Transfer capabilities – which are described in the announcement letter, and with the movement to a Continuous Delivery model customers should expect to see more capabilities being delivered in mod levels on top of MQ V9 in the future.

 

With the recent announcement of the End of Support for MQ V7.1 – announced here – along with the related end of support of the older separate versions of MQ FTE and MQ AMS, this latest release of MQ V9, along with the recent announcement of the update to the MQ Appliance provides customers with a strong set of choices of how to take advantage of the latest new releases as they plan to move off the older releases of MQ they may be using, keeping their deployment of MQ up to date and supported.

When you are taking advantage of the benefits of IBM MQ, you may not need to have to work as hard as Ed Moses did to be #1.

UPDATE: Mark Taylor has provided one of his highly useful videos detailing more of the new function in MQ V9. Watch it here.

 

Going faster by not moving – IBM Appliance M2001

April 19, 2016

totoise-rocket-patch

Go faster. Faster. Move it! Or actually don’t move it. There are times when to go faster you need to stop moving. We are all familiar with the parable of the tortoise and the hare – where slow and steady wins the race. But what about not moving at all? Sometimes that makes you go much faster. And in the case of the latest update to the IBM MQ Appliance that is exactly what we are doing. Hopefully you already know about the MQ Appliance, which IBM releases early in 2015, and have continued to enhance since its release. You can read my original entry here, and the update at the end of last year here.

 

But today, April 19th, IBM is announcing another update to the MQ Appliance which not only provides additional functional enhancement, by allowing queue managers to both synchronously replicate for HA and also asynchronously replicate for DR, and adds support for the AMQP based MQ Light API, but also sees a small but important hardware update, making this a slightly refreshed model – the MQ Appliance M2001.

HA+DR

There are 2 key hardware changes in this model update. To help support the simultaneous HA and DR function, which would use both existing 10Gb network cards, the existing 2 port connection is being replaced with a 4 port connection, providing 4 of these 10GB network ports, enabling 2 to be used for HA and DR and ensuring 2 can be used by applications connecting to the appliance, as well as the existing 1Gb ports.

M2001

The second hardware change is the replacement of the existing pair of 1.2TB hard disk drives (HDD) with a pair of 3.2TB solid state drives (SSD). As well as the benefit of the greatly increased storage capacity, the major benefit of using SSDs is the increase in performance for persistent message throughput. The MQ Appliance is a highly capable system which can process a lot of MQ messages. However, when using persistent messaging, which needs to be written to disk, it is critical that the storage can keep pace with the high rate of workload being handled by the system and at times with heavy workloads the spinning disk simply couldn’t move fast enough. IBM has selected the latest generation of SSDs to provide large capacity, high performance for both reading and writing data at high rates, and also this latest generation of SSDs, even if the MQ Appliance is used heavily all day, every day, should last for the 5 year supported lifespan of the MQ Appliance. Therefore, this provides the payoff from our ‘tortoise and hare’ parable – with no moving parts in the SSDs, they can be a lot faster than spinning disks. Expect to see updated performance figures for the new MQ Appliance M2001 around the time of its availability (June 10th 2016), but early figures suggest for some workloads performance improvements of up to 3 times have been seen.

 

There continue to be 2 editions of the MQ Appliance – the M2001A, providing full access to all the processor cores, and the M2001B, which provides access only to a subset of the cores – with an upgrade available from the B to the A system if needed. For customers who may have already purchased the MQ Appliance M2000, please talk to your IBM sales rep to see whether your appliance can take advantage of an upgrade of the HDDs and network card if available.

 

With the improved HA and DR functions, the increased storage capacity and the greatly increased performance, IBM believes this enhanced MQ Appliance makes even more sense to be used as the heart of your IBM MQ deployment, or as a highly available pair of appliances that can be deployed anywhere you need MQ capability. And for customers who may be running older versions of MQ which were recently subject to an announcement of End of Support – as can be seen here – then the latest version of the IBM MQ Appliance can represent a very good deployment option which is then far simpler to deploy as well as to maintain.

 

By moving from spinning disks, to SSDs with no moving parts, you really can go faster by standing still.

 

 

Setting out markers for MQ’s road ahead

February 16, 2016

2016-road-ahead

Working as the Offering Manager for IBM MQ and the IBM Messaging Portfolio, there are lots of parts of my day-to-day work that I can’t share on here until we announce it. However there are times when we can provide a small look ahead at what’s coming. This is called in IBM a “Statement of Direction”. And today IBM MQ has released a Statement of Direction for both IBM MQ and for the IBM MQ Appliance.

You can read the Statement of Direction here.
As you will see in reading it we are talking about a couple of important points. I will deal with the MQ Appliance statement first. As covered elsewhere in this blog, there has been a lot of interest in the MQ Appliance since we announced it at IBM InterConnect 2015 – just about 1 year ago. One of its key features has been about the High Availability function – the simple way to connect up two appliances and to allow for seamless failover between them benefitting from synchronous replication.
At the end of 2015, as detailed here IBM extended this High Availability option with asynchronous replication to other MQ Appliances, which could be deployed further away, offering Disaster Recovery. However, deployments needed to choose either one style of replication or another, on a Queue Manager by Queue Manager basis. So a Queue Manager on a MQ Appliance could be defined for High Availability, or for Disaster Recovery, but not both.
This created an obvious question when we discussed this with customers, who in some cases would want to have local MQ Appliances offering High Availability, but in the case of a whole site failure, wanted to then offer Disaster Recovery off-site. As giving forward looking statements can be an issue without legal clearance, we have ensured that with this Statement of Direction we can clearly state and assure customers that IBM indeed does intend to support the ability of Queue Managers to be configured for both High Availability and Disaster Recovery in a future update.

DR-phase2

For MQ itself the Statement of Direction covers less function, and more the delivery and support approach used for MQ itself. For many years IBM has released updates to IBM MQ every 2-3 years as major new versions, and sometimes with additional interim updates as incremental releases. But over the last few years IBM has been adding function into the regular fixpack deliverables where we also include maintenance updates alongside the new function.
While this approach allows IBM to add useful new functions between releases, and thus getting it to customers earlier, it can lead some customers to choose to keep their MQ implementations on older releases until IBM stops adding new function to that particular release. The concern is that adding new function in a release that will be used in production can create the need to have a major new testing cycle, even if IBM has designed that the new function is off by default.
As IBM thinks customers would benefit from being at the latest level of code, and certainly IBM wants to encourage customers to stay up to date with the latest fixpacks, IBM has decided to offer two separate code delivery and support options.

One option will be the Long Term Support Stream. A new version of MQ will be released, and from that point on, there will be no new function shipped on that code-stream. The fixpacks that IBM will continue to ship on a regular basis will only contain fixes to existing functions and no new functions will be added. As such it should be simpler and safer for customers to move more rapidly to this level of code and to then stay on it as fixes are rolled out, improving stability and performance.
The second option will be the Continuous Delivery option. Based off the same original code drop as the Long Term Support option, subsequent updates will be delivered containing not just fixes but also incremental new function. Each mod-level update will be designed to continue to add new function. And, important to understand, customers who choose to deploy the Continuous Delivery stream will have to keep taking the additional functional increments and fixes if they want to stay on that stream by moving to the most recent mod-level. If they decide they want to be on the Long Term Support stream then will need to change the MQ installed which will likely cause a degree of disruption as they will effectively be moving to a different release. While this continuous delivery of function will ensure that customers of MQ will have new functions that enhance MQ and the operation of their environment, those customers will need to be able to continue to update their environment with each update as it is delivered. For many customers this might be appropriate as they have a need for the new function or they may want to apply it only to a particular environment and set of applications.

LTS

After a number of functional updates to the Continuous Delivery Stream of IBM MQ, over probably a period of 2 years or so, it is expected that the incremental set of new functions delivered in the Continuous Delivery Stream will be released as the new starting point for the next version of the Long Term Support stream, and will reset the version for the Continuous Delivery Stream as well. The cycle them will repeat again, with fixes applied to the Long Term Support Stream and new mod-level updates with new function (as well as fixes) delivered to the Continuous Delivery Stream.

This new approach for delivering MQ may be significantly important for some customers as they make future plans, and IBM therefore thought it was important to set this out in a Statement of Direction prior to a future announcement of a new release of IBM MQ supporting this model.

As for when any new releases to backup these Statements of Direction are coming out? Well, keep watching this space.

A message awakens: What is IBM MQ and why do you need it?

January 5, 2016

Stormtroopers

Why are we here? Not in the existential way. The answer to that is not in this blog. That would probably require more than the page of general MQ related discourse that I generally include. No, this is more why are you reading about IBM MQ? And maybe more pertinently why am I writing about it.

Do you ever read the book of the film? The novelisation. It’s what happens when instead of a book being turned into a film, generally with about 2/3 of the detail and exposition cut out, you have an original film, and then it is made into a book. Generally these tend to be less satisfying than an original novel.

Earlier in 2015, in an effort to try and communicate more with the tens of thousands of people who use IBM integration products, the team in IBM Hursley have been doing live Google Hangouts on multiple subjects – and these are then saved as YouTube videos. With my colleague John McNamara I have done a number of these and 3 of them have been titled “What is messaging” to try to cover why you might find messaging valuable and useful as opposed to some of the other choices around for communicating and exchanging data.

You can see the 3 videos here: Part 1  and Part 2 and also Part 3 

John and Leif

In those videos we tried, as best we could, to be product agnostic – to focus instead on messaging as an approach rather than a specific implementation such as IBM MQ. However the question naturally arises why should you specifically use IBM MQ?

Now in the years I have been writing this blog I have written a few posts that talk to the usefulness of IBM MQ – see here, here and here. However why is it that IBM MQ is still the selection of so many businesses today?

Again, as with the ‘why are we here’ question above, this isn’t something that can be quickly and easily summed up in an easy to read blog entry. So what I will do is try and call out some of the major reasons in simple terms, and then hopefully, as time permits through the year, I will try to add more detail.

Why use IBM MQ then?

  • It works – it does what you need it to do
    • There are many thousands of the world’s leading businesses using IBM MQ, and not just using it – but they are depending on it. They trust it to work, and do what it is asked to do, connecting their business simply, securely, rapidly and reliably.
  • Businesses depend on it – as above – for critical parts of their business – at its heart IBM MQ is built around transactions
    • The business world, and IBM MQ, is largely based on the notion of ‘once and once only’ transactions. While there are other approaches, so much of the critical aspects of business depend on this style of transactionality.
    • It is not easy to offer reliable, persistent messaging that provides once and once only delivery. That’s why many other messaging providers can’t offer this, and why many businesses select IBM MQ
  • IBM MQ scales to meet your business need
    • Developing a small scale application that needs messaging is great and it can be simple to use one of many different messaging tools
    • Ensuring this messaging tool works as the application usage scales is another matter. IBM MQ scales horizontally and vertically, running highly efficiently on single machines or spanning multiple machines in large clusters.
    • Whether sending a few messages per day or scaling to a billion messages per day, it is likely you want good performance. As well as being efficient in scaling, IBM MQ also offers high performance to move messages rapidly between application endpoints.
  • Your business is at risk – how secure is your messaging software?
    • You can’t afford to take risks with your business data, or the information your customers entrust with you. IBM MQ has multiple layers of security on the system itself and the data being moved and supports the latest encryption standards. Can you afford not to protect your messaging layer?
  • Highly available – it’s what you need and your customers expect
    • You can’t afford to go offline – you need to run all day, everyday. IBM MQ can be right there with you, with built-in support for High Availability as well as being able to use multiple different approaches such as vendor-based clustering, or virtualization.
  • Everything you need, functions and tooling
    • While the problems solved by messaging are well understood – and there is a great benefit from simplifying applications, some of that is lost when the need for additional functions require multiple different messaging offerings. With richness of function and a complete span of capabilities IBM MQ offers a single solution
    • Deploying a messaging solution is only part of using the solution. There is a need for management and tooling to provide insight in what is happening to all the messages, and to identify exceptions etc. With support from multiple different tooling vendors, and dozens of additional free tools and add-ons, as well as the ability to create your own utilities, IBM MQ can be tailored to meet the needs of your business solution.
  • Help when you need it, where you need it
    • With more than 20 years of leadership, and substantial market presence, there are thousands of skilled professionals able to provide guidance in how and why to use IBM MQ, how to configure it, how to program with it, and how to deploy and maintain it. And all that supported by IBM’s global 24×7 support organization to help when needed.

So that’s a few of the high points about IBM MQ. I will look to write more in detail about some of these through the year ahead – although I am also pretty sure I will be adding some new entries about product announcements and enhancements as well. Watch this space.

Appliance close up

What can go wrong will go wrong! How the MQ Appliance helps save the day.

November 30, 2015

Dilbert-DR

Since IBM announced the MQ Appliance earlier in 2015, there has been a huge amount of interest in the solution from pretty much everyone. All the customers and business partners I have talked to (along with the many my IBM colleagues have also been talking to) have almost always seen a place for the MQ Appliance in their organizations.

As expected some of these use cases reflect one of our anticipated scenarios of using the MQ Appliance – deploying in a remote location away from the main data centre. Other use cases are based in the data centre with the MQ Appliance being used either to roll out new MQ capacity quickly and simply or to consolidate an existing MQ deployment that might be installed and running on multiple different machines which can make it complex and expensive to maintain, especially when deploying updates or making configuration changes.

MQAppliance

Other that the simple and quick deployment and the ease of maintenance that the MQ Appliance provides, probably the function which generates the most interest from customers and potential customers is the High Availability function. MQ is used pretty universally for work that is critical to the business. The messages being moved between applications and systems contain business critical data and it is crucial that these messages are delivered once and once only and in the case of failure at any point, the messages are recoverable and the business can continue. No one wants to lose the message with the new customer details or the big order.

 

So the High Availability (HA) in the MQ Appliance was seen as key – it was simple to set up – essentially just a single menu selection when defining a new Queue Manager and you would have another appliance ready to synchronously replicate the persistent messages and logs so that in the case of a failure in the production Queue Manager, a replacement queue manager is started on the second MQ Appliance with full access to the messages and logs already available on that appliance. This simple yet rapid and usable solution is compelling, and can also be used, with manual failover control, to enable seamless operation while applying fixpacks on the appliance.

 

However one of the key details to understand about the HA support was that this used synchronous replication of the data between the disks on each appliance, and as the original message can’t be counted as complete until the replication is also complete, the HA appliance needs to be close enough so that the latency of the replication doesn’t impact the application writing the message. The published recommendation is for latency of less than 10ms, but for best operations latency of 2ms or less is preferred.

 

Now, with the 8.0.0.4 fixpack available on the MQ Appliance from November 30 2015, we have added another key feature – which addresses the need for replication over longer distances where latency is always going to be too high for synchronous replication. The 8.0.0.4 fixpack adds asynchronous replication enabling offsite replication over far longer distances than supported for HA as there is no impact to each individual message completion – the replication takes place independently. This style of replication is typically used for requirements such as Disaster Recovery (DR), to enable business continuity out of region with the ability to continue work as close to the point of failure as possible.

 

Customers using this DR feature with the MQ Appliance will be able to configure individual Queue Managers in their appliance to replicate their persistent messages to another MQ Appliance that can be hundreds, or even thousands of kilometres away. And unlike the HA configuration where appliances need to be a defined and fixed pair, there are much more flexible options for this style of asynchronous replication.

 

As mentioned the DR configuration is done on a Queue Manager by Queue Manager basis – but different Queue Managers on the same production appliance can be replicated to different DR appliances. Also Queue Managers defined on different production appliances can all replicate to the same individual DR appliance.

 

As before with the HA appliance, there can be ongoing work and other active Queue Managers on the appliance being used as the DR appliance – there is no formal limitation for appliances to be DR or HA appliances – any appliance can be configured to offer this in conjunction with the other workload running on it.

 

With the addition of this asynchronous replication for Disaster Recovery, the MQ Appliance can be used for more deployment use cases as the ability to recover from failures to a running environment in another data centre is always going to be crucial, as so many businesses depend on MQ to keep them running.

<BLOG UPDATE> With this MQ Appliance fixpack delivering such an important update we also have blogs from our Appliance development lead Ant Beardsmore here, and from our Appliance HA and DR architect John Colgrave here going into more details on the enhancements and the technical details of how DR works.

With simple configuration for all these scenarios, rapid deployment and ‘push-button’ maintenance, it is no wonder so many businesses are looking at using the IBM MQ Appliance. Want to know more? Check out our main webpage. After all, if things can go wrong, they will go wrong. That’s why you use IBM MQ after all. It is better to be ready and to be able to cope with these disruptions. Your business needs to keep running. With the MQ Appliance you can do that with the minimum of effort.

ApplianceDR

When is a quantum leap not really a quantum leap?

November 3, 2015

Electronimage

A quantum leap, if I remember correctly, is the movement of an electron from one energy level to another energy level. So it is not really a big change – just the change from one discrete level to another discrete level. However the phrase “quantum leap” is often used to describe a big change, when in fact it is describing a very small change. Just like a quantum leap, MQ get updated from one fixpack to another. Sometimes this can be thought of as a small change. However in the case of the latest update to MQ V8, this change can represent significant change and new opportunities for MQ users. When an electron does a quantum leap and drops down to a lower energy level and emits a photon, let’s shine a light on the changes in the latest update to IBM MQ.

Last week, on October 23 2015, IBM brought out an update to IBM MQ V8 that included not just fixes, but some really important new features and functions. I will call out some of these here, and hopefully also link to some other sources of information to find out more about the parts I don’t cover.

The key enhancements I am going to cover are the addition of full MQ Light API support within IBM MQ, and also what we are calling the redistributable MQ Client. There are plenty of others (another interesting new feature is setting a Message Expiry Cap) and these are described in Mark Taylor’s short presentation – which can be found here 

Let’s start with the addition of the MQ Light API as a fully supported option. You can read my previous blog on MQ Light here   – but as a recap, MQ Light is a new API for messaging which is designed to allow developers working with Javascript, Ruby, PHP, etc. to make use of messaging as a part of their applications. This enables them to code microservices, and build applications making use of buffering, but doesn’t force them to learn the richer, more complex MQ API or JMS, or require them to code in C or Java. MQ Light is a good way for many enterprises to start to change some of their infrastructure into Hybrid deployments, supporting both on-premise and cloud deployments.  In this latest update, IBM MQ itself now supports the MQ Light API. So these developers can continue to script their applications making use of the MQ Light API in whichever environment they prefer, but now IBM MQ itself acts as the messaging provider. As these MQ Light  applications publish their messages or listen for their subscriptions using the AMQP protocol, this means that AMQP clients can now for the first time connect in to IBM MQ which is then receiving published messages and forwarding them on, or routing the subscriptions to the applications. By ensuring that only a single messaging runtime environment is needed, this lowers the operational burden, reducing complexity for the infrastructure team, and keeps costs down, while keeping the application developers happy with their preferred development environment and API. And the infrastructure team can ensure they support the growing demand for Hybrid infrastructure for cloud deployed applications, while still meeting their enterprise qualities of service for their middleware infrastructure.

The other key update concerns the MQ Client. The MQ Client is how many MQ customers make use of MQ, embedding the client libraries in their applications which then send and receive MQ messages. In this new update the MQ Clients are now also available as tar or zip files for easy embedding in the applications themselves. As well as removing the requirement for a separate install process, the license has been updated to allow applications that include the MQ Clients to be distributed inside and outside the enterprise without requiring permission from IBM as had been the case in the past, which had prevented the easy inclusion of the MQ Client libraries in many solutions. From now on, the MQ Client can be included in any solution, easily packaged and distributed to allow the seamless distribution and deployment of MQ connected applications anywhere required. I am looking forward to seeing many more customer and vendor applications including the MQ Client libraries from now on.

These two changes are substantial enhancements and dramatically change the options available for businesses looking to use IBM MQ for messaging, making this a real leap into the future, for both on-premise business critical applications and the rapidly changing world of Hybrid Integration.

For more details on using MQ Light and AMQP with IBM MQ to enable Hybrid deployments and more see here.

And for more information on the re-distributable MQ Client see here.

A leap into the future, and into the past was the subject of another ‘Quantum Leap’ – this time a TV series from the late 80s/early 90s. Scientist Sam Beckett would ‘leap’ from body to body throughout time. I guess that’s another example of a quantum leap not being quite a quantum leap. As he used to say when he leapt into someone else: “oh boy”.

QL-ohboy

Simpler and cheaper – MQ MFT changing for your benefit

June 2, 2015

Change is always with us. IT infrastructure needs are changed. Application needs change. Skills profiles change. Even workloads and expected response times change. These changes we see in the market drive how we view our products. We frequently update MQ products, perhaps too frequently for some of our customers. As well as adding to and updating the functions and capabilities of MQ, we also try to update or change the packaging and the pricing of our various MQ offerings. We do this to try to respond to the changing needs of the market and the feedback we get from our customers.

As a way of describing this process, we have been recently talking about the different deployment choices available for IBM MQ. Check out this recent webcast on this.

The fundamental thought here is that your business should be able to use the value that MQ provides; however you choose to deploy MQ and consume it. The presentation in the webcast highlights a number of different ways in which your business might want to deploy MQ. This could be maybe reviewing the new MQ Appliance as a deployment choice, deploying the complete MQ set of capabilities using MQ Advanced or seeing whether you want to deploy and use IBM MQ in the cloud – whether that is a public cloud like Microsoft Azure or IBM SoftLayer, or a private/hybrid cloud infrastructure running on your own hardware on-premise, using something such as IBM PureApplication.

Manwithfiles

Going back to MQ Advanced, IBM announced on May 26th 2015, slightly new packaging and pricing for MQ Advanced. Included in this announcement were also various MQ Managed File Transfer parts. These parts were updated to reflect the needs of our customers – given their growing use of using Managed File Transfer with MQ.

As Senior Product Manager for IBM Messaging I talk to many customers through the year, and one of the constant pieces of feedback I get is about the ever-present need for better handling of file transfers. This is an area where every business has a solution, or 2, or 3 today. No one is happy with their existing offering, and most, even if they are existing MQ customers, are unaware that MQ can help.

MQMFT image

MQ’s Managed File Transfer solution can read data from a file, and send it as a MQ message over the MQ network. Once received on the remote system, the MQ MFT solution can then recreate the original file, achieving the movement of the file with greater security and reliability thanks to IBM MQ. This can help to address many of the issues businesses have with moving files, while also simplifying their infrastructure and consolidating on MQ. After initially using MQ MFT to move files, many businesses then take the next step to make use of one of the unique points of MQ MFT which is ‘file to message’ movement. As the file contents are moved as MQ messages, this data can then be directly consumed as MQ messages – meaning that the file contents don’t need to be written back as a file, identified, and then read in again. Instead the data can be delivered directly to the application as a MQ message.

The May 26th announcement simplified the packaging and lowered the pricing for how customers could purchase the MQ MFT capability – either as an extension to existing MQ licenses or as part of the MQ Advanced bundle. The MQ Appliance can also be a part of a MQ Managed File Transfer solution – acting as the co-ordination Queue Manager to allow the MQ MFT Agents to send and receive the file data as MQ messages. With  more and more MQ customers choosing to use and deploy MQ MFT we are changing the packaging to ensure they can do this more cheaply by removing the Connect:Direct and Control Center products we had bundled in as they haven’t been used as widely as the MQ MFT capabilities.

ApplianceMFT

Don’t forget that if you buy the MQ Advanced offering you not only get the MQ MFT Service part but also the MQ AMS capability for end-to-end encryption. This has also been a hot topic of conversation with customers and if you want to know more you can read my previous blog about it here.


Follow

Get every new post delivered to your Inbox.

Join 488 other followers