Posts Tagged ‘High Availability’

Two steps forward, no steps back with IBM MQ V9.0.4

October 24, 2017

hopscotch

Compromise is everywhere. We are told to take the rough with the smooth. The easy with the hard. The quick win and the hard slog. And with software we often have to accept compromises. Especially so these days with the drive for new function forcing some compromises with stable deployments.

Not so with the latest update to the MQ family of products. For the last 15 months IBM has been delivering updates to MQ using a Continuous Delivery stream. There have been many useful additions, but they have always required adoption of the latest version to take advantage of the new features. With the latest update moving to MQ V9.0.4, there are even more substantial updates of useful features for both base MQ and MQ Advanced. However in recognition of the need for customers to keep some systems back-level while also wanting to take advantage of new features, some of these updates are designed to allow existing deployed systems to take advantage of the new capabilities, both without being updated and without breaking the Continuous Delivery and Long Term Support principles.

In addition to this extremely useful update, which I will get to in a minute, which can be used across the entire MQ estate, there are some groundbreaking updates that will allow huge changes in the way MQ is used, deployed and managed in this update. It is more leaps forward rather than steps forward.

For MQ Advanced we have 3 key new capabilities:

  • A new ‘easy HA’ feature – Replicated Data Queue Managers
  • More flexible Managed File Transfer deployments
  • Availability of an enhanced Blockchain bridge

For MQ Base (which is part of MQ Advanced) there are a number of other enhancements

  • Additional commands supported as part of the REST API for admin
  • Availability of a ‘catch-all’ for MQSC commands as part of the REST API for admin
  • Ability to use a single MQ V9.0.4 Queue Manager as a single point gateway for REST API based admin of other MQ environments including older MQ versions such as MQ V9 LTS and MQ V8.
  • Ability to use MQ V9.0.4 as a proxy for IBM Cloud Product Insights reporting across older deployed versions of MQ
  • Availability of an enhanced MQ bridge for Salesforce
  • Initial availability of a new programmatic REST API for messaging applications

 

All of these features are called out in the new announcement letter for MQ V9.0.4 here. And there are further updates available for the MQ Appliance listed in the specific announcement letter for it here and in another blog entry here. There are also announcement letters for IBM MQ z/OS V9.0.4 and IBM MQ Advanced for z/OS VUE V9.0.4

However, let’s try and call out some details of the key points of the MQ V9.0.4 update below:

RDQM1

The new High Availability feature (officially described as Replicated Data Queue Managers or RDQM) provides a significant new way to configure High Availability. It is only available for MQ Advanced users on x86 Red Hat Linux. It is designed as a 3 node system which uses replication of messages and logs between the local disks available to each Queue Manager. This style of replication of local disks was previously only available with the MQ Appliance. As moving to this new style of HA will allow customers to stop using network storage for MQ, we anticipate it will be very popular. As well as the disk level replication, Floating IP will be used to help applications move seamlessly to a failover QM. And 3 nodes help to prevent ‘split-brain’ situations where 2 nodes are simultaneously active.

The licensing of the above deployment requires MQ Advanced as already stated. However as long as all Queue Managers on all 3 nodes are Replicated Data Queue Managers, and all 3 systems are the same capacity, then only one node needs to have a MQ Advanced license entitlement. The other 2 nodes can be licensed with MQ Advanced High Availability Replica parts (these parts used to be called Idle Standby parts).

RESTproxy

The changes to the REST API for admin are also significant. Over the last few releases more and more ‘verbs’ have been added to allow REST API calls to configure and manage MQ. This was designed to allow more modern tools to be built as an alternative to MQSC and PCF based tooling. The latest V9.0.4 release adds more verbs and also a way to call the remaining equivalent MQSC functions within a REST API structure. However what is perhaps more interesting is that a single V9.0.4 Queue Manager can now act as a ‘gateway’ Queue Manager to allow these new REST API driven tools to configure and manage Queue Managers that are older and don’t include this new Continuous Delivery function. This is hopefully a very good way of providing the best of both worlds. Allowing the older production Queue Managers to remain deployed but still take advantage of new features.

Similar to this ‘bridge’ feature is one for IBM Cloud Product Insights, where the ability to publish deployed Queue Manager data to Cloud Product Insights was limited to releases on the Continuous Delivery stream, but now a single V9.0.4 Queue Manager enables older installs to publish data to this useful dashboard tool.

The MQ bridge for Salesforce has been enhanced to allow MQ to publish data into Salesforce, instead of simply receiving push notifications from Salesforce.

Customers with MQ Advanced who want to explore the possibilities offered by Blockchains now can deploy a bridge which enables MQ applications to query the Blockchain, and also provide data input into it. An earlier version of this was available only to customers with MQ Advanced for z/OS VUE, but this version is available to customers using MQ Advanced on distributed platforms.

MQ Advanced customers also get more flexibility in how they can deploy the file logger in MQ Managed File Transfer scenarios, as this logger can now be deployed on a different machine to the MQ Queue Manager.

And finally, feedback from customers told us that developers were looking to make sure of MQ, but with fewer dependencies, to free them up from client and language bindings. As such we have also added the first layer of support for a new set of programmatic REST APIs for messaging applications. This will replace the previous HTTPBridge function which has already been deprecated. Over the next few releases it is hoped that more functions will be supported in this REST API for messaging to allow additional messaging calls to be supported.

Counting up the advances it does look like it is more than 2 steps forward, and certainly no steps back. And with the ability to use some of these features alongside your older MQ releases, what are you waiting for? Download it from here today. Or try it on Amazon AWS Quick Start.

Want to know more. Check out the webcast. Register or replay at this link.

Advertisements

Let your troubles float away with the IBM MQ Appliance

November 15, 2016

balloons

Sometimes you instinctively know when something is right. It just seems to fit. To all fall into place. When you solve a mathematical equation. When you put on a jacket. When you pick up a hammer. You just know it is feels right.

Since IBM released the IBM MQ Appliance in 2015, we have had a lot of customers look at it, and for many of them it has seemed to be something just right for them – just what they were looking for, as it simplified their infrastructure and reduced the tasks of configuring, operating and maintaining their MQ installs.

However, there is plenty of opportunity for improvement, both in adding new features and in improving those already there. And some of the early customer feedback about the MQ Appliance has been critical in some of the enhancements that have already been delivered and also feedback has been critical to some of the features just delivered in the latest update to the IBM MQ Appliance M2001, providing MQ V9.0.1 on the MQ Appliance. Note that this latest software update is also available for customers still running the MQ Appliance M2000.

floating

One of the key new features is the provision of Floating IP support to aid in the High Availability failover configurations. The MQ Appliance provides High Availability by connecting appliances as a pair, and individual Queue Managers can failover from one appliance to another quickly and seamlessly, with the persistent messages and logs already replicated synchronously. However, in order to support this, the MQ client used by the application needed to be configured with not just the IP address of the primary appliance but of the second appliance in the pair as well. This wasn’t always convenient for customers to require all the MQ clients and applications to have a string of IP addresses to prepare for failover.

To address this, and make the experience of using the MQ Appliance even better for our customers, in the latest V9.0.1 level of code, High Availability configurations now allow for Floating IP – which means that as the first MQ Appliance fails over, the second appliance not only starts up a Queue Manager, but it starts up the IP address from the primary, enabling the MQ applications to connect to the second appliance even if they only have a single IP address configured. This should make using the MQ Appliance an even better experience for a much wider set of deployments, without requiring too much of a change to the applications.

As already mentioned above, the MQ Appliance now ships with the MQ 9.0.1 continuous delivery release. This means that the MQ Appliance now benefits from the MQ V9 functions such as the new MQ AMS confidentiality option. This also means that all the new and upcoming features in the MQ continuous delivery stream will be available to the MQ Appliance as those releases come out, with more access to the new REST API for admin and configuration as well as a refreshed MQ Console.

 

monitoringmanagementappliance

Also, as well as some usability improvement for management of the appliance and the MQ operational aspects, this update includes s number of key features exposed from some of the underlying firmware. Key among these are support for SNMP and enhanced security, such as role based authorization, and LDAP authentication for appliance admin accounts. These, again, should make the MQ Appliance fit even better into an organization and be applicable to more use cases.

With further updates to come as part of the Continuous Delivery stream for MQ and the MQ Appliance, there will be more improvements to come to continue to make the experience feel even better. So get ready to float away from your troubles with the latest update to the MQ Appliance.

UPDATE: An excellent blog on MQDev developerWorks site by Ian Harwood. Another blog specifically on the MQ Appliance update by Ant Beardsmore.

Flash aaaahh – saviour of the universe: IBM MQ Appliance M2001

June 10, 2016

flash_gordon_facebook_cover_by_audrey41lorgeoux-d538dgo

Anyone who is a fan of cheesy sci-fi movies, or soundtracks by Queen will have the words “Flash, Flash I love you but we only have 14 hours to save the earth” running through their head, along with the line in the song that goes “Flash aaaaah, saviour of the universe”. And of course he did save the universe from Ming the Merciless.

But what if I told you Flash could also save your business? Not Flash Gordon of course, but flash storage, in the form of the SSDs that are now a part of the IBM MQ Appliance M2001 which is now generally available (June 10th 2016). We did cover this in an earlier blogpost, but I thought I would take advantage of our initial shipment date to cover just how critical the IBM MQ Appliance, backed by state of the art 3.2TB SSDs can be to your business.

MQ Appliance M2001

Each MQ Appliance M2001 model has 2 of the 3.2TB SSDs in a RAID 1 configuration. This means that every persistent message and all log data is written not just once to the SSD storage, but twice giving you complete redundancy of data. And a key part of the MQ Appliance functionality is the High Availability configuration – essentially nothing more than a simple menu option when creating a queue manager – allowing you to have the MQ queue manager on one MQ Appliance synchronously replicated to another MQ Appliance. This means that any message written to the SSDs on one MQ Appliance is not just copied to the second pair of SSDs but is written under the same unit of work that writes the messages on the first MQ Appliance. This therefore means you have 4 copies of the message stored for both reliability and availability.

Another part of the MQ Appliance update was the ability to do not just synchronous replication for High Availability but also asynchronous replication for Disaster Recovery to another MQ Appliance. Therefore you can point the MQ queue manager at another, typically off-site MQ Appliance and the same message will replicate there, ensuring there are another 2 copies of the message, and providing your business with a highly resilient messaging system designed to ensure optimum reliability and availability of messages.

After all, think about how important your messages are to your business. In effect, they are your business. Your messages are your business transactions, your new orders, your customer address details, your stock levels and distribution information. Lose your messages and you lose everything.

With the latest SSD technology inside the MQ Appliance you are calling on Flash to save your business – and with the MQ Appliance M2001, Flash saves the day again.

 

[Flash Gordon image title image above is from  http://orig07.deviantart.net/fc1d/f/2012/163/5/d/flash_gordon_facebook_cover_by_audrey41lorgeoux-d538dgo.jpg]

Going faster by not moving – IBM Appliance M2001

April 19, 2016

totoise-rocket-patch

Go faster. Faster. Move it! Or actually don’t move it. There are times when to go faster you need to stop moving. We are all familiar with the parable of the tortoise and the hare – where slow and steady wins the race. But what about not moving at all? Sometimes that makes you go much faster. And in the case of the latest update to the IBM MQ Appliance that is exactly what we are doing. Hopefully you already know about the MQ Appliance, which IBM releases early in 2015, and have continued to enhance since its release. You can read my original entry here, and the update at the end of last year here.

 

But today, April 19th, IBM is announcing another update to the MQ Appliance which not only provides additional functional enhancement, by allowing queue managers to both synchronously replicate for HA and also asynchronously replicate for DR, and adds support for the AMQP based MQ Light API, but also sees a small but important hardware update, making this a slightly refreshed model – the MQ Appliance M2001.

HA+DR

There are 2 key hardware changes in this model update. To help support the simultaneous HA and DR function, which would use both existing 10Gb network cards, the existing 2 port connection is being replaced with a 4 port connection, providing 4 of these 10GB network ports, enabling 2 to be used for HA and DR and ensuring 2 can be used by applications connecting to the appliance, as well as the existing 1Gb ports.

M2001

The second hardware change is the replacement of the existing pair of 1.2TB hard disk drives (HDD) with a pair of 3.2TB solid state drives (SSD). As well as the benefit of the greatly increased storage capacity, the major benefit of using SSDs is the increase in performance for persistent message throughput. The MQ Appliance is a highly capable system which can process a lot of MQ messages. However, when using persistent messaging, which needs to be written to disk, it is critical that the storage can keep pace with the high rate of workload being handled by the system and at times with heavy workloads the spinning disk simply couldn’t move fast enough. IBM has selected the latest generation of SSDs to provide large capacity, high performance for both reading and writing data at high rates, and also this latest generation of SSDs, even if the MQ Appliance is used heavily all day, every day, should last for the 5 year supported lifespan of the MQ Appliance. Therefore, this provides the payoff from our ‘tortoise and hare’ parable – with no moving parts in the SSDs, they can be a lot faster than spinning disks. Expect to see updated performance figures for the new MQ Appliance M2001 around the time of its availability (June 10th 2016), but early figures suggest for some workloads performance improvements of up to 3 times have been seen.

 

There continue to be 2 editions of the MQ Appliance – the M2001A, providing full access to all the processor cores, and the M2001B, which provides access only to a subset of the cores – with an upgrade available from the B to the A system if needed. For customers who may have already purchased the MQ Appliance M2000, please talk to your IBM sales rep to see whether your appliance can take advantage of an upgrade of the HDDs and network card if available.

 

With the improved HA and DR functions, the increased storage capacity and the greatly increased performance, IBM believes this enhanced MQ Appliance makes even more sense to be used as the heart of your IBM MQ deployment, or as a highly available pair of appliances that can be deployed anywhere you need MQ capability. And for customers who may be running older versions of MQ which were recently subject to an announcement of End of Support – as can be seen here – then the latest version of the IBM MQ Appliance can represent a very good deployment option which is then far simpler to deploy as well as to maintain.

 

By moving from spinning disks, to SSDs with no moving parts, you really can go faster by standing still.

 

 

What can go wrong will go wrong! How the MQ Appliance helps save the day.

November 30, 2015

Dilbert-DR

Since IBM announced the MQ Appliance earlier in 2015, there has been a huge amount of interest in the solution from pretty much everyone. All the customers and business partners I have talked to (along with the many my IBM colleagues have also been talking to) have almost always seen a place for the MQ Appliance in their organizations.

As expected some of these use cases reflect one of our anticipated scenarios of using the MQ Appliance – deploying in a remote location away from the main data centre. Other use cases are based in the data centre with the MQ Appliance being used either to roll out new MQ capacity quickly and simply or to consolidate an existing MQ deployment that might be installed and running on multiple different machines which can make it complex and expensive to maintain, especially when deploying updates or making configuration changes.

MQAppliance

Other that the simple and quick deployment and the ease of maintenance that the MQ Appliance provides, probably the function which generates the most interest from customers and potential customers is the High Availability function. MQ is used pretty universally for work that is critical to the business. The messages being moved between applications and systems contain business critical data and it is crucial that these messages are delivered once and once only and in the case of failure at any point, the messages are recoverable and the business can continue. No one wants to lose the message with the new customer details or the big order.

 

So the High Availability (HA) in the MQ Appliance was seen as key – it was simple to set up – essentially just a single menu selection when defining a new Queue Manager and you would have another appliance ready to synchronously replicate the persistent messages and logs so that in the case of a failure in the production Queue Manager, a replacement queue manager is started on the second MQ Appliance with full access to the messages and logs already available on that appliance. This simple yet rapid and usable solution is compelling, and can also be used, with manual failover control, to enable seamless operation while applying fixpacks on the appliance.

 

However one of the key details to understand about the HA support was that this used synchronous replication of the data between the disks on each appliance, and as the original message can’t be counted as complete until the replication is also complete, the HA appliance needs to be close enough so that the latency of the replication doesn’t impact the application writing the message. The published recommendation is for latency of less than 10ms, but for best operations latency of 2ms or less is preferred.

 

Now, with the 8.0.0.4 fixpack available on the MQ Appliance from November 30 2015, we have added another key feature – which addresses the need for replication over longer distances where latency is always going to be too high for synchronous replication. The 8.0.0.4 fixpack adds asynchronous replication enabling offsite replication over far longer distances than supported for HA as there is no impact to each individual message completion – the replication takes place independently. This style of replication is typically used for requirements such as Disaster Recovery (DR), to enable business continuity out of region with the ability to continue work as close to the point of failure as possible.

 

Customers using this DR feature with the MQ Appliance will be able to configure individual Queue Managers in their appliance to replicate their persistent messages to another MQ Appliance that can be hundreds, or even thousands of kilometres away. And unlike the HA configuration where appliances need to be a defined and fixed pair, there are much more flexible options for this style of asynchronous replication.

 

As mentioned the DR configuration is done on a Queue Manager by Queue Manager basis – but different Queue Managers on the same production appliance can be replicated to different DR appliances. Also Queue Managers defined on different production appliances can all replicate to the same individual DR appliance.

 

As before with the HA appliance, there can be ongoing work and other active Queue Managers on the appliance being used as the DR appliance – there is no formal limitation for appliances to be DR or HA appliances – any appliance can be configured to offer this in conjunction with the other workload running on it.

 

With the addition of this asynchronous replication for Disaster Recovery, the MQ Appliance can be used for more deployment use cases as the ability to recover from failures to a running environment in another data centre is always going to be crucial, as so many businesses depend on MQ to keep them running.

<BLOG UPDATE> With this MQ Appliance fixpack delivering such an important update we also have blogs from our Appliance development lead Ant Beardsmore here, and from our Appliance HA and DR architect John Colgrave here going into more details on the enhancements and the technical details of how DR works.

With simple configuration for all these scenarios, rapid deployment and ‘push-button’ maintenance, it is no wonder so many businesses are looking at using the IBM MQ Appliance. Want to know more? Check out our main webpage. After all, if things can go wrong, they will go wrong. That’s why you use IBM MQ after all. It is better to be ready and to be able to cope with these disruptions. Your business needs to keep running. With the MQ Appliance you can do that with the minimum of effort.

ApplianceDR

WebSphere MQ Advanced for Developers – another exciting step forward

March 19, 2013

Yes. It’s that time again. Another blog entry, so we must have something new to say about WebSphere MQ. And yes we do.

I’m very pleased that we have another announcement to make today. WebSphere MQ Advanced for Developers. You can read the announcement letter itself here. But in the meantime what are the key points you really ought to know?

1)     This is an exciting new development in that we have added a new way to buy licenses of WebSphere MQ, and indeed the entire WebSphere MQ Advanced stack, but priced for individual developer deployment and use. We continue to have both the separate WebSphere MQ licenses and WebSphere MQ Advanced licenses, which are both priced by PVU, and suitable for production and testing. But this new licensing option is priced per ‘Authorised User Single Install’ which effectively means a fixed price per developer that wants to use it for their development use. We believe that by making the WebSphere MQ portfolio more available for developers, giving them individual access we will increase their skills, improve their ability to use WebSphere MQ in innovative ways, and also speed the time to develop more productive applications. Read the announcement letter and get the new part from Friday 22nd March.

2)     As part of this announcement we are also taking advantage of the opportunity to bring everyone up to date on the enhancements we have been making to the support for connecting to M2M devices and mobiles. Over the last couple of months IBM has been publishing new Messaging Clients to support Android and iOS mobile devices. These have been published on our new Messaging Community on developerWorks. This announcement has been timed to coincide with the first fixpack for WebSphere MQ V7.5 which adds in additional function that also allows connectivity from MQTT clients over WebSockets. This creates new opportunities for building new types of applications that can connect to the enterprise over WebSockets, which we expect to be very popular, especially when combined with the new Javascript API which supports both this style of connection as well as supporting the clients on Android or iOS devices. These clients support our Worklight offering, and with IBM MobileFirst this is a very exciting time to be adding this type of support.

3)     Finally as part of this update IBM has also made some changes to the licensing to support High Availability with WebSphere MQ. In recent years WebSphere MQ added its own approach for High Availability – called Multi-Instance Queue Managers, which allowed WebSphere MQ to automatically watch itself and failover if required. An additional ‘Idle Standby’ license was required for the failover system, which was substantially cheaper than a full license. In this update we have extended these Idle Standby licenses to also apply to other configurations that provide automatic failover – such as PowerHA, or 3rd party configurations like Veritas. Now the failover systems in these environments can also be licensed with the Idle Standby parts.

Hopefully these look like a good set of updates – and they have kept us busy working towards them. Don’t forget you can download the free trial for WebSphere MQ here. And visit the Messaging Community to find more stuff, including the Messaging Clients for M2M and Mobile. And look for a webcast from me on the Global WebSphere Community on April 11th.