Accelerating MQ with the release of IBM MQ V9.1.4

Sometimes change happens slowly, and gradually. We see this a lot with updates to IBM MQ over the last few years following the Continuous Delivery cycle. Innovations are rolled out, release by release. The product gains an enhancement and subsequent releases see cumulative improvements to that initial enhancement until, by the time all the enhancements get rolled up into a Long Term Support release, there is a rich set of capabilities that deliver more than the sum of the parts.

 

Enhancements don’t always happen that way. Sometimes they might be small, discrete and delivered in one go. And just occasionally, a significant enhancement comes along out of the blue, and provides potentially huge benefits in one release. In MQ V9.1.4, announcing today, customers with MQ Advanced or MQ Appliance entitlements get access to a breakthrough innovation: the Aspera FASP.io streaming gateway.

 

To explain what this is, and what it could do, it is helpful to understand Aspera technology which is outlined in my previous blog

The FASP.io streaming gateway is a new innovation using the FASP protocol. Deployed within the customer environment with a MQ Queue Manager (either with MQ Advanced entitlement or running on the MQ Appliance) configured to connect to it. Specified messages from MQ would flow from the Queue Manager to the gateway, which would then route them over FASP to another FASP.io streaming gateway which is similarly configured to another appropriately entitled Queue Manager, which would then receive the MQ message.

So why do all this? What would be the benefit? Let’s consider the situation of a business sending MQ messages between locations over hundreds or thousands of miles, such as between New York and Singapore. Individual MQ messages can be up to 100MBs in size. And if the data originated in a file, this could be a much larger total amount of data which is split into multiple MQ messages. Using regular MQ configurations the messages would flow over TCP/IP. Sending large amounts of data over long distances using TCP/IP can be much slower than the notional line speed would suggest if the line is lossy or experiences high latency.

 

In these situations, using the Aspera FASP protocol can provide a predictable and much faster way to transport that data. Therefore, for some use cases, sending MQ messages over FASP will see them delivered much more quickly than would have been the case, which could be an extremely valuable new enhancement if that time helps the business move and respond more quickly. We hope to publish some early performance data shortly that offers an understanding of the size of the acceleration that is possible. And this is available at no additional cost for customers with MQ Advanced or MQ Appliance.

 

Other than this new feature, what else is new in MQ V9.1.4?

 

The uniform cluster feature delivered initially in 9.1.2 and enhanced in 9.1.3 is further enhanced in MQ V9.1.4. We have added support for .Net and XMS .Net applications. We have also simplified the configuration of Queue Managers in the uniform cluster and speeded the time to rebalancing, as well as provided more information about the rebalancing that takes place. This is proving to be highly attractive to customers, especially as they modernize their messaging deployments with increased numbers of queue managers providing more scalability and flexibility, while being decoupled from the application instances.

 

Security remains a critical feature of IBM MQ and we have now added in support for TLS 1.3 for the first time. This is initially supported for C/C++ MQ client applications and will be further extended in the future. In the future we will be stopping support for SSL v3, and TLS 1.0. And for MQ on z/OS the pervasive encryption support added on the z14 hardware sees increased support for some MQ datasets. MQ on z/OS announcement letters are here and here.

 

Adding to the security aspect, many businesses send and receive MQ messages through their firewall. Due to MQ persisting data in storage, IBM doesn’t recommend deploying MQ Queue Managers in the DMZ, and instead has provided the MQ Internet PassThru (MQ IPT) to be deployed in the DMZ. This proxy has been available as a Supportpac from IBM, and although it was fully supported, this delivery outside MQ made it hard for some customers to deploy and use it. With MQ V9.1.4, MQ IPT is now a part of the MQ package and therefore should be more widely available for deployment. And for customers with MQ Advanced entitlement, we have added HSM support for MQ IPT to enable the digital security keys to make use of this enhanced security option.

The MQ Managed File Transfer feature of MQ Advanced also continues to be enhanced. The MFT Agent is now Highly Available, with another instance able to continue to transfer in the case of a failure. And also, the REST API support for MQ MFT is further extended with support for Create Monitor added.

 

Also, for MQ Advanced, the MQ Bridge to Blockchain adds support for Hyperledger Fabric for improved interaction support between MQ and the Blockchain

 

The last point I will mention is around Red Hat OpenShift. Following the acquisition of Red Hat by IBM, the MQ Advanced certified container now supports deployment directly on Red Hat OpenShift, without the need for IBM Cloud Private. The MQ Advanced certified container can either be deployed on its own in OpenShift or as a part of the IBM Cloud Pak for Integration.

I did a webinar to cover the new MQ V9.1.4 content – you can find the replay link here on the IBM Middleware Community

There is also our official blog update for this release.

As with every release, IBM MQ moves forward. Sometimes in small steps and sometimes it accelerates into the future, just like a MQ message sent using the FASP.io gateway. Forza! MQ.

The best things in life are ‘3’. Now announcing IBM MQ V9.1.3

There are many time in old Monty Python sketches where the number 3 seems to come up. There is a 3 headed giant in Monty Python and the Holy Grail. There is also a problem where King Arthur is trying to count to 3 to throw the ‘Holy Hand Grenade of Antioch’ but goes “1, 2, 5”.

 

As we move through the 2^nd set of Continuous Delivery releases we are expecting to follow the same pattern for the 9.1.x releases as we did for the 9.0.x releases then we would anticipate that the final release will be 9.1.5, but we aren’t there yet. We have just announced MQ V9.1.3 on all platforms including the MQ Appliance. You can read the announcement letter for the distributed and Appliance offerings here. And you can read the letters for MQ z/OS here, and MQ z/OS VUE here. An important additional announcement to be aware of is that IBM is announcing withdrawal the separate MQ Advanced Message Security and MQ Managed File Transfer offerings on z/OS, with MQ Advanced for z/OS VUE being the recommended way to get these extended capabilities going forward.

For this blog I will call out a number of the key new features in MQ V9.1.3, and why I think they are important.

Let’s start with a feature that was first delivered in MQ V9.1.2, is of strategic importance and has been extended in MQ 9.1.3. This is a feature called Uniform Clusters and it allows MQ itself to balance application connections across multiple different queue managers. The initial release only supported this balancing for C applications and in this release the capability is extended to JMS applications. Why is this a useful feature? If you have multiple application connections to a set of queue managers, there is no easy way to ensure a fair distribution of workload across the queue managers. And then imagine what might happen if you remove or add queue managers for maintenance or to adjust available capacity. How can you rebalance workload, especially when new queue managers are being added? This feature allows MQ itself to be aware of the group of queue managers to spread the work across, and will take care of the balancing and rebalancing needed. As workload and queue managers become more dynamic with hybrid cloud deployments and containers, then this will become increasingly essential.

Recent releases over the last year or so have seen new feature enabling use of a REST API for admin, as well as also using REST for messaging. MQ V9.1.3 sees enhancements in both of these areas. The REST API for admin now allows calls to ‘runmqsc’ to use JSON inputs through the REST interface to return JSON output. The JSON input and output means that it will be much easier to send commands and to understand and take actions based on the output of the commands. This will help more customers and vendors build new tooling, or to update their existing tooling to be more powerful and dynamic and to use modern tooling frameworks.

REST Messaging offers the ability to send and receive MQ data without using MQ clients. Previously there was only the ability to use PUT and GET commands, but MQ V9.1.3 adds support to Browse messages.

 

New enhancements in MQ Advanced include a number of updates for the MQ MFT feature. One of these enhancements extends the FTP protocol bridge to now support FTP servers that run on the IBM i platform. With MQ Advanced V9.1.3, customers who use FTP to move files into and out of the IBM i platform can have them intercepted by the FTP Protocol Bridge and moved into the MQ network.

And for those increasing numbers of customers using MQ Advanced container images, in Kubernetes environments there is now the option to configure multi-instance queue managers with active and standby pods, or to use a single resilient queue manager with Kubernetes and system monitoring for high availability.

 

For the MQ Appliance there is a useful enhancement to the HA and DR functions that builds on the capabilities and enhancements previously available. Our MQ Appliance customers really appreciate the High Availability and Disaster Recovery configurations for the appliances. Now with MQ V9.1.3 notifications about HA and DR status, or changes in status are written to the MQ Appliance system log. In MQ V9.1.2, the MQ Appliance system log could be configured to stream off the appliance as with other MQ log targets. This combination of features would allow 3^rd party monitoring tools to detect HA and DR status changes and rapidly alert MQ Appliance customers of failover activity. An additional feature of the latest MQ Appliance update is also to report on the data and time when data synchronization was stopped or lost between Appliances in a HA pair, or in a DR configuration. This will be useful for both offline analysis and for DR restart consistency.

Read the official IBM blog by Ian Harwood about the new release here.

 

This latest MQ V9.1.3 will be available to download on July 11^th 2019. Are you ready for ‘3-dom’?

Banish those winter blues with IBM MQ V9.1.2

In the depths of winter in the UK we are told that Monday 21^st January is referred to as Blue Monday, when the fun of New Year has died away and it is clearly a long time to go until the arrival of the lighter warmer days of spring. But now, just a couple of weeks after Blue Monday, Big Blue IBM is trying to relieve the gloom with the announcement of the latest CD release: IBM MQ V9.1.2. You can read the announcement letter here.

 

As the 2^nd Continuous Delivery release of MQ V9.1, this builds on the previous release with a number of enhancements and new capabilities.

Probably the one that will be of most interest to people is a new capability which is the first step in what will be an ongoing series of updates to MQ. We are calling this a Uniform Cluster, and this specific enhancement is designed to make it easier to balance workload across queue managers which could be both growing and shrinking. This workload balancing will be without the need for the applications to co-ordinate changes in the MQ Queue Manager destinations. Instead MQ will itself balance the workload across the set of Queue Managers defined to be a part of this ‘Uniform Cluster’. Initially this is only for applications written in C. This area of MQ is likely to continue to be an area of focus, as further enhancements could easily be considered with a view to MQ being far easier to scale up and scale down, much as a cloud native service would be expected to do.

 

Another key enhancement is around the use of REST messaging. When this feature was initially introduced, it sparked lots of interest, as there are many use cases where it would be helpful to call MQ without having MQ Client libraries. In this release, connection pools are supported allowing for the caching of connections for reuse, which should improve throughput and reduce resource use.

 

Other updates in the base MQ capabilities include .NET core support for Linux to add to the Windows support added in MQ V9.1.1. Also improvements to scalability and availability when working with WebSphere Liberty for XA transactions.

 

Increasingly important to many MQ customers is MQ Advanced. The MQ MFT feature of MQ Advanced, which is widely used to onboard file data into MQ and then send and consume that data as MQ messages gets further REST API functions to enhance administration. This continues what we have seen in the last few releases for MQ MFT.

 

Other interesting improvements include updates to the Salesforce and Blockchain bridges, and the MQ Appliance sees errors logs integrated with system log external targets.

 

There are a number of other really interesting updates to the MQ family which have also come out at this time.

 

Probably everyone is seeing a lot of the same interest in container deployments. And IBM MQ has been supporting container deployments for many years, and recently have put out an IBM Cloud Pak to better support deployment on IBM Cloud Private. However we have now also released a container image of MQ Advanced for Developers for Pivotal Cloud Foundry. This will be available shortly.

 

The MQ Cloud offering, which provides a hosted MQ environment maintained by IBM has been seeing lots of growth and enhancement, with new data centers being added for both IBM Cloud and AWS, as well as adding functional support for the MQ AMS end to end encryption and the MQ MFT features. The latest update adds a Lite plan, allowing ongoing free use of a hosted MQ environment, without the need for a credit card, limited to 1000 messages per month. Check it out here and now!

 

And finally, something else for the developers. While MQ continues to be a robust production platform on Linux, Windows and other environments, there hasn’t been any IBM provided releases for Mac. If you wanted to develop MQ applications on Mac you would need a VM with a supported OS. However we have now released the MQ client for Mac – you can download today from here and start developing much more simply today.

UPDATE: Now we have availability of MQ V9.1.2 here is a blog from Ian Harwood expanding some of the points and with links to access MQ etc.

And if all that doesn’t blow away the winter blues, what will? Maybe a trip to San Francisco for the Think 2019 Conference? I have a number of presentations there so come by and say hello. Otherwise there will be a number of other events through the year. Let’s hope for some sunny and warm weather!

Is your business getting indigestion? IBM MQ can ease that pain by ingesting your file data into MQ including MQ on Cloud.

Christmas and New Year is over for another year. At this time of year, it can be easy to eat and drink too much. Consuming too much can lead to indigestion and the results can be unpleasant.

 

But have you considered it might be similar when you move data through your enterprise? Data can be large. Data can be small. But once it exists, it has a purpose. And that means it has a use, and value. In that case it should delivered, in a timely way, with security and reliability, to where it can add value to the business.

 

However, moving the data can be a problem. Data can be moved by the application as it is created. And certainly, IBM MQ has a long history of being an ideal solution for this, as it is designed to connect applications, exchanging data reliably and asynchronously.

MQ messages have a maximum size of 100MB. Which is actually very large for individual application generated messages, especially if you are sending the data out as it is created, so while some use cases do use very large message sizes, mostly it is much smaller. And not only is MQ optimized for this traffic, enabling it to send millions, or even billions of messages per day through your network, your own infrastructure is likely built to meet this need.

 

But consider when data is created, or pulled in from elsewhere, and may be at rest in the filing system. It needs to move through the business to where it will add value. But this data in the filing system might be thousands, or millions of individual records, imported or built up over time. Trying to send gigabytes, or even terabytes of data in one lump is going to give your network the equivalent of indigestion. It’s going to be blocked up until it can pass through. Traditional file transfer approaches suffer from this issue.

 

Let’s think of how this might happen. You are a retailer. Some of your stores process their transactions as they happen, flying through the network as each one is small. Others instead batch them up and send them as a file. The file can be very large, and if coming from a remote location could take minutes or even hours to come through the network, because of the way that networks can slow down data transfer rates because of errors. This impacts the ability of the business to act on this data.

 

An important feature of MQ Advanced (and MQ Appliance), and now MQ on Cloud is the ability to ‘ingest’ the data from files on the file system into MQ. This data is then moved as MQ messages through your network. As even the largest files are automatically broken into chunks suitable for sending as MQ messages, with all the reliability, security and assured delivery that MQ provides, your business gets the benefit of the data delivery, without suffering ill effects from the movement.

 

Moving all data, from applications and the file system, all through a single reliable high-performance pipe like IBM MQ gives your business the assurance that all data is handled with the right care and attention. And your business suffers no ill effects even when handling the biggest inputs. Allowing more of the data traffic to move rapidly and reliably through your network, without everything slowing down.

Your data is no longer getting stuck in a file, or in a remote system. It won’t even get lost moving between systems. It is moving freely between systems as it moves as MQ messages. No single message is too large for the network. And the business gets to benefit from your data now being handled and processed directly as MQ messages. It is no longer file data, so no longer stuck in the slow lane. Data ingestion is better than indigestion. Accelerate your data use by ingesting your file data with MQ on Cloud, MQ Advanced or MQ Appliance.

Don’t forget you can download and try all the features of MQ Advanced for free from this download page and you can also try MQ on Cloud in just a few clicks here.

All aboard the 9.1.x CD train. First stop is IBM MQ V9.1.1.

I am sure everyone knows the phrase about buses. If you miss one, don’t worry. There’ll be another one along in a minute. And while it could be said that applies to Continuous Delivery releases, I think it is more like getting on board a train. The destination is the next Long Term Support release, and you think you know what stops will be coming up. But maybe you don’t know exactly what you will find at each destination. You know there will be something new to discover at each stop. You could almost think of the train growing at each stop with the content of each new continuous delivery release, ready to be delivered finally to the Long Term Support destination.

 

Which brings us to the latest MQ CD release, MQ V9.1.1, announcing today, which is the first CD release in the 9.1.x set of releases. The experience we have of our 9.0.x CD releases is that we have seen a lot of interest from customers. Some have been able to move quickly to take up the CD stream into their environments and run them in production, at least for some of their queue managers. Others have been able to experiment with the new features in their test environments to see whether it is worth their while adopting the content early. And there seems to be a larger set of users who, while they haven’t been adopting the CD content into the production systems, the earlier availability and visibility of the new content has helped them move much more rapidly to adoption and use of the MQ V9.1 LTS release than we might have previously expected. I have personally talked with a lot of existing MQ customers who have either already started using MQ V9.1 LTS or are planning to move to use it very shortly.

The MQ V9.1.1 release isn’t a destination in itself. It is the first part of our continuing journey. The MQ team works to accommodate a mix of strategic development priorities into releases to move the MQ offering forward, as well as other customer driven priorities, and reacting to and supporting other offerings and platforms as they change and adapt. Let’s find out how this mix has shaped the release. As well as suggesting you read the announcement content in the announcement letter, I will call out a few of the interesting new features.

 

One important new set of capabilities, driven by customer requests, is around the choice and negotiation of the use of TLS ciphers. Security of the MQ environment is hugely important in the current environment and is likely to remain a key area of focus. The importance of security and data protection is one reason customers are moving to MQ Advanced or MQ Appliance as a way to get the end to end encryption in MQ AMS. But this release focusing on enhancement to the security used in the TLS ciphers – used for encryption on the wire, not encryption at rest. As time passes, some ciphers become less secure and customers need to take prompt action in their environments to ensure the ciphers they use are updated to meet their own business requirements as well as the needs of the different systems.

In MQ v9.1.1 the choice of ciphers can be negotiated dynamically from a set or ‘whitelist’ available on each MQ channel. This reduces the potential for downtime and administrative overhead through faster movement to new ciphers when an old cipher is deprecated. Weaker ciphers can be removed from the list of allowable ciphers without needing to wait for a security fix update from IBM.

 

Another update driven by customer requests is the new support in MQ V9.1.1 for .NET Core for Windows. Customers who choose .NET as a framework for running applications on Windows environments have been looking to move to .NET Core. Following a number of requests, we have now added support for .NET Core for Windows environments to help support those customers.

 

As we have seen in the 9.0.x CD stream, one of the important set of capabilities that was added was the REST API for Admin for MQ. And at the end of that set of releases we started to look at adding REST API calls for the administration of MQ Managed File Transfer features, available with MQ Advanced and MQ Appliance. Many customers find it value to ingest and move data through MQ, even when the starting point or destination for this data is a file on the file system. To MQ, it is all just data moving in MQ messages. Therefore, from an administration point of view, it is important to offer similar features and controls for managing the movement of this data through MQ as is available for MQ exchanges of application data. In MQ V9.1.1 the MQ MFT feature gains REST API calls to list the resource monitors as an alternative to previous methods.

 

A further update is to provide support for pausing message delivery to Message Driven Beans running in WebSphere Liberty, in addition to the support previously made available for WebSphere Application Server.

 

The MQ V9.1.1 release offers a good foundation to start the journey through the various 9.1.x CD releases. There was a mix of updates driven by customer needs, wider platform and offering support as well as some functions to enhance longer term MQ strategic plans. We are now pulling out of this station and heading to the next one. Hitch up the V9.1.1 wagon to your V9.1 MQ train, hop on board and enjoy the ride.

Putting out a new release like IBM MQ V9.0.5 is more than a 9-5 job

At least in the UK, the traditional hours worked in a day job were 9 to 5. You would ‘clock-in’ at 9am and leave at 5pm. I guess it is common as there was a 1980s film called “9 to 5” starring Dolly Parton. These days office life is rather more flexible, and certainly the idea of clocking in and out at fixed times is gone.

 

For 25 years, virtually every major IT infrastructure has been able to rely on the secure and reliable exchange of data between applications and systems thanks to IBM MQ. Previously called MQSeries, then WebSphere MQ, this software offering, developed in the IBM Hursley Lab in the UK has been a critical part of the business world. So much so that most people living their lives have no idea they use IBM MQ so much on a daily basis as it ‘just works’.

 

There is a great team of developers who work hard day-in and day-out to enhance and update IBM MQ, and . We have now released IBM MQ V9.0.5, going GA on Friday March 16^th. And our developers have worked for months, giving up evenings and weekends to not just add new features, but to make sure it is another offering that works when put into use. So not 9-5 at all.

 

Now for some customers this will be more of a prelude to the main act. This is referring to V9.0.5 being a Continuous Delivery release. When we brought out V9.0 we split it into 2 streams: Continuous Delivery and Long Term Support. This release marks the final release in the initial set of Continuous Delivery releases. The next release will be the first of a new Long Term Support release. And customers can expect that the functions delivered in the 5 CD releases will be made available in the new Long Term Support release.

 

When that new LTS release is available, you can expect me to summarize all the new features, but for now in this blog I will call out a few of the new features in V9.0.5.

 

The new Easy HA feature (Replicated Data Queue Managers) delivered in MQ Advanced V9.0.4 gets updated to add support for a Disaster Recovery mode, with manual takeover after either synchronous, or asynchronous replication between a pair of MQ servers.

 

The MQ Managed File Transfer capability, available with MQ Advanced or MQ Appliance gets the first support for the REST API admin interface for listing current transfers and querying MFT Agent status.

 

MQ Advanced itself will do more to identify itself when it is installed, and so prevent compliance issues, and ensures that components can recognize Queue Managers.

 

Other updates include a MQ Console refresh, and for customers who use MQ with WebSphere Application Server, performance enhancement through implicit syncpointing.

 

For MQ Appliance users there is an enhancement for better reliability by allowing aggregated IP interfaces for the Floating IP feature. This removes a potential single point of failure.

 

And for users of MQ Advanced for z/OS Value Unit Edition there have been improvements including enhancements to MQ AMS which will see increased performance.

Perhaps even more exciting is the new availability of a hosted instance of MQ on the Cloud. More about this can be found here, but it creates a great opportunity to quickly and easily make use of MQ without needing to install, deploy or manage the environment. Just configure and go! Nice that after 5 years of talking about it on this blog we have an explicit offering running in the cloud. This is of course alongside MQ already being able to run in AWS as a QuickStart. Or deployed as containers in IBM Cloud private.

 

As well as looking forward in the future to a new Long Term Support release, the statement of direction indicated that the Blockchain bridge, available in MQ Advanced, will be updated to be based on the Hyperledger Composer interfaces. And additionally, customers deploying MQ in containers will in the future be able to track the size of the container, and the duration of use, and license based on that container size, rather than the full capacity of the system where the container is running. The intent will be to support existing pricing metrics such as PVUs and VPC monthly metrics, but also a future VPC Hourly metric.

IBM MQ, along with many other IBM and business partner solutions will be some of the highlights discussed at IBM Think in Las Vegas running March 19^th-22^nd. I will be there and I hope to see some of you there as well. Famously Las Vegas never sleeps, so I guess that’s something else that’s not 9 to 5. Lucky we have IBM MQ V9.0.5 now though.

 

Reaching for SANity with the IBM MQ Appliance

We all like to keep our word, and even though a Statement of Direction is not always a promise, at IBM we like to try and deliver what we set out, as long as the market and need hasn’t changed.

And for customers who have been following the MQ Appliance since the start, back in March 2015, you might recall that there was a Statement of Direction about adding support for the fibre channel cards that were part of the MQ Appliance hardware to allow connectivity to a SAN for customers who wanted that. And we are delighted that as part of the MQ V9.0.4 update on the MQ Appliance, that IBM is delivered on that Statement of Direction.

MQ V9.0.4 is a very significant update for MQ and most of the updates that are not specific to the MQ Appliance are described in my other blog entry here. However, in addition to those features there are specific MQ Appliance updates, and a dedicated announcement letter which I will try to explain in more detail here.

Firstly, the SAN support. One of the reasons the MQ Appliance has been so popular with customers is the built-in storage (2x 3.2TB SSDs in a RAID 1 configuration) and the simple HA configuration allowing a pair of MQ Appliances to replicate messages and log data between them on a QM by QM basis with just a couple of clicks on setup. And a lot of the updates to the MQ Appliance has been to support the wider use of this popular deployment use case.

However, some customers have always wanted us to add support for SANs for some of their use cases. One use case was that 3.2TB of onboard storage might not be enough. If you consider when a consuming application might fail over a long weekend and the queue depths might get very high. One of our customers recently said, “an empty queue is a happy queue” and this is true, but in the case of a protracted failure you want to ensure your queues can hold all the messages. So, if you are concerned about this scenario, then you might want to have the Queues supporting that use case on external storage where there is no effective limit to queue depth.

The other use case, which tends to come up more commonly in customer discussions is around the thought of “consistency groups”. This is when you are trying to recover from a site failure at your disaster recovery location. This is typically a manual task, unlike the automated High Availability configuration. Part of this manual task will be to establish the last known transactions of not just MQ but other parts of the IT infrastructure such as applications and databases. This is likely to be easier if all the data points from all these ‘moving parts’ are stored on a common storage area, and this is replicated consistently between the SAN on one site and the SAN on the disaster recovery site. So this use case is now supported by the MQ Appliance with customer selected Queue Managers able to choose the SAN storage instead of the internal storage and have a MQ Appliance in the disaster recovery site read the replicated messages and data from the SAN there.

A second update to the MQ Appliance specific features is to provide customers with a quicker and simpler way to ensure that Queue Managers on the MQ Appliance have the best allocation of resources on the MQ Appliance. When Queue Managers are initially created the MQ Appliance determines how much space to allow to them. However if workload on a Queue Manager grows faster than other Queue Managers, it is likely that the allocated resources might need to grow to match the likely workload. However, it hasn’t been easy to do that. But with the MQ V9.0.4 update on the MQ Appliance IBM has added dynamic disk allocation, enabling resources for a Queue Manager to be increases even after it was initially created. This will make the ongoing operation and support of production workloads on the MQ Appliance quicker and simpler.

Finally, an update off the MQ Appliance will have a positive impact on potential use cases for the MQ Appliance. Virtually every customer still moves large amounts of data around their business as files and file contents. Much of this is unmanaged, insecure and unreliable, and for a number of years IBM has provided a solution: MQ Managed File Transfer. This enables file contents to move from point to point over the MQ network as MQ messages, taking advantage of MQ’s reliability, security and manageability. Part of the MQ Managed File Transfer functions was a logger to track which files were sent over MQ, and this, if used, always needed to run on the same machine as the Queue Manager. This prevented some MQ MFT use cases from being deployed in environments where the MQ Appliance was the only MQ Queue Manager. Now, with MQ V9.0.4 the file logger component of MQ MFT can be deployed remotely from the Queue Manager, meaning it can be used in locations by pointing to a remote MQ Appliance, without the need to install anything on the MQ Appliance, which has always been prohibited. It’s always good to allow customers more flexibility in deployment.

This combination of enhancements, along with the many non-Appliance specific MQ updates in the MQ V9.0.4 release means that there should be a lot of increased opportunities to consider the MQ Appliance as the right deployment option. After all many customers who review it with regards to overall costs find it has the lowest Total Cost of Ownership, as well as outstanding reliability.

Two steps forward, no steps back with IBM MQ V9.0.4

Compromise is everywhere. We are told to take the rough with the smooth. The easy with the hard. The quick win and the hard slog. And with software we often have to accept compromises. Especially so these days with the drive for new function forcing some compromises with stable deployments.

Not so with the latest update to the MQ family of products. For the last 15 months IBM has been delivering updates to MQ using a Continuous Delivery stream. There have been many useful additions, but they have always required adoption of the latest version to take advantage of the new features. With the latest update moving to MQ V9.0.4, there are even more substantial updates of useful features for both base MQ and MQ Advanced. However in recognition of the need for customers to keep some systems back-level while also wanting to take advantage of new features, some of these updates are designed to allow existing deployed systems to take advantage of the new capabilities, both without being updated and without breaking the Continuous Delivery and Long Term Support principles.

In addition to this extremely useful update, which I will get to in a minute, which can be used across the entire MQ estate, there are some groundbreaking updates that will allow huge changes in the way MQ is used, deployed and managed in this update. It is more leaps forward rather than steps forward.

For MQ Advanced we have 3 key new capabilities:

• A new ‘easy HA’ feature – Replicated Data Queue Managers
• More flexible Managed File Transfer deployments
• Availability of an enhanced Blockchain bridge

For MQ Base (which is part of MQ Advanced) there are a number of other enhancements

• Additional commands supported as part of the REST API for admin
• Availability of a ‘catch-all’ for MQSC commands as part of the REST API for admin
• Ability to use a single MQ V9.0.4 Queue Manager as a single point gateway for REST API based admin of other MQ environments including older MQ versions such as MQ V9 LTS and MQ V8.
• Ability to use MQ V9.0.4 as a proxy for IBM Cloud Product Insights reporting across older deployed versions of MQ
• Availability of an enhanced MQ bridge for Salesforce
• Initial availability of a new programmatic REST API for messaging applications

 

All of these features are called out in the new announcement letter for MQ V9.0.4 here. And there are further updates available for the MQ Appliance listed in the specific announcement letter for it here and in another blog entry here. There are also announcement letters for IBM MQ z/OS V9.0.4 and IBM MQ Advanced for z/OS VUE V9.0.4

However, let’s try and call out some details of the key points of the MQ V9.0.4 update below:

The new High Availability feature (officially described as Replicated Data Queue Managers or RDQM) provides a significant new way to configure High Availability. It is only available for MQ Advanced users on x86 Red Hat Linux. It is designed as a 3 node system which uses replication of messages and logs between the local disks available to each Queue Manager. This style of replication of local disks was previously only available with the MQ Appliance. As moving to this new style of HA will allow customers to stop using network storage for MQ, we anticipate it will be very popular. As well as the disk level replication, Floating IP will be used to help applications move seamlessly to a failover QM. And 3 nodes help to prevent ‘split-brain’ situations where 2 nodes are simultaneously active.

The licensing of the above deployment requires MQ Advanced as already stated. However as long as all Queue Managers on all 3 nodes are Replicated Data Queue Managers, and all 3 systems are the same capacity, then only one node needs to have a MQ Advanced license entitlement. The other 2 nodes can be licensed with MQ Advanced High Availability Replica parts (these parts used to be called Idle Standby parts).

The changes to the REST API for admin are also significant. Over the last few releases more and more ‘verbs’ have been added to allow REST API calls to configure and manage MQ. This was designed to allow more modern tools to be built as an alternative to MQSC and PCF based tooling. The latest V9.0.4 release adds more verbs and also a way to call the remaining equivalent MQSC functions within a REST API structure. However what is perhaps more interesting is that a single V9.0.4 Queue Manager can now act as a ‘gateway’ Queue Manager to allow these new REST API driven tools to configure and manage Queue Managers that are older and don’t include this new Continuous Delivery function. This is hopefully a very good way of providing the best of both worlds. Allowing the older production Queue Managers to remain deployed but still take advantage of new features.

Similar to this ‘bridge’ feature is one for IBM Cloud Product Insights, where the ability to publish deployed Queue Manager data to Cloud Product Insights was limited to releases on the Continuous Delivery stream, but now a single V9.0.4 Queue Manager enables older installs to publish data to this useful dashboard tool.

The MQ bridge for Salesforce has been enhanced to allow MQ to publish data into Salesforce, instead of simply receiving push notifications from Salesforce.

Customers with MQ Advanced who want to explore the possibilities offered by Blockchains now can deploy a bridge which enables MQ applications to query the Blockchain, and also provide data input into it. An earlier version of this was available only to customers with MQ Advanced for z/OS VUE, but this version is available to customers using MQ Advanced on distributed platforms.

MQ Advanced customers also get more flexibility in how they can deploy the file logger in MQ Managed File Transfer scenarios, as this logger can now be deployed on a different machine to the MQ Queue Manager.

And finally, feedback from customers told us that developers were looking to make sure of MQ, but with fewer dependencies, to free them up from client and language bindings. As such we have also added the first layer of support for a new set of programmatic REST APIs for messaging applications. This will replace the previous HTTPBridge function which has already been deprecated. Over the next few releases it is hoped that more functions will be supported in this REST API for messaging to allow additional messaging calls to be supported.

Counting up the advances it does look like it is more than 2 steps forward, and certainly no steps back. And with the ability to use some of these features alongside your older MQ releases, what are you waiting for? Download it from here today. Or try it on Amazon AWS Quick Start.

Want to know more. Check out the webcast. Register or replay at this link.

Buried Treasure – embedding IBM MQ clients and MFT Agents into applications

I haven’t been doing this blog so long that I am going to repeat myself. Or at least not yet. But last year I did a blog on why you would use MQ – and that is broadly the topic of this entry as well but it comes from a specific use-case perspective. Plus – warning – it is longer than usual – sorry. Why do businesses, in their thousands, use IBM MQ – and its many different yet critical functions? Sadly, and I say this as the Offering/Product Manager for MQ, no one wakes up in the morning and decides they want to buy more IBM MQ – but they do so because of the benefit using MQ provides for the applications that run their business.

 

IBM MQ enables the exchange of data between applications, systems, services and files with reliability and security. It does this with scalability and simplicity. It has proved itself in doing this over the last 20+ years that much of the modern online business world takes IBM MQ, and its capabilities for granted.

 

The IT infrastructure is evolving rapidly – as it always is. As such there is both growth in new applications and existing applications are being updated and enhanced. Today’s applications typically have to be more resilient than ever, but also more portable – to be deployed pretty much anywhere. In most businesses applications will be extended out to business partners as the wider ecosystem is more tightly integrated than ever before.

 

These changes drive a greater need for seamless connectivity throughout the infrastructure and it makes it more important that all business data can be simply and quickly moved inside and outside the business. So how has IBM been working on IBM MQ to enable this? And will IBM MQ be able to help all customers – whether they are trying to connect and exchange data between applications, systems, services and files – not just the latest and greatest APIs?

 

IBM MQ allows for connectivity and exchange of data through MQ Clients and MQ MFT Agents and to make it easier for these to be used in many different use cases, IBM has been making changes to the packaging and licensing of these.

One of the key changes was at the end of 2015, there was an update to the license documentation to allow for the redistribution of MQ Clients. IBM makes the MQ client libraries available for free download. These are then built into the MQ enabled applications to allow these applications to send and receive MQ messages. There is no cost for the MQ Clients – as they require a licensed MQ Queue Managers in order to function. However, until late 2015, the license prevent redistribution of these MQ Client files. This meant that if a business built the MQ Clients into an application, it wasn’t permitted to then distribute this application outside the business – i.e. it couldn’t share it with a business partner to allow that partner to work closely as an integrated partner. To allow this under the terms, the partner would need to either install the MQ Client library themselves or agree licensing terms to redistribute the MQ Client with IBM. This restriction was not helpful to these businesses or to the IBM MQ business and therefore it was changed to allow redistribution.

 

Now let’s look at a scenario – Company A uses MQ to exchange information throughout its business. It has suppliers (Company B and Company C) and it wants to streamline the manufacturing processes to enable them to get production statistics and thus help to plan for more efficient resupplies to their factories and warehouses. To do this it wants to provide them with a copy of their own in-house written application that uses MQ. Now that IBM allows for redistribution of the MQ Clients, Company A can simply provide their application to the partner companies to enable them to communicate seamlessly with no need to even be aware of the MQ Client embedded within the application. MQ messages can flow securely between the companies – and as only Company A has a MQ Queue Manager, they are the only ones licensed for MQ – and there is no additional MQ cost for this configuration. Note that companies exchanging MQ messages like this might want to make use of the MQ Internet Pass-thru feature to simplify passing messaging through their firewalls.

 

Now let’s imagine Company D. They are also part of the supply chain ecosystem for company A, and also many other businesses. But the stock control and distribution management systems are built mainly on files and file data. They keep these files updated with stock quantities and prices, but they find it simpler to keep using this method rather than online application updates and exchanges. They are used to sending these files to their customers using FTP but they always have a number of issues around FTP failures, reliability issues, and having to spend time diagnosing the problems inherent in these transfers.

 

Company A have a solution – the Managed File Transfer capability that is a part of IBM MQ Advanced. In place of regular FTP, the data inside the files can be sent as MQ messages from Company D to Company A, taking advantage of MQ’s reliability, security and management of data. And best of all Company D don’t need to change the way they handle data as they can still focus on keeping the file contents updated, but Company A can provide a program that can also embed the MQ MFT Agent which can run and extract the contents of the file and send it as MQ Messages to Company A. Just as with the MQ Client, the MQ MFT Agent is designed for easy embedding in an application, and benefits from also being redistributable under the license. The key difference is that MQ MFT Agents are free but only when they connect to MQ Queue Managers that benefit from the MQ Advanced license entitlement or are in the MQ Appliance. In providing this application making use of the MFT Agent to Company D, Company A is taking advantage of the recent change to make the Agent license redistributable, as well as the fact there is now no cost to embed MFT Agents and distribute them anywhere, as long as they connect to their MQ Advanced Queue Managers. Also, the packaging changed to ensure the MFT Agent was available as a standalone zip file for easier embedding.

 

As a business, your buried treasure may be hidden in your data. You owe it to yourself to ensure it is used as widely as possible and as timely as possible. But to do this you need buried treasure in your applications as well – and this time the buried treasure is the MQ Clients and MQ MFT Agents you can now embed in those applications. Hidden in your code, but providing value every day – maybe not buried treasure, but the goose that lays golden eggs?

Not too much of a good thing: MQ V9.0.3

After a gap of a few months I blogged earlier today about deploying MQ Queue Managers in a DMZ so it might seem a bit much to be blogging again so soon. However I will try to keep it short and snappy so you find these entries like a Japanese meal – small portions, but so many courses! And of course, delicious.

So it wasn’t long ago – just March – when I blogged about MQ V9.0.2 on MQ and MQ Advanced on distributed platforms and MQ V9.0.2 on the MQ Appliance. Remember that IBM is delivering MQ V9 as a continuous delivery release. This means that we deliver smallish amounts of hopefully easily consumable and usable function. And these functions, on the whole, will build incrementally to deliver eventually a substantial piece of new function.

 

One of these ongoing deliverables, that has been building over the last few releases is the growing REST API for administration of MQ. New capabilities in this release include read and update of the queue manager configuration, plus querying of the status.

 

Also, on top of the enhancements made to MQ Managed File Transfer, available with MQ Advanced or MQ Appliance, delivered in MQ V9.0.0, V9.0.1 and V9.0.2, there are even more usability enhancements in this release, focusing on problem determination when there may have been an issue in the completion of a file transfer. This is in addition to the license changes made recently that makes this far more attractive for deploying MQ MFT Agents widely through the business.

 

And for the MQ Appliance there was an update to allow an easier transition for some configurations to move to use the end to end encryption provided by MQ AMS when some MQ Clients may not support it, by doing the encryption on the MQ Appliance rather than the MQ Client side.

 

There are now announcement letters for MQ V9.0.3 and MQ Appliance V9.0.3 updates published but perhaps some of the most interesting updates of the MQ V9.0.3 releases was on the z/OS offering. There is already an announcement  letter about this – but this update specifically targeted the MQ Advanced for z/OS Value Unit Edition offering with a set of unique extensions for this delivered as a connector pack on top of the core MQ Advanced for z/OS VUE offering.

This connector pack included a Bridge to Blockchain, allowing MQ Advanced for z/OS VUE to query information on the Blockchain. Also there are changes to the licensing and deployment model of MQ Managed File Transfer components on z/OS. And support for MQ Advanced for z/OS VUE to publish information to the IBM Cloud Product Insights service.

 

There are some additional details on our development blog on MQ V9.0.3 here.

 

So that was a quick run through of the updates in IBM MQ V9.0.3. All you need now is some green tea to wash it down.