Posts Tagged ‘Blockchain’

Accelerating MQ with the release of IBM MQ V9.1.4

December 3, 2019

koscs_1975_914_rm_102.jpg

Sometimes change happens slowly, and gradually. We see this a lot with updates to IBM MQ over the last few years following the Continuous Delivery cycle. Innovations are rolled out, release by release. The product gains an enhancement and subsequent releases see cumulative improvements to that initial enhancement until, by the time all the enhancements get rolled up into a Long Term Support release, there is a rich set of capabilities that deliver more than the sum of the parts.

 

Enhancements don’t always happen that way. Sometimes they might be small, discrete and delivered in one go. And just occasionally, a significant enhancement comes along out of the blue, and provides potentially huge benefits in one release. In MQ V9.1.4, announcing today, customers with MQ Advanced or MQ Appliance entitlements get access to a breakthrough innovation: the Aspera FASP.io streaming gateway.

 

To explain what this is, and what it could do, it is helpful to understand Aspera technology which is outlined in my previous blog

The FASP.io streaming gateway is a new innovation using the FASP protocol. Deployed within the customer environment with a MQ Queue Manager (either with MQ Advanced entitlement or running on the MQ Appliance) configured to connect to it. Specified messages from MQ would flow from the Queue Manager to the gateway, which would then route them over FASP to another FASP.io streaming gateway which is similarly configured to another appropriately entitled Queue Manager, which would then receive the MQ message.

Screenshot 2019-12-03 at 11.24.46

So why do all this? What would be the benefit? Let’s consider the situation of a business sending MQ messages between locations over hundreds or thousands of miles, such as between New York and Singapore. Individual MQ messages can be up to 100MBs in size. And if the data originated in a file, this could be a much larger total amount of data which is split into multiple MQ messages. Using regular MQ configurations the messages would flow over TCP/IP. Sending large amounts of data over long distances using TCP/IP can be much slower than the notional line speed would suggest if the line is lossy or experiences high latency.

 

In these situations, using the Aspera FASP protocol can provide a predictable and much faster way to transport that data. Therefore, for some use cases, sending MQ messages over FASP will see them delivered much more quickly than would have been the case, which could be an extremely valuable new enhancement if that time helps the business move and respond more quickly. We hope to publish some early performance data shortly that offers an understanding of the size of the acceleration that is possible. And this is available at no additional cost for customers with MQ Advanced or MQ Appliance.

 

Other than this new feature, what else is new in MQ V9.1.4?

 

The uniform cluster feature delivered initially in 9.1.2 and enhanced in 9.1.3 is further enhanced in MQ V9.1.4. We have added support for .Net and XMS .Net applications. We have also simplified the configuration of Queue Managers in the uniform cluster and speeded the time to rebalancing, as well as provided more information about the rebalancing that takes place. This is proving to be highly attractive to customers, especially as they modernize their messaging deployments with increased numbers of queue managers providing more scalability and flexibility, while being decoupled from the application instances.

 

Security remains a critical feature of IBM MQ and we have now added in support for TLS 1.3 for the first time. This is initially supported for C/C++ MQ client applications and will be further extended in the future. In the future we will be stopping support for SSL v3, and TLS 1.0. And for MQ on z/OS the pervasive encryption support added on the z14 hardware sees increased support for some MQ datasets. MQ on z/OS announcement letters are here and here.

 

Adding to the security aspect, many businesses send and receive MQ messages through their firewall. Due to MQ persisting data in storage, IBM doesn’t recommend deploying MQ Queue Managers in the DMZ, and instead has provided the MQ Internet PassThru (MQ IPT) to be deployed in the DMZ. This proxy has been available as a Supportpac from IBM, and although it was fully supported, this delivery outside MQ made it hard for some customers to deploy and use it. With MQ V9.1.4, MQ IPT is now a part of the MQ package and therefore should be more widely available for deployment. And for customers with MQ Advanced entitlement, we have added HSM support for MQ IPT to enable the digital security keys to make use of this enhanced security option.

Screenshot 2019-12-03 at 11.25.16

The MQ Managed File Transfer feature of MQ Advanced also continues to be enhanced. The MFT Agent is now Highly Available, with another instance able to continue to transfer in the case of a failure. And also, the REST API support for MQ MFT is further extended with support for Create Monitor added.

 

Also, for MQ Advanced, the MQ Bridge to Blockchain adds support for Hyperledger Fabric for improved interaction support between MQ and the Blockchain

 

The last point I will mention is around Red Hat OpenShift. Following the acquisition of Red Hat by IBM, the MQ Advanced certified container now supports deployment directly on Red Hat OpenShift, without the need for IBM Cloud Private. The MQ Advanced certified container can either be deployed on its own in OpenShift or as a part of the IBM Cloud Pak for Integration.

I did a webinar to cover the new MQ V9.1.4 content – you can find the replay link here on the IBM Middleware Community

There is also our official blog update for this release.

As with every release, IBM MQ moves forward. Sometimes in small steps and sometimes it accelerates into the future, just like a MQ message sent using the FASP.io gateway. Forza! MQ.

Inline-SIM_IMG_513581

Queuing around the Block – How IBM MQ connectivity helps make Blockchain real for business

June 17, 2019

Screenshot 2019-06-17 at 17.32.09

The IT market moves in cycles. It reinvents or rediscovers technology on a regular basis. It might apply old technology to new problems or apply new technology to old problems. This can lead to looking at new or updated solutions and thinking that it must replace an existing technology. We find this happens a lot where people assume a new solution is a replacement or an alternative to IBM MQ which is widely used in many different use cases and has been around for years even though there is no real overlap or similarity. There is always someone new who wants a shot at the title…

shotatthetitle

Blockchain is currently exploring the different opportunities that exist for it today in the marketplace. We have probably moved on from where we were a few years ago when it was being touted as the solution to everything, to where we are now where it is being used in a number of different real customer use cases and we are seeing where it fits and adds value. These examples tend to vary according to the industry where it is being deployed, and also it depends on which Blockchain technology is being used with solutions based on Hyperledger Fabric (IBM), Ethereum (ConsenSys), and Corda (R3) all seemingly getting traction in different areas at this early stage in the market.

Screenshot 2019-06-17 at 17.57.32.png

Now might be a good time to talk about both IBM’s Hyperledger Fabric based Blockchain and a messaging product like IBM MQ, contrasting them and showing how and why they should work together. MQ is designed to do just one thing – to move data in the form of messages between applications, systems and services, and to do so reliably and securely. The use case for this is to enable applications to exchange data that normally then triggers some action or activity. This might happen infrequently but typically in many customer environments there are millions or billions of these messages every day. These messages aren’t just representing the data flowing around the business and between the applications, they actually are the data flowing between these MQ enabled applications. Anything that actually happens in the business as a result of the data in these messages will take place due to the actions of these applications in the processing of this data, or in applications that are triggered into action on the basis of these messages being sent or received. If money is being transferred between accounts, then the applications are making the debit or credit, but on the basis of the information passed in the message. And a response might be generated to confirm the action back to the originator, but these actions are all discrete.

Screenshot 2019-06-17 at 18.30.16

How is this similar or different to blockchain? First while blockchain might mean different things to different people, it is primarily seen as a type of distributed ledger where multiple parties are agreeing on a set of actions that might relate to the exchange of physical or digital assets but could also cover many other different exchanges. To enable these exchanges there might be actions required in the physical world, but there will certainly be a need to have corresponding digital exchanges of information to allow the internal systems of each participant to be updated to reflect the current or changing state of the items on the blockchain. These additional digital updates would most likely involve the exchange of multiple MQ messages between application to ensure the reliable update of systems based on the change of information.  Each individual update on the blockchain would likely trigger thousands of related MQ messages as the back end systems are updated in sync with the blockchain, or the blockchain gets updated as a result of these MQ triggered updates.

Screenshot 2019-06-17 at 16.22.17

It is because of this dependent relationship between MQ and Blockchain implementations that IBM has included a Bridge to Blockchain as a part of the MQ Advanced entitlement. As businesses start to explore how to leverage Blockchains like IBM’s Hyperledger, either as a global payments solution, or a food trust solution or anything else, knowing it will be simple to ensure that MQ enabled applications can be connected to the blockchain and to ensure that any updates on either side are accurately reflected will help to accelerate solutions, either as a proof point, or in production.

In this instance, although there is queueing, it is message queueing so there is no need to stand in line and wait before starting your blockchain journey. Move up to MQ Advanced and get on with making Blockchain real.

queuing round

 

Everyone gets the point. MQ V9.1 delivers the latest features in a Long Term Support release.

July 3, 2018

Screen Shot 2018-07-03 at 09.42.14They say anticipation is half the fun. And one of the good things about the split release approach for MQ with Continuous Delivery releases and Long Term Support Releases is that as new function is developed and made available in the CD stream, customers intending to use the LTS release can build their anticipation for the new function for up to 2 years.

 

Of course, there is nothing to stop early experimentation with the CD releases even though you may be waiting for the LTS availability. But the good news is that the wait is now over and IBM has published the announcement letter for MQ V9.1 and MQ Advanced on distributed platforms here. Also MQ V9.1 is being announced for the MQ Appliance, as well as a new model of the MQ Appliance – the M2002. You can read that announcement letter here, and a blog about it here. Also we have announced MQ V9.1 for z/OS – there is an announcement letter here for the MLC offering, and another announcement letter for MQ Advanced for z/OS VUE and other z/OS OTC offerings here.

 

Has it been worth the wait? What has been the most anticipated new capability? It’s not like a Christmas present where you are not sure what’s under the tree. Almost every feature, function and enhancement in MQ V9.1 has been already available in one of the CD releases, so there shouldn’t be much of a surprise. You can read some of my past blog entries covering the prior V9.0.x releases (V9.0.1, V9.0.2, V9.0.3, V9.0.4, and V9.0.5)

 

And don’t forget than the previous LTS release – MQ V9.0 included important updates that have proved very useful such as the enhancements to MQ AMS providing end to end encryption, including encryption at rest without performance impacts, which can be very helpful in addressing GDPR requirements.

 

However, let’s cover here some of the most interesting areas of focus over the last couple of years of function, and which ones seem to have attracted the most customer interest.

 

There are many different areas of enhancement, which hopefully means pretty much all users have something to interest them.

Screen Shot 2018-07-03 at 09.52.28

Simple and more powerful Administration

  • MQ Console – a customized browser based for configuration and operations
  • REST API for admin – an extensive set of APIs enabling new tools to be written using REST HTTP calls, usable across older releases as well
  • Improved awareness of MQ activities and logging – Publishing MQ statistics to Prometheus and Grafana; forwarding MQ error logs to ElasticSearch or Splunk; Error logs output JSON for improved parsing
  • Automation of Linear Logging – simplifying the operations and administration of logging and management of those logs.

 

Supporting Developers

  • REST API for messaging – Enabling developers writing simple applications and micro-services to access MQ capabilities.
  • Additional API and protocol support – as well as publishing a new online tutorial for using MQ
  • Connecting to Salesforce – the MQ bridge to Salesforce allows for the 2 way publishing of information between SalesForce and MQ
  • The MQ Bridge to blockchain – only available for MQ Advanced or MQ Appliance customers.

RDQM1

High Availability and Disaster Recovery without complexity and cost

  • Replicated Data Queue Managers for HA – synchronous replication across 3 nodes using local disks instead of network attached storage.
  • Replicated Data Queue Managers for DR – manual failover with synchronous or asynchronous replication across 2 nodes.
  • RDQM requires MQ Advanced licenses. But with specific licenses to reduce cost.

 

Managed File Transfers

  • Licensing, packaging and pricing changes. MFT Agents are now free with MQ Advanced or MQ Appliance, and both embeddable and redistributable.
  • FTP Protocol Bridge enhancements
  • Improved reliability and monitoring for Transfers

 

 

z/OS enhancements

Many of the updates described above also apply to MQ on z/OS. There are also some additional enhancements specific to z/OS

  • AMS Confidentiality Performance. MQ Advanced for z/OS VUE sees enhancements in performance of this feature in MQ V9.1
  • Extended deployment for MFT – with MQ Advanced for z/OS VUE.
  • The MQ Bridge for blockchain now using the Hyperledger Composer API to build out the connectivity.
  • Connecting CICS and MQ – Java programs running on a CICS Liberty JVM server can now use MQ classes for JMS to access MQ capabilities.

 

AS MQ now moves to MQ 9.1, this time the point is available for everyone. All the features above, and more I haven’t had a chance to describe will be available later in July 2018. Whether deploying in on-premises environments, on physical Appliances, on VMs, in containers, on private clouds like IBM Cloud Private, or public clouds like IBM Cloud, AWS, or Azure, the Long Term Support release now means the 2 years of functional enhancements, tested already in multiple Continuous Delivery releases are now available for more to use.

UPDATE: MQ V9.1 now available as of July 23rd 2018. Read more here.

And there is plenty more to come. Watch this space both for more updates and use cases of these features, and well as future updates in the next Continuous Delivery releases.

Putting out a new release like IBM MQ V9.0.5 is more than a 9-5 job

March 16, 2018

9-5clocks

At least in the UK, the traditional hours worked in a day job were 9 to 5. You would ‘clock-in’ at 9am and leave at 5pm. I guess it is common as there was a 1980s film called “9 to 5” starring Dolly Parton. These days office life is rather more flexible, and certainly the idea of clocking in and out at fixed times is gone.

 

For 25 years, virtually every major IT infrastructure has been able to rely on the secure and reliable exchange of data between applications and systems thanks to IBM MQ. Previously called MQSeries, then WebSphere MQ, this software offering, developed in the IBM Hursley Lab in the UK has been a critical part of the business world. So much so that most people living their lives have no idea they use IBM MQ so much on a daily basis as it ‘just works’.

 

There is a great team of developers who work hard day-in and day-out to enhance and update IBM MQ, and . We have now released IBM MQ V9.0.5, going GA on Friday March 16th. And our developers have worked for months, giving up evenings and weekends to not just add new features, but to make sure it is another offering that works when put into use. So not 9-5 at all.

 

Now for some customers this will be more of a prelude to the main act. This is referring to V9.0.5 being a Continuous Delivery release. When we brought out V9.0 we split it into 2 streams: Continuous Delivery and Long Term Support. This release marks the final release in the initial set of Continuous Delivery releases. The next release will be the first of a new Long Term Support release. And customers can expect that the functions delivered in the 5 CD releases will be made available in the new Long Term Support release.

 

When that new LTS release is available, you can expect me to summarize all the new features, but for now in this blog I will call out a few of the new features in V9.0.5.

 

The new Easy HA feature (Replicated Data Queue Managers) delivered in MQ Advanced V9.0.4 gets updated to add support for a Disaster Recovery mode, with manual takeover after either synchronous, or asynchronous replication between a pair of MQ servers.

 

The MQ Managed File Transfer capability, available with MQ Advanced or MQ Appliance gets the first support for the REST API admin interface for listing current transfers and querying MFT Agent status.

 

MQ Advanced itself will do more to identify itself when it is installed, and so prevent compliance issues, and ensures that components can recognize Queue Managers.

 

Other updates include a MQ Console refresh, and for customers who use MQ with WebSphere Application Server, performance enhancement through implicit syncpointing.

 

For MQ Appliance users there is an enhancement for better reliability by allowing aggregated IP interfaces for the Floating IP feature. This removes a potential single point of failure.

 

And for users of MQ Advanced for z/OS Value Unit Edition there have been improvements including enhancements to MQ AMS which will see increased performance.

MQ clouds puttenham

Perhaps even more exciting is the new availability of a hosted instance of MQ on the Cloud. More about this can be found here, but it creates a great opportunity to quickly and easily make use of MQ without needing to install, deploy or manage the environment. Just configure and go! Nice that after 5 years of talking about it on this blog we have an explicit offering running in the cloud. This is of course alongside MQ already being able to run in AWS as a QuickStart. Or deployed as containers in IBM Cloud private.

 

As well as looking forward in the future to a new Long Term Support release, the statement of direction indicated that the Blockchain bridge, available in MQ Advanced, will be updated to be based on the Hyperledger Composer interfaces. And additionally, customers deploying MQ in containers will in the future be able to track the size of the container, and the duration of use, and license based on that container size, rather than the full capacity of the system where the container is running. The intent will be to support existing pricing metrics such as PVUs and VPC monthly metrics, but also a future VPC Hourly metric.

ibmthink

IBM MQ, along with many other IBM and business partner solutions will be some of the highlights discussed at IBM Think in Las Vegas running March 19th-22nd. I will be there and I hope to see some of you there as well. Famously Las Vegas never sleeps, so I guess that’s something else that’s not 9 to 5. Lucky we have IBM MQ V9.0.5 now though.

9to5dolly

 

Two steps forward, no steps back with IBM MQ V9.0.4

October 24, 2017

hopscotch

Compromise is everywhere. We are told to take the rough with the smooth. The easy with the hard. The quick win and the hard slog. And with software we often have to accept compromises. Especially so these days with the drive for new function forcing some compromises with stable deployments.

Not so with the latest update to the MQ family of products. For the last 15 months IBM has been delivering updates to MQ using a Continuous Delivery stream. There have been many useful additions, but they have always required adoption of the latest version to take advantage of the new features. With the latest update moving to MQ V9.0.4, there are even more substantial updates of useful features for both base MQ and MQ Advanced. However in recognition of the need for customers to keep some systems back-level while also wanting to take advantage of new features, some of these updates are designed to allow existing deployed systems to take advantage of the new capabilities, both without being updated and without breaking the Continuous Delivery and Long Term Support principles.

In addition to this extremely useful update, which I will get to in a minute, which can be used across the entire MQ estate, there are some groundbreaking updates that will allow huge changes in the way MQ is used, deployed and managed in this update. It is more leaps forward rather than steps forward.

For MQ Advanced we have 3 key new capabilities:

  • A new ‘easy HA’ feature – Replicated Data Queue Managers
  • More flexible Managed File Transfer deployments
  • Availability of an enhanced Blockchain bridge

For MQ Base (which is part of MQ Advanced) there are a number of other enhancements

  • Additional commands supported as part of the REST API for admin
  • Availability of a ‘catch-all’ for MQSC commands as part of the REST API for admin
  • Ability to use a single MQ V9.0.4 Queue Manager as a single point gateway for REST API based admin of other MQ environments including older MQ versions such as MQ V9 LTS and MQ V8.
  • Ability to use MQ V9.0.4 as a proxy for IBM Cloud Product Insights reporting across older deployed versions of MQ
  • Availability of an enhanced MQ bridge for Salesforce
  • Initial availability of a new programmatic REST API for messaging applications

 

All of these features are called out in the new announcement letter for MQ V9.0.4 here. And there are further updates available for the MQ Appliance listed in the specific announcement letter for it here and in another blog entry here. There are also announcement letters for IBM MQ z/OS V9.0.4 and IBM MQ Advanced for z/OS VUE V9.0.4

However, let’s try and call out some details of the key points of the MQ V9.0.4 update below:

RDQM1

The new High Availability feature (officially described as Replicated Data Queue Managers or RDQM) provides a significant new way to configure High Availability. It is only available for MQ Advanced users on x86 Red Hat Linux. It is designed as a 3 node system which uses replication of messages and logs between the local disks available to each Queue Manager. This style of replication of local disks was previously only available with the MQ Appliance. As moving to this new style of HA will allow customers to stop using network storage for MQ, we anticipate it will be very popular. As well as the disk level replication, Floating IP will be used to help applications move seamlessly to a failover QM. And 3 nodes help to prevent ‘split-brain’ situations where 2 nodes are simultaneously active.

The licensing of the above deployment requires MQ Advanced as already stated. However as long as all Queue Managers on all 3 nodes are Replicated Data Queue Managers, and all 3 systems are the same capacity, then only one node needs to have a MQ Advanced license entitlement. The other 2 nodes can be licensed with MQ Advanced High Availability Replica parts (these parts used to be called Idle Standby parts).

RESTproxy

The changes to the REST API for admin are also significant. Over the last few releases more and more ‘verbs’ have been added to allow REST API calls to configure and manage MQ. This was designed to allow more modern tools to be built as an alternative to MQSC and PCF based tooling. The latest V9.0.4 release adds more verbs and also a way to call the remaining equivalent MQSC functions within a REST API structure. However what is perhaps more interesting is that a single V9.0.4 Queue Manager can now act as a ‘gateway’ Queue Manager to allow these new REST API driven tools to configure and manage Queue Managers that are older and don’t include this new Continuous Delivery function. This is hopefully a very good way of providing the best of both worlds. Allowing the older production Queue Managers to remain deployed but still take advantage of new features.

Similar to this ‘bridge’ feature is one for IBM Cloud Product Insights, where the ability to publish deployed Queue Manager data to Cloud Product Insights was limited to releases on the Continuous Delivery stream, but now a single V9.0.4 Queue Manager enables older installs to publish data to this useful dashboard tool.

The MQ bridge for Salesforce has been enhanced to allow MQ to publish data into Salesforce, instead of simply receiving push notifications from Salesforce.

Customers with MQ Advanced who want to explore the possibilities offered by Blockchains now can deploy a bridge which enables MQ applications to query the Blockchain, and also provide data input into it. An earlier version of this was available only to customers with MQ Advanced for z/OS VUE, but this version is available to customers using MQ Advanced on distributed platforms.

MQ Advanced customers also get more flexibility in how they can deploy the file logger in MQ Managed File Transfer scenarios, as this logger can now be deployed on a different machine to the MQ Queue Manager.

And finally, feedback from customers told us that developers were looking to make sure of MQ, but with fewer dependencies, to free them up from client and language bindings. As such we have also added the first layer of support for a new set of programmatic REST APIs for messaging applications. This will replace the previous HTTPBridge function which has already been deprecated. Over the next few releases it is hoped that more functions will be supported in this REST API for messaging to allow additional messaging calls to be supported.

Counting up the advances it does look like it is more than 2 steps forward, and certainly no steps back. And with the ability to use some of these features alongside your older MQ releases, what are you waiting for? Download it from here today. Or try it on Amazon AWS Quick Start.

Want to know more. Check out the webcast. Register or replay at this link.

Not too much of a good thing: MQ V9.0.3

June 6, 2017

After a gap of a few months I blogged earlier today about deploying MQ Queue Managers in a DMZ so it might seem a bit much to be blogging again so soon. However I will try to keep it short and snappy so you find these entries like a Japanese meal – small portions, but so many courses! And of course, delicious.

japanese meal

So it wasn’t long ago – just March – when I blogged about MQ V9.0.2 on MQ and MQ Advanced on distributed platforms and MQ V9.0.2 on the MQ Appliance. Remember that IBM is delivering MQ V9 as a continuous delivery release. This means that we deliver smallish amounts of hopefully easily consumable and usable function. And these functions, on the whole, will build incrementally to deliver eventually a substantial piece of new function.

 

One of these ongoing deliverables, that has been building over the last few releases is the growing REST API for administration of MQ. New capabilities in this release include read and update of the queue manager configuration, plus querying of the status.

 

Also, on top of the enhancements made to MQ Managed File Transfer, available with MQ Advanced or MQ Appliance, delivered in MQ V9.0.0, V9.0.1 and V9.0.2, there are even more usability enhancements in this release, focusing on problem determination when there may have been an issue in the completion of a file transfer. This is in addition to the license changes made recently that makes this far more attractive for deploying MQ MFT Agents widely through the business.

 

And for the MQ Appliance there was an update to allow an easier transition for some configurations to move to use the end to end encryption provided by MQ AMS when some MQ Clients may not support it, by doing the encryption on the MQ Appliance rather than the MQ Client side.

 

There are now announcement letters for MQ V9.0.3 and MQ Appliance V9.0.3 updates published but perhaps some of the most interesting updates of the MQ V9.0.3 releases was on the z/OS offering. There is already an announcement  letter about this – but this update specifically targeted the MQ Advanced for z/OS Value Unit Edition offering with a set of unique extensions for this delivered as a connector pack on top of the core MQ Advanced for z/OS VUE offering.

This connector pack included a Bridge to Blockchain, allowing MQ Advanced for z/OS VUE to query information on the Blockchain. Also there are changes to the licensing and deployment model of MQ Managed File Transfer components on z/OS. And support for MQ Advanced for z/OS VUE to publish information to the IBM Cloud Product Insights service.

 

There are some additional details on our development blog on MQ V9.0.3 here.

 

So that was a quick run through of the updates in IBM MQ V9.0.3. All you need now is some green tea to wash it down.

japan green tea