Archive for the ‘IBM MQ’ Category

Reaching for SANity with the IBM MQ Appliance

October 24, 2017

Appliance reaching

We all like to keep our word, and even though a Statement of Direction is not always a promise, at IBM we like to try and deliver what we set out, as long as the market and need hasn’t changed.

And for customers who have been following the MQ Appliance since the start, back in March 2015, you might recall that there was a Statement of Direction about adding support for the fibre channel cards that were part of the MQ Appliance hardware to allow connectivity to a SAN for customers who wanted that. And we are delighted that as part of the MQ V9.0.4 update on the MQ Appliance, that IBM is delivered on that Statement of Direction.

MQ V9.0.4 is a very significant update for MQ and most of the updates that are not specific to the MQ Appliance are described in my other blog entry here. However, in addition to those features there are specific MQ Appliance updates, and a dedicated announcement letter which I will try to explain in more detail here.

Firstly, the SAN support. One of the reasons the MQ Appliance has been so popular with customers is the built-in storage (2x 3.2TB SSDs in a RAID 1 configuration) and the simple HA configuration allowing a pair of MQ Appliances to replicate messages and log data between them on a QM by QM basis with just a couple of clicks on setup. And a lot of the updates to the MQ Appliance has been to support the wider use of this popular deployment use case.

Appliance SAN options

However, some customers have always wanted us to add support for SANs for some of their use cases. One use case was that 3.2TB of onboard storage might not be enough. If you consider when a consuming application might fail over a long weekend and the queue depths might get very high. One of our customers recently said, “an empty queue is a happy queue” and this is true, but in the case of a protracted failure you want to ensure your queues can hold all the messages. So, if you are concerned about this scenario, then you might want to have the Queues supporting that use case on external storage where there is no effective limit to queue depth.

The other use case, which tends to come up more commonly in customer discussions is around the thought of “consistency groups”. This is when you are trying to recover from a site failure at your disaster recovery location. This is typically a manual task, unlike the automated High Availability configuration. Part of this manual task will be to establish the last known transactions of not just MQ but other parts of the IT infrastructure such as applications and databases. This is likely to be easier if all the data points from all these ‘moving parts’ are stored on a common storage area, and this is replicated consistently between the SAN on one site and the SAN on the disaster recovery site. So this use case is now supported by the MQ Appliance with customer selected Queue Managers able to choose the SAN storage instead of the internal storage and have a MQ Appliance in the disaster recovery site read the replicated messages and data from the SAN there.

A second update to the MQ Appliance specific features is to provide customers with a quicker and simpler way to ensure that Queue Managers on the MQ Appliance have the best allocation of resources on the MQ Appliance. When Queue Managers are initially created the MQ Appliance determines how much space to allow to them. However if workload on a Queue Manager grows faster than other Queue Managers, it is likely that the allocated resources might need to grow to match the likely workload. However, it hasn’t been easy to do that. But with the MQ V9.0.4 update on the MQ Appliance IBM has added dynamic disk allocation, enabling resources for a Queue Manager to be increases even after it was initially created. This will make the ongoing operation and support of production workloads on the MQ Appliance quicker and simpler.

Appliance MFT

Finally, an update off the MQ Appliance will have a positive impact on potential use cases for the MQ Appliance. Virtually every customer still moves large amounts of data around their business as files and file contents. Much of this is unmanaged, insecure and unreliable, and for a number of years IBM has provided a solution: MQ Managed File Transfer. This enables file contents to move from point to point over the MQ network as MQ messages, taking advantage of MQ’s reliability, security and manageability. Part of the MQ Managed File Transfer functions was a logger to track which files were sent over MQ, and this, if used, always needed to run on the same machine as the Queue Manager. This prevented some MQ MFT use cases from being deployed in environments where the MQ Appliance was the only MQ Queue Manager. Now, with MQ V9.0.4 the file logger component of MQ MFT can be deployed remotely from the Queue Manager, meaning it can be used in locations by pointing to a remote MQ Appliance, without the need to install anything on the MQ Appliance, which has always been prohibited. It’s always good to allow customers more flexibility in deployment.

This combination of enhancements, along with the many non-Appliance specific MQ updates in the MQ V9.0.4 release means that there should be a lot of increased opportunities to consider the MQ Appliance as the right deployment option. After all many customers who review it with regards to overall costs find it has the lowest Total Cost of Ownership, as well as outstanding reliability.

Advertisements

Two steps forward, no steps back with IBM MQ V9.0.4

October 24, 2017

hopscotch

Compromise is everywhere. We are told to take the rough with the smooth. The easy with the hard. The quick win and the hard slog. And with software we often have to accept compromises. Especially so these days with the drive for new function forcing some compromises with stable deployments.

Not so with the latest update to the MQ family of products. For the last 15 months IBM has been delivering updates to MQ using a Continuous Delivery stream. There have been many useful additions, but they have always required adoption of the latest version to take advantage of the new features. With the latest update moving to MQ V9.0.4, there are even more substantial updates of useful features for both base MQ and MQ Advanced. However in recognition of the need for customers to keep some systems back-level while also wanting to take advantage of new features, some of these updates are designed to allow existing deployed systems to take advantage of the new capabilities, both without being updated and without breaking the Continuous Delivery and Long Term Support principles.

In addition to this extremely useful update, which I will get to in a minute, which can be used across the entire MQ estate, there are some groundbreaking updates that will allow huge changes in the way MQ is used, deployed and managed in this update. It is more leaps forward rather than steps forward.

For MQ Advanced we have 3 key new capabilities:

  • A new ‘easy HA’ feature – Replicated Data Queue Managers
  • More flexible Managed File Transfer deployments
  • Availability of an enhanced Blockchain bridge

For MQ Base (which is part of MQ Advanced) there are a number of other enhancements

  • Additional commands supported as part of the REST API for admin
  • Availability of a ‘catch-all’ for MQSC commands as part of the REST API for admin
  • Ability to use a single MQ V9.0.4 Queue Manager as a single point gateway for REST API based admin of other MQ environments including older MQ versions such as MQ V9 LTS and MQ V8.
  • Ability to use MQ V9.0.4 as a proxy for IBM Cloud Product Insights reporting across older deployed versions of MQ
  • Availability of an enhanced MQ bridge for Salesforce
  • Initial availability of a new programmatic REST API for messaging applications

 

All of these features are called out in the new announcement letter for MQ V9.0.4 here. And there are further updates available for the MQ Appliance listed in the specific announcement letter for it here and in another blog entry here. There are also announcement letters for IBM MQ z/OS V9.0.4 and IBM MQ Advanced for z/OS VUE V9.0.4

However, let’s try and call out some details of the key points of the MQ V9.0.4 update below:

RDQM1

The new High Availability feature (officially described as Replicated Data Queue Managers or RDQM) provides a significant new way to configure High Availability. It is only available for MQ Advanced users on x86 Red Hat Linux. It is designed as a 3 node system which uses replication of messages and logs between the local disks available to each Queue Manager. This style of replication of local disks was previously only available with the MQ Appliance. As moving to this new style of HA will allow customers to stop using network storage for MQ, we anticipate it will be very popular. As well as the disk level replication, Floating IP will be used to help applications move seamlessly to a failover QM. And 3 nodes help to prevent ‘split-brain’ situations where 2 nodes are simultaneously active.

The licensing of the above deployment requires MQ Advanced as already stated. However as long as all Queue Managers on all 3 nodes are Replicated Data Queue Managers, and all 3 systems are the same capacity, then only one node needs to have a MQ Advanced license entitlement. The other 2 nodes can be licensed with MQ Advanced High Availability Replica parts (these parts used to be called Idle Standby parts).

RESTproxy

The changes to the REST API for admin are also significant. Over the last few releases more and more ‘verbs’ have been added to allow REST API calls to configure and manage MQ. This was designed to allow more modern tools to be built as an alternative to MQSC and PCF based tooling. The latest V9.0.4 release adds more verbs and also a way to call the remaining equivalent MQSC functions within a REST API structure. However what is perhaps more interesting is that a single V9.0.4 Queue Manager can now act as a ‘gateway’ Queue Manager to allow these new REST API driven tools to configure and manage Queue Managers that are older and don’t include this new Continuous Delivery function. This is hopefully a very good way of providing the best of both worlds. Allowing the older production Queue Managers to remain deployed but still take advantage of new features.

Similar to this ‘bridge’ feature is one for IBM Cloud Product Insights, where the ability to publish deployed Queue Manager data to Cloud Product Insights was limited to releases on the Continuous Delivery stream, but now a single V9.0.4 Queue Manager enables older installs to publish data to this useful dashboard tool.

The MQ bridge for Salesforce has been enhanced to allow MQ to publish data into Salesforce, instead of simply receiving push notifications from Salesforce.

Customers with MQ Advanced who want to explore the possibilities offered by Blockchains now can deploy a bridge which enables MQ applications to query the Blockchain, and also provide data input into it. An earlier version of this was available only to customers with MQ Advanced for z/OS VUE, but this version is available to customers using MQ Advanced on distributed platforms.

MQ Advanced customers also get more flexibility in how they can deploy the file logger in MQ Managed File Transfer scenarios, as this logger can now be deployed on a different machine to the MQ Queue Manager.

And finally, feedback from customers told us that developers were looking to make sure of MQ, but with fewer dependencies, to free them up from client and language bindings. As such we have also added the first layer of support for a new set of programmatic REST APIs for messaging applications. This will replace the previous HTTPBridge function which has already been deprecated. Over the next few releases it is hoped that more functions will be supported in this REST API for messaging to allow additional messaging calls to be supported.

Counting up the advances it does look like it is more than 2 steps forward, and certainly no steps back. And with the ability to use some of these features alongside your older MQ releases, what are you waiting for? Download it from here today. Or try it on Amazon AWS Quick Start.

Want to know more. Check out the webcast. Register or replay at this link.

What is GDPR and how does it affect IBM MQ use?

September 26, 2017

Imagine as a business that you were given the opportunity to grow turnover by 4%, at a stroke, or to increase revenue by €20 million. This would be certainly a key focus area. Well, there is good news and bad news, because these figures are accurate, and can apply to your business, but they are fines that will be applied to your business if you are in breach of the GDPR regulations that will be enforced in May 2018. And your business will be liable for whichever is the greater amount. As such, it is a subject that demands attention. And it can apply to any business if it concerns the personal data of EU citizens, even if your business is not based in the EU.

Screen Shot 2017-09-26 at 11.42.31

GDPR is a complex piece of legislation, and not one that can be solved through any one single act or solution. It requires understanding of the legislation, and then a thorough review of existing governance and processes, especially those involved in data handling and security, and ensuring that all people involved in all these aspects are aware of changes being made, and why these are important and must be complied with.

Amongst the key criteria for GDPR compliance are a number of aspects that are likely to need to be reflected in the choices made in MQ deployment to help to meet the compliance needs. However, it must be understood that taking this action around MQ alone will not achieve GDPR compliance, but simply be a part of that compliance.

Given that GDPR is concerned with data protection, it should be clear that data privacy is key in reaching compliance. This isn’t the only aspect, as there are multiple additional aspects such as the ‘right to be forgotten’ providing a requirement to remove data, and also the need to track the movement of data through all systems. Considering all these aspects together, it should be clear that reducing the movement of data to modes of transport that allow for end to end encryption, as well as logging, reporting and monitoring for the movement of data are likely to be seen as essential to aid in GDPR compliance.

Steps that you can take to help demonstrate your MQ environment is helping your business comply with GDPR regulations:

  • As well as using authentication and authorization to secure your MQ system, end-to-end encryption is available as part of MQ Advanced and MQ Appliance to supplement this
    • Using end-to-end encryption could be the only way to protect personal data wherever in the organization it moves, as it moves as it reduces the need to ensure control of all intermediate systems to protect the data.
    • End-to-end encryption can help to demonstrate privacy by design as part of your compliance verification process.
  • IBM MQ can be configured to log all messages and accesses which can be used to track all movement of data, and who had access to it.
    • Making use of the tools available with IBM MQ to monitor and report on message movement can be an essential part of good data governance
    • Building new tools using the REST API for MQ admin to offer a custom view of MQ configuration and operation could be a critical aspect of generating reports on data movement and protection.
  • Holding personal data in files is widespread in many businesses
    • Holding and moving those files around your business could add further vulnerabilities.
    • File data needs to be handled and managed with the same care as application data as it is just as likely to be personal data that needs to be protected.
    • As part of the MQ Advanced and MQ Appliance entitlement, businesses can move file data securely, and with monitoring and tracking, through the MQ network, helping to meet GDPR compliance without additional complexity

 

As mentioned at the start, there is no single solution that can address all the aspects of GDPR within your business. Ensuring your MQ environment is configured to securely move data with end to end encryption, with comprehensive logging and reporting of messaging access and movement can be a critical step in the wider compliance task.

 

Work with your IBM representative to ensure you hear more about the benefits of MQ Advanced entitlement to allow your business to move file data and to encrypt messages and data end to end, and thus reduce the risk of data being exposed in a security breach. Or review whether the MQ Appliance would be a good fit for your business, providing the same benefits in end to end encryption and file data movement.

 

Read more about MQ Advanced here: https://www-03.ibm.com/software/products/en/ibm-mq#othertab3

Read more about the MQ Appliance here: https://www-03.ibm.com/software/products/en/ibm-mq#othertab4

For additional information about how IBM can help you with GDPR see here: https://www.ibm.com/analytics/us/en/technology/general-data-protection-regulation/

GDPR robot

How fast is fast? Low Latency Messaging has a new home.

July 11, 2017

fast_image2a

For many years one of the least appreciated parts of the MQ portfolio was MQ Low Latency Messaging. This was a very special product to meet a unique market need: Financial Markets, Exchanges and Front Office trading. The requirements here were to move data in the form of messages very quickly, at the same speed to all recipients, and with the lowest possible latency. The reason for this was to offer a ‘fair and equal’ distribution of information about the market data to allow for trading to take place on a level playing field.

Unlike IBM MQ which moves messages from an application through the Queue Manager to a receiving application, the MQ Low Latency Messaging (MQ LLM) offering is a peer to peer distribution platform, which ensures that messages are delivered as quickly and directly as possible to each end point. And this is of course exactly what is required as part of a market exchange solution. As well as distributing the messages rapidly there is a MessageStore where the same messages can be persisted to keep a record of what messages were distributed and to whom.

Despite the difference between MQ LLM and IBM MQ itself, both products can exchange messages if needed. This might be to track trades happening in the front office trading platform and to send them to the back office transactional systems.

This offering has been available without significant updates for a number of years and although still highly capable and extremely fast in performance, it is time to take it to the next level. And the best way to do this is to have it enhanced further by a company with years of experience in this space. Confinity Solutions, based in Germany, has acquired the license from IBM to produce the successor product to IBM WebSphere MQ Low Latency Messaging. This new product, Low Latency Messaging from Confinity Solutions will be resold by IBM to allow both existing and new customers in this space to have a sales and support point of contact with IBM for this product. The announcement letter for this new offering, as available from IBM can be found here. And at the same time IBM is announcing the withdrawal from marketing and End of Support for the existing IBM WebSphere MQ LLM offering.

Confinity Solutions is happy to provide a 1:1 entitlement to existing MQ LLM customers who are current and up to date with their S&S entitlements. Existing customers should reach out to Confinity Solutions through their website, or via their IBM representative. Confinity Solutions offerings can be seen as the next iteration of the existing MQ LLM product, with a few important differences. It will only be available on Windows and Linux platforms. And also, instead of requiring separate licensing for the MessageStore, this is now available as a part of the Low Latency Messaging product from Confinity Solutions, but not a part of the Low Latency Messaging Core product from Confinity Solutions. Both these products, and non-production entitlements are available from IBM, and also separately purchasable ‘Elite Support’ which is in place of IBM Subscription and Support as this is a product resold by IBM, instead of being an IBM product.

IBM regular support and then extended support will continue to be available for a number of years on the existing MQ LLM offering. The IBM recommendation is to evaluate the new Low Latency Messaging Offering from Confinity Solutions as the ideal follow-on product. As it is developed from the same source code, this is the closest match to the existing offering.

After all it is generally better to be the cheetah than the impala.

Cheetah_hunting_impala-photo-safari_Masai_Mara_3285.jpg

Space is big. Really Big. If it was data it could be transferred with IBM Aspera.

July 4, 2017

The genius that was Douglas Adams wrote: “Space is big. Really big. You just won’t believe how vastly hugely mindbogglingly big it is. I mean you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space.”

The same could be said for some of the data files being created in businesses today. These could be anything from medical images to video files, collections of ERP data or data replication. I remember just a few years ago, a file that was more than a few MB was considered large. But these days it can be quite common to think nothing of files being hundreds of MB or even multiple GB.

To a great extent, with fast internal networks and lots of storage, file size is almost irrelevant these days. Except for one key area: the movement of large files, or large amounts of data over long distances. Even if you think you have a large capacity high speed network, once you start to move large amounts of data over a significant distance, you hits problems that are a combination of physics and network transfer protocols. The distance, and the need to send responses as part of the protocol means that the larger the amount of data being sent and the longer the distance, the transfer time starts to get exponentially longer.

person-pushing-rock-up-hill

Moving a large file is no problem on a high speed LAN, or even across a fast WAN link over a short distance, once you start moving across potentially thousands of miles, your transfer times become unworkable. This is the problem that IBM Aspera offerings solve by using the FASP protocol to move data at high predictable speed even over long distances.

While we are all now used to our business transactions being essentially instantaneous, this hasn’t been the case in moving large data files for business use as these might have had to be scheduled out of process time due to the time taken instead of being moved as part of a business process as would happen with transactional data.

world network links.jpg

A typical business might be using MQ to move transactional data between applications and systems. They might also be using MQ Managed File Transfer to move file contents. But with MQ’s message size limit of 100MB per message, these wouldn’t be large enough to see a problem even when moving large distances. But supposing MQ is also looking to trigger the movement of much larger files, say between the office in New York and the office in Singapore. A design file of 10GB would be impractical to send over FTP as it would take too long, but using Aspera, then depending on the network link speed it could be sent in minutes, if not seconds. This transfer then becomes possible to be just another standard part of the business process, and can be viewed in this way as an extension to IBM MQ and IBM MQ MFT.

Aspera speeds

To that point IBM recently announced a new set of parts to make it easier for customers using IBM MQ or other integration products to start using Aspera offerings to meet this need. See the announcement letter for IBM Aspera High Speed Transfer for MQ here and start to accelerate your data transfer needs.

MQ Aspera

The rewards of patience – IBM MQ V8 for HPE NonStop

July 4, 2017

Rewards of Patience

Time can be subjective and objective. Subjective time is referenced by the frame of the observer. A period of a few seconds can seem very short, or incredibly long. Putting your hand on a hot surface teaches you a second can seem a long time. Yet reading the book by Penfolds winery “The Rewards of Patience” can make you understand that 10 years can be no time at all.

How then should we measure the time it takes to release a new version of an enterprise software product where it might be used for a decade or more without change. For at the heart of key parts of the world’s infrastructure are systems that run HPE’s NonStop Server platform, and on those systems they also use IBM’s MQ software. These systems are very stable and it is a testament to that longevity that the last time that IBM released a new version of MQ on this platform was in 2006.

However, just as a good wine ages until it is ready to enjoy, so the time has now been reached for a new version of IBM MQ to be available on the HPE NonStop platform as promised in a previous Statement of Direction. This announcement on June 20th 2017 of IBM MQ V8 for HPE NonStop comes at a time when customers using the HPE NonStop platform are looking at new hardware such as the new x86 based machines now available to support the platform. The new version of IBM MQ supports both the x86 and Itanium platforms, giving customers a choice of deployment with the same function on each.

Existing customers of the prior version of MQ on the HPE NonStop platform who have been paying S&S will be entitled to move to the latest version. Customers should be careful in assessing the number of PVUs of the system they are deploying on as the x86 based NS7 machines in the HPE NonStop line are rated at 120 PVUs per core, rather than 100 PVUs per core.

As mentioned in the announcement letter there are no immediate plans to announce End of Support for the MQ V5.3.1 offering on this platform until functional equivalence is reached. Expect further updates to allow this to be achieved. Please read the announcement letter linked above to understand what has been delivered in this version to date.

An additional blog is available on DeveloperWorks.

Buried Treasure – embedding IBM MQ clients and MFT Agents into applications

June 13, 2017

treasure

I haven’t been doing this blog so long that I am going to repeat myself. Or at least not yet. But last year I did a blog on why you would use MQ – and that is broadly the topic of this entry as well but it comes from a specific use-case perspective. Plus – warning – it is longer than usual – sorry. Why do businesses, in their thousands, use IBM MQ – and its many different yet critical functions? Sadly, and I say this as the Offering/Product Manager for MQ, no one wakes up in the morning and decides they want to buy more IBM MQ – but they do so because of the benefit using MQ provides for the applications that run their business.

 

IBM MQ enables the exchange of data between applications, systems, services and files with reliability and security. It does this with scalability and simplicity. It has proved itself in doing this over the last 20+ years that much of the modern online business world takes IBM MQ, and its capabilities for granted.

 

The IT infrastructure is evolving rapidly – as it always is. As such there is both growth in new applications and existing applications are being updated and enhanced. Today’s applications typically have to be more resilient than ever, but also more portable – to be deployed pretty much anywhere. In most businesses applications will be extended out to business partners as the wider ecosystem is more tightly integrated than ever before.

 

These changes drive a greater need for seamless connectivity throughout the infrastructure and it makes it more important that all business data can be simply and quickly moved inside and outside the business. So how has IBM been working on IBM MQ to enable this? And will IBM MQ be able to help all customers – whether they are trying to connect and exchange data between applications, systems, services and files – not just the latest and greatest APIs?

 

IBM MQ allows for connectivity and exchange of data through MQ Clients and MQ MFT Agents and to make it easier for these to be used in many different use cases, IBM has been making changes to the packaging and licensing of these.

MFT Agents

One of the key changes was at the end of 2015, there was an update to the license documentation to allow for the redistribution of MQ Clients. IBM makes the MQ client libraries available for free download. These are then built into the MQ enabled applications to allow these applications to send and receive MQ messages. There is no cost for the MQ Clients – as they require a licensed MQ Queue Managers in order to function. However, until late 2015, the license prevent redistribution of these MQ Client files. This meant that if a business built the MQ Clients into an application, it wasn’t permitted to then distribute this application outside the business – i.e. it couldn’t share it with a business partner to allow that partner to work closely as an integrated partner. To allow this under the terms, the partner would need to either install the MQ Client library themselves or agree licensing terms to redistribute the MQ Client with IBM. This restriction was not helpful to these businesses or to the IBM MQ business and therefore it was changed to allow redistribution.

 

Now let’s look at a scenario – Company A uses MQ to exchange information throughout its business. It has suppliers (Company B and Company C) and it wants to streamline the manufacturing processes to enable them to get production statistics and thus help to plan for more efficient resupplies to their factories and warehouses. To do this it wants to provide them with a copy of their own in-house written application that uses MQ. Now that IBM allows for redistribution of the MQ Clients, Company A can simply provide their application to the partner companies to enable them to communicate seamlessly with no need to even be aware of the MQ Client embedded within the application. MQ messages can flow securely between the companies – and as only Company A has a MQ Queue Manager, they are the only ones licensed for MQ – and there is no additional MQ cost for this configuration. Note that companies exchanging MQ messages like this might want to make use of the MQ Internet Pass-thru feature to simplify passing messaging through their firewalls.

 

Now let’s imagine Company D. They are also part of the supply chain ecosystem for company A, and also many other businesses. But the stock control and distribution management systems are built mainly on files and file data. They keep these files updated with stock quantities and prices, but they find it simpler to keep using this method rather than online application updates and exchanges. They are used to sending these files to their customers using FTP but they always have a number of issues around FTP failures, reliability issues, and having to spend time diagnosing the problems inherent in these transfers.

 

Company A have a solution – the Managed File Transfer capability that is a part of IBM MQ Advanced. In place of regular FTP, the data inside the files can be sent as MQ messages from Company D to Company A, taking advantage of MQ’s reliability, security and management of data. And best of all Company D don’t need to change the way they handle data as they can still focus on keeping the file contents updated, but Company A can provide a program that can also embed the MQ MFT Agent which can run and extract the contents of the file and send it as MQ Messages to Company A. Just as with the MQ Client, the MQ MFT Agent is designed for easy embedding in an application, and benefits from also being redistributable under the license. The key difference is that MQ MFT Agents are free but only when they connect to MQ Queue Managers that benefit from the MQ Advanced license entitlement or are in the MQ Appliance. In providing this application making use of the MFT Agent to Company D, Company A is taking advantage of the recent change to make the Agent license redistributable, as well as the fact there is now no cost to embed MFT Agents and distribute them anywhere, as long as they connect to their MQ Advanced Queue Managers. Also, the packaging changed to ensure the MFT Agent was available as a standalone zip file for easier embedding.

 

As a business, your buried treasure may be hidden in your data. You owe it to yourself to ensure it is used as widely as possible and as timely as possible. But to do this you need buried treasure in your applications as well – and this time the buried treasure is the MQ Clients and MQ MFT Agents you can now embed in those applications. Hidden in your code, but providing value every day – maybe not buried treasure, but the goose that lays golden eggs?

Goose Golden Egg

Not too much of a good thing: MQ V9.0.3

June 6, 2017

After a gap of a few months I blogged earlier today about deploying MQ Queue Managers in a DMZ so it might seem a bit much to be blogging again so soon. However I will try to keep it short and snappy so you find these entries like a Japanese meal – small portions, but so many courses! And of course, delicious.

japanese meal

So it wasn’t long ago – just March – when I blogged about MQ V9.0.2 on MQ and MQ Advanced on distributed platforms and MQ V9.0.2 on the MQ Appliance. Remember that IBM is delivering MQ V9 as a continuous delivery release. This means that we deliver smallish amounts of hopefully easily consumable and usable function. And these functions, on the whole, will build incrementally to deliver eventually a substantial piece of new function.

 

One of these ongoing deliverables, that has been building over the last few releases is the growing REST API for administration of MQ. New capabilities in this release include read and update of the queue manager configuration, plus querying of the status.

 

Also, on top of the enhancements made to MQ Managed File Transfer, available with MQ Advanced or MQ Appliance, delivered in MQ V9.0.0, V9.0.1 and V9.0.2, there are even more usability enhancements in this release, focusing on problem determination when there may have been an issue in the completion of a file transfer. This is in addition to the license changes made recently that makes this far more attractive for deploying MQ MFT Agents widely through the business.

 

And for the MQ Appliance there was an update to allow an easier transition for some configurations to move to use the end to end encryption provided by MQ AMS when some MQ Clients may not support it, by doing the encryption on the MQ Appliance rather than the MQ Client side.

 

There are now announcement letters for MQ V9.0.3 and MQ Appliance V9.0.3 updates published but perhaps some of the most interesting updates of the MQ V9.0.3 releases was on the z/OS offering. There is already an announcement  letter about this – but this update specifically targeted the MQ Advanced for z/OS Value Unit Edition offering with a set of unique extensions for this delivered as a connector pack on top of the core MQ Advanced for z/OS VUE offering.

This connector pack included a Bridge to Blockchain, allowing MQ Advanced for z/OS VUE to query information on the Blockchain. Also there are changes to the licensing and deployment model of MQ Managed File Transfer components on z/OS. And support for MQ Advanced for z/OS VUE to publish information to the IBM Cloud Product Insights service.

 

There are some additional details on our development blog on MQ V9.0.3 here.

 

So that was a quick run through of the updates in IBM MQ V9.0.3. All you need now is some green tea to wash it down.

japan green tea

When is a wall a great wall? When it’s a firewall?

June 6, 2017

hankleycommonatlanticwall21

Today is June 6th – and the 73rd anniversary of the D-Day landings in Normandy in World War 2. There were 156000 soldiers landed who attacked the defences on those beaches – the dreaded Atlantic Wall. But they had been preparing for this and had even built walls to practice assaulting, such as the one shown above in Hankley Common in Surrey (down the road from where I live).

Not all walls can withstand assault. But they are almost all built for a specific purpose – to provide safe and secure separation. This holds true for today’s firewalls as well as historical defensive walls.

firewall

Hundreds if not thousands of IBM’s customers use IBM MQ to communicate with business partners or separate parts of their own businesses beyond their enterprise firewall. There are a number of ways to do this – including deploying MQ Internet Passthru (MQIPT), opening ports for MQ connectivity, or deploying MQ servers in the DMZ. Not all DMZs are quite as scary or indeed obvious as the one separating North and South Korea. But they exist for good reason – to protect what’s behind the firewall. There is a huge cost associated with data breaches.

koreaDMZ
The issue some customers have with deploying MQ servers in the DMZ, is that this can lead to messages being persisted to disk in the DMZ – and while devices like IBM DataPower appliances are designed to run in the DMZ this is because they are, on the whole, stateless with no information persisted. This is not the case with IBM MQ, and thus the data on the disk in the DMZ poses a concern due to the increased risk in this environment. This is the primary reason that MQ IPT is used – to avoid the persistence of MQ data here.

IBM doesn’t prevent customers deploying MQ Servers or indeed MQ Appliances in the DMZ – despite typically recommending that customer choose not to do that – there is no impact in terms of their IBM contract or support if they do – this deployment of IBM MQ is still supported – but IBM wants to make sure that customers consider the implication and risk of this (as we do with all their MQ deployment choices – as this is typically critical for their business).
Our concern with the deployment of the MQ Appliance into a DMZ has been that due to being based on the DP hardware customers might see it as addressing these concerns and deploying it as a secure solution to DMZ deployment – whereas the fundamental issue of persisted data still exists. This can be mitigated in various ways such as the end to end encryption of AMS included in the Appliance – but there is no absolute lock-down of the Appliance and therefore we have that statement included in the documentation to ensure that customers make their choice knowingly.
thisisfine

There are therefore a number of different options to allow the movement of MQ messages through the firewall without it going horribly wrong. Customers can deploy MQ or the MQ Appliance into the DMZ if they want to – taking the precautions that are sensible to mitigate risks. IBM will support them with PMRs they raise, but we would work to ensure they are aware that they can be increasing the risk of data compromise and that they should take steps to lock down the environment as much as possible, and use MQ AMS for end to end encryption if using MQ Advanced or MQ Appliance.

greatwall

Walls are essential, but the best walls make the best neighbours, and with IBM MQ deployed successfully and securely, you can ensure your firewall is a great wall, but that it doesn’t lock your business in – but helps it to grow with safety.

Building higher – IBM MQ V9.0.2

March 16, 2017

When a building is being constructed, it can be hard, from moment to moment to see progress. Yes – you see lots of activity. Lots of people are busy doing all sorts of important jobs, but it can be hard to see what they are all doing. You need to find a way to keep track of how they are doing. What progress are they making, and what milestones are they hitting.

building construction

In delivering updates to IBM MQ, now that we are on a ‘Continuous Delivery’ schedule, we set these milestones of deliveries around 3 times a year. We don’t plan to do IBM announcement letters with every update, but will do blogs here and elsewhere for some of the updates, with official announcements for others. For IBM MQ V9.0.1, there was an announcement letter, and I blogged about it here, but with IBM MQ V9.0.2 there are only blogs – both this one and our development blog from Ian Harwood you can find on developerWorks here. Also there is a YouTube video talking about the new update.

So, what has the development team has been working on in MQ V9.0.2? As with the 9.0.1 update there are several areas of enhancement and new function including:

  • Additional REST API coverage
  • Further updates to the MQ Console
  • Improvements in MQ MFT specifically in MFT Agent status reporting
  • Simplification in managed MQ logging on distributed platforms
  • MQ Appliance support for HA key renewal and 9.0.2 REST API verbs
  • Support for IBM Cloud Product Insights for registration and usage
  • Integration with Salesforce messaging events
  • Native Debian installer support for Ubuntu
  • Availability of MQ Advanced for Developers in the IBM Bluemix Container Service

 

Perhaps as with our description about building construction above, the delivery of any of these features might not be significant, although I think that the logging improvements will make a substantial difference to the many aspects of the use of MQ in the thousands of customers using it today.

 

What hopefully does become apparent is our ongoing support for the continuous delivery process. While some of these updates are brand new and have taken a lot of work, others are continuing to build on the work done in the MQ V9 and MQ V9.0.1 deliveries. These incremental deliveries of REST API support, and now the new Cloud Product Insight support will continue in future Continuous Delivery releases, making these features and the product more useful.

 

Let’s look at a handful of these new features starting with the logging support. Logging is very much the heart of IBM MQ and it is these recovery logs which allow MQ to recover from a failure, therefore providing the reliable and robust nature of IBM MQ. While circular logs are easier to manage, many customer use linear logs but these come with a lot of administrative overhead. The new feature allows for automatic management, recording and reuse of logs, lowering both the administrative overheads and improving the overall throughput in the system

 

IBM Cloud Product Insights is a new cloud hosted offering that many different IBM products will be able to work with. Additional features will be added to work with this over time, but initially there is support for registration and usage. You will be able to register your instances of IBM MQ and track them on the Product Insights dashboard. At this time you will be able to see what level of IBM MQ is install, where, and when it was last running. You will also see some usage information such as the number of persistent and non-persistent messages put, and the total size of data being moved through MQ. There is also a beta of log management, where MQ error logs will be shared with the Product Insights dashboard.

MQSalesforce

You may have seen the recent announcement of IBM and Salesforce working together more closely. We are very pleased that one of the ways this relationship is being demonstrated is through a bridge between Salesforce and MQ. When an event happens in Salesforce such as a change to data or a new application being run (Salesforce Platform Events or PushTopics), there is now the ability to trigger a MQ message to provide information about that event without the MQ application needing to be directly connected to Salesforce, simplifying your environment but making your systems more connected.

 

And finally, we now have a version of MQ Advanced for Developers available in the Bluemix Container Service. This means that the fastest way to create a development environment for IBM MQ might be with a couple of clicks to provision MQ Advanced for Developers. With pre-configured defaults to simplify administration, there has never been an easier way to get started with IBM MQ. What are you waiting for?