Posts Tagged ‘MQ Managed File Transfer’

All aboard the 9.1.x CD train. First stop is IBM MQ V9.1.1.

November 27, 2018

steam-train-north-shore-scenic-railroad-two-harbors-minnesota-17-9-00159

I am sure everyone knows the phrase about buses. If you miss one, don’t worry. There’ll be another one along in a minute. And while it could be said that applies to Continuous Delivery releases, I think it is more like getting on board a train. The destination is the next Long Term Support release, and you think you know what stops will be coming up. But maybe you don’t know exactly what you will find at each destination. You know there will be something new to discover at each stop. You could almost think of the train growing at each stop with the content of each new continuous delivery release, ready to be delivered finally to the Long Term Support destination.

 

Which brings us to the latest MQ CD release, MQ V9.1.1, announcing today, which is the first CD release in the 9.1.x set of releases. The experience we have of our 9.0.x CD releases is that we have seen a lot of interest from customers. Some have been able to move quickly to take up the CD stream into their environments and run them in production, at least for some of their queue managers. Others have been able to experiment with the new features in their test environments to see whether it is worth their while adopting the content early. And there seems to be a larger set of users who, while they haven’t been adopting the CD content into the production systems, the earlier availability and visibility of the new content has helped them move much more rapidly to adoption and use of the MQ V9.1 LTS release than we might have previously expected. I have personally talked with a lot of existing MQ customers who have either already started using MQ V9.1 LTS or are planning to move to use it very shortly.

Screenshot 2018-11-27 at 08.47.09

The MQ V9.1.1 release isn’t a destination in itself. It is the first part of our continuing journey. The MQ team works to accommodate a mix of strategic development priorities into releases to move the MQ offering forward, as well as other customer driven priorities, and reacting to and supporting other offerings and platforms as they change and adapt. Let’s find out how this mix has shaped the release. As well as suggesting you read the announcement content in the announcement letter, I will call out a few of the interesting new features.

 

One important new set of capabilities, driven by customer requests, is around the choice and negotiation of the use of TLS ciphers. Security of the MQ environment is hugely important in the current environment and is likely to remain a key area of focus. The importance of security and data protection is one reason customers are moving to MQ Advanced or MQ Appliance as a way to get the end to end encryption in MQ AMS. But this release focusing on enhancement to the security used in the TLS ciphers – used for encryption on the wire, not encryption at rest. As time passes, some ciphers become less secure and customers need to take prompt action in their environments to ensure the ciphers they use are updated to meet their own business requirements as well as the needs of the different systems.

In MQ v9.1.1 the choice of ciphers can be negotiated dynamically from a set or ‘whitelist’ available on each MQ channel. This reduces the potential for downtime and administrative overhead through faster movement to new ciphers when an old cipher is deprecated. Weaker ciphers can be removed from the list of allowable ciphers without needing to wait for a security fix update from IBM.

 

Another update driven by customer requests is the new support in MQ V9.1.1 for .NET Core for Windows. Customers who choose .NET as a framework for running applications on Windows environments have been looking to move to .NET Core. Following a number of requests, we have now added support for .NET Core for Windows environments to help support those customers.

 

As we have seen in the 9.0.x CD stream, one of the important set of capabilities that was added was the REST API for Admin for MQ. And at the end of that set of releases we started to look at adding REST API calls for the administration of MQ Managed File Transfer features, available with MQ Advanced and MQ Appliance. Many customers find it value to ingest and move data through MQ, even when the starting point or destination for this data is a file on the file system. To MQ, it is all just data moving in MQ messages. Therefore, from an administration point of view, it is important to offer similar features and controls for managing the movement of this data through MQ as is available for MQ exchanges of application data. In MQ V9.1.1 the MQ MFT feature gains REST API calls to list the resource monitors as an alternative to previous methods.

 

A further update is to provide support for pausing message delivery to Message Driven Beans running in WebSphere Liberty, in addition to the support previously made available for WebSphere Application Server.

 

The MQ V9.1.1 release offers a good foundation to start the journey through the various 9.1.x CD releases. There was a mix of updates driven by customer needs, wider platform and offering support as well as some functions to enhance longer term MQ strategic plans. We are now pulling out of this station and heading to the next one. Hitch up the V9.1.1 wagon to your V9.1 MQ train, hop on board and enjoy the ride.

glacier-express-furka-pass

Advertisements

Data in motion is data adding value. Using MQ for data transfer from files as well as applications

September 21, 2018

gold bar

Data. It’s sometimes described as the new gold. Certainly, it is valuable. But how exactly does it compare to gold?

The value of gold is that it can be made into gold objects such as jewelry which will have both the value of the metal and have additional value because it has been put to higher value use. But there are also costs associated with owning gold, keeping it safe, and moving it safely and reliably from where it is to where it needs to be. In order to realize the value, the gold jewelry must be moved from where it has been created to where it is needed.

 

What about data? Like a delicate piece of gold jewelry, some people might value a piece of data highly, but to others it is of little or no value. While gold is essentially fungible – as any gold can be fashioned into something, each piece of data is unique, not just in itself but set in its own context. Think of when you are trying to complete on a purchase of a house. An agreed mortgage is all very well, but what’s important is that the transfer of funds happens at a specific time to a specific account to allow the purchase to go through.

 

While a piece of data might represent something, the value of the data is only really achieved when it is moved to the right place at the right time. Having the data held securely in a system is not valuable on its own. It needs to be moved to be valuable. Data is created somewhere in the infrastructure but needs to be consumed somewhere else to add value.

 

This gives us a number of problems with data. Storing it safely; moving it safely; knowing it has arrived, so it can be put to use. This is again similar to gold. You have to keep it safe, and certainly you have to look after it as it moves as well. And if you have made a piece of jewelry for someone, you have to let them know you have it ready for them.

 

In your business you will have huge quantities of data. Data that is being created every second and stored in your file system. Buried treasure. I have already discussed this in here before. So what are you going to do?  You need to move it safely and securely to the part of the business that can make use of it. Probably the faster you move it, the better. And you certainly don’t want to lose it. Or have it stolen as it moves. And wouldn’t it be helpful if as it was delivered to the destination, the target application could immediately be made aware of it.

 

All this of course is handled for you by the Managed File Transfer component of MQ Advanced or MQ Appliance. The name is slightly misleading as it doesn’t move files but instead the contents of files can be moved as MQ messages. This means they take advantage of MQ’s unmatched secure and reliable delivery. No lost messages. No lost data. Your data gold will reach where it needs to go. And it can even be delivered directly to the target application without being written to the file system again.

 

Just as you wouldn’t want to keep all your gold in a vault where it wouldn’t add value to your business, don’t keep your data held up not adding value. Put it to work by moving it through MQ. When you have MQ Advanced or MQ Appliance you can deploy unlimited numbers of MFT Agents inside or outside your business. You can even embed them inside applications you share with your partners or suppliers. And the latest updates in MQ V9.1 add further enhancements to the MFT functions.

treasure

Think of all the buried treasure that has been lost over the years. Don’t let your data join those wasted resources. Get started today by learning MQ or with downloading a MQ trial. Or see what you can do with a hosted MQ on IBM Cloud or now on Amazon Web Services.

 

Not all data is treasure of course. You have to understand what’s valuable. But if it is valuable, then you should ask yourself why you aren’t moving it reliably through MQ Advanced, taking advantage of end-to-end encryption. After all you don’t want to go on a pointless quest with nothing at the end of it. Or find that your treasure has been taken.

journey_cost

Everyone gets the point. MQ V9.1 delivers the latest features in a Long Term Support release.

July 3, 2018

Screen Shot 2018-07-03 at 09.42.14They say anticipation is half the fun. And one of the good things about the split release approach for MQ with Continuous Delivery releases and Long Term Support Releases is that as new function is developed and made available in the CD stream, customers intending to use the LTS release can build their anticipation for the new function for up to 2 years.

 

Of course, there is nothing to stop early experimentation with the CD releases even though you may be waiting for the LTS availability. But the good news is that the wait is now over and IBM has published the announcement letter for MQ V9.1 and MQ Advanced on distributed platforms here. Also MQ V9.1 is being announced for the MQ Appliance, as well as a new model of the MQ Appliance – the M2002. You can read that announcement letter here, and a blog about it here. Also we have announced MQ V9.1 for z/OS – there is an announcement letter here for the MLC offering, and another announcement letter for MQ Advanced for z/OS VUE and other z/OS OTC offerings here.

 

Has it been worth the wait? What has been the most anticipated new capability? It’s not like a Christmas present where you are not sure what’s under the tree. Almost every feature, function and enhancement in MQ V9.1 has been already available in one of the CD releases, so there shouldn’t be much of a surprise. You can read some of my past blog entries covering the prior V9.0.x releases (V9.0.1, V9.0.2, V9.0.3, V9.0.4, and V9.0.5)

 

And don’t forget than the previous LTS release – MQ V9.0 included important updates that have proved very useful such as the enhancements to MQ AMS providing end to end encryption, including encryption at rest without performance impacts, which can be very helpful in addressing GDPR requirements.

 

However, let’s cover here some of the most interesting areas of focus over the last couple of years of function, and which ones seem to have attracted the most customer interest.

 

There are many different areas of enhancement, which hopefully means pretty much all users have something to interest them.

Screen Shot 2018-07-03 at 09.52.28

Simple and more powerful Administration

  • MQ Console – a customized browser based for configuration and operations
  • REST API for admin – an extensive set of APIs enabling new tools to be written using REST HTTP calls, usable across older releases as well
  • Improved awareness of MQ activities and logging – Publishing MQ statistics to Prometheus and Grafana; forwarding MQ error logs to ElasticSearch or Splunk; Error logs output JSON for improved parsing
  • Automation of Linear Logging – simplifying the operations and administration of logging and management of those logs.

 

Supporting Developers

  • REST API for messaging – Enabling developers writing simple applications and micro-services to access MQ capabilities.
  • Additional API and protocol support – as well as publishing a new online tutorial for using MQ
  • Connecting to Salesforce – the MQ bridge to Salesforce allows for the 2 way publishing of information between SalesForce and MQ
  • The MQ Bridge to blockchain – only available for MQ Advanced or MQ Appliance customers.

RDQM1

High Availability and Disaster Recovery without complexity and cost

  • Replicated Data Queue Managers for HA – synchronous replication across 3 nodes using local disks instead of network attached storage.
  • Replicated Data Queue Managers for DR – manual failover with synchronous or asynchronous replication across 2 nodes.
  • RDQM requires MQ Advanced licenses. But with specific licenses to reduce cost.

 

Managed File Transfers

  • Licensing, packaging and pricing changes. MFT Agents are now free with MQ Advanced or MQ Appliance, and both embeddable and redistributable.
  • FTP Protocol Bridge enhancements
  • Improved reliability and monitoring for Transfers

 

 

z/OS enhancements

Many of the updates described above also apply to MQ on z/OS. There are also some additional enhancements specific to z/OS

  • AMS Confidentiality Performance. MQ Advanced for z/OS VUE sees enhancements in performance of this feature in MQ V9.1
  • Extended deployment for MFT – with MQ Advanced for z/OS VUE.
  • The MQ Bridge for blockchain now using the Hyperledger Composer API to build out the connectivity.
  • Connecting CICS and MQ – Java programs running on a CICS Liberty JVM server can now use MQ classes for JMS to access MQ capabilities.

 

AS MQ now moves to MQ 9.1, this time the point is available for everyone. All the features above, and more I haven’t had a chance to describe will be available later in July 2018. Whether deploying in on-premises environments, on physical Appliances, on VMs, in containers, on private clouds like IBM Cloud Private, or public clouds like IBM Cloud, AWS, or Azure, the Long Term Support release now means the 2 years of functional enhancements, tested already in multiple Continuous Delivery releases are now available for more to use.

UPDATE: MQ V9.1 now available as of July 23rd 2018. Read more here.

And there is plenty more to come. Watch this space both for more updates and use cases of these features, and well as future updates in the next Continuous Delivery releases.

Putting out a new release like IBM MQ V9.0.5 is more than a 9-5 job

March 16, 2018

9-5clocks

At least in the UK, the traditional hours worked in a day job were 9 to 5. You would ‘clock-in’ at 9am and leave at 5pm. I guess it is common as there was a 1980s film called “9 to 5” starring Dolly Parton. These days office life is rather more flexible, and certainly the idea of clocking in and out at fixed times is gone.

 

For 25 years, virtually every major IT infrastructure has been able to rely on the secure and reliable exchange of data between applications and systems thanks to IBM MQ. Previously called MQSeries, then WebSphere MQ, this software offering, developed in the IBM Hursley Lab in the UK has been a critical part of the business world. So much so that most people living their lives have no idea they use IBM MQ so much on a daily basis as it ‘just works’.

 

There is a great team of developers who work hard day-in and day-out to enhance and update IBM MQ, and . We have now released IBM MQ V9.0.5, going GA on Friday March 16th. And our developers have worked for months, giving up evenings and weekends to not just add new features, but to make sure it is another offering that works when put into use. So not 9-5 at all.

 

Now for some customers this will be more of a prelude to the main act. This is referring to V9.0.5 being a Continuous Delivery release. When we brought out V9.0 we split it into 2 streams: Continuous Delivery and Long Term Support. This release marks the final release in the initial set of Continuous Delivery releases. The next release will be the first of a new Long Term Support release. And customers can expect that the functions delivered in the 5 CD releases will be made available in the new Long Term Support release.

 

When that new LTS release is available, you can expect me to summarize all the new features, but for now in this blog I will call out a few of the new features in V9.0.5.

 

The new Easy HA feature (Replicated Data Queue Managers) delivered in MQ Advanced V9.0.4 gets updated to add support for a Disaster Recovery mode, with manual takeover after either synchronous, or asynchronous replication between a pair of MQ servers.

 

The MQ Managed File Transfer capability, available with MQ Advanced or MQ Appliance gets the first support for the REST API admin interface for listing current transfers and querying MFT Agent status.

 

MQ Advanced itself will do more to identify itself when it is installed, and so prevent compliance issues, and ensures that components can recognize Queue Managers.

 

Other updates include a MQ Console refresh, and for customers who use MQ with WebSphere Application Server, performance enhancement through implicit syncpointing.

 

For MQ Appliance users there is an enhancement for better reliability by allowing aggregated IP interfaces for the Floating IP feature. This removes a potential single point of failure.

 

And for users of MQ Advanced for z/OS Value Unit Edition there have been improvements including enhancements to MQ AMS which will see increased performance.

MQ clouds puttenham

Perhaps even more exciting is the new availability of a hosted instance of MQ on the Cloud. More about this can be found here, but it creates a great opportunity to quickly and easily make use of MQ without needing to install, deploy or manage the environment. Just configure and go! Nice that after 5 years of talking about it on this blog we have an explicit offering running in the cloud. This is of course alongside MQ already being able to run in AWS as a QuickStart. Or deployed as containers in IBM Cloud private.

 

As well as looking forward in the future to a new Long Term Support release, the statement of direction indicated that the Blockchain bridge, available in MQ Advanced, will be updated to be based on the Hyperledger Composer interfaces. And additionally, customers deploying MQ in containers will in the future be able to track the size of the container, and the duration of use, and license based on that container size, rather than the full capacity of the system where the container is running. The intent will be to support existing pricing metrics such as PVUs and VPC monthly metrics, but also a future VPC Hourly metric.

ibmthink

IBM MQ, along with many other IBM and business partner solutions will be some of the highlights discussed at IBM Think in Las Vegas running March 19th-22nd. I will be there and I hope to see some of you there as well. Famously Las Vegas never sleeps, so I guess that’s something else that’s not 9 to 5. Lucky we have IBM MQ V9.0.5 now though.

9to5dolly

 

What is GDPR and how does it affect IBM MQ use?

September 26, 2017

Imagine as a business that you were given the opportunity to grow turnover by 4%, at a stroke, or to increase revenue by €20 million. This would be certainly a key focus area. Well, there is good news and bad news, because these figures are accurate, and can apply to your business, but they are fines that will be applied to your business if you are in breach of the GDPR regulations that will be enforced in May 2018. And your business will be liable for whichever is the greater amount. As such, it is a subject that demands attention. And it can apply to any business if it concerns the personal data of EU citizens, even if your business is not based in the EU.

Screen Shot 2017-09-26 at 11.42.31

GDPR is a complex piece of legislation, and not one that can be solved through any one single act or solution. It requires understanding of the legislation, and then a thorough review of existing governance and processes, especially those involved in data handling and security, and ensuring that all people involved in all these aspects are aware of changes being made, and why these are important and must be complied with.

Amongst the key criteria for GDPR compliance are a number of aspects that are likely to need to be reflected in the choices made in MQ deployment to help to meet the compliance needs. However, it must be understood that taking this action around MQ alone will not achieve GDPR compliance, but simply be a part of that compliance.

Given that GDPR is concerned with data protection, it should be clear that data privacy is key in reaching compliance. This isn’t the only aspect, as there are multiple additional aspects such as the ‘right to be forgotten’ providing a requirement to remove data, and also the need to track the movement of data through all systems. Considering all these aspects together, it should be clear that reducing the movement of data to modes of transport that allow for end to end encryption, as well as logging, reporting and monitoring for the movement of data are likely to be seen as essential to aid in GDPR compliance.

Steps that you can take to help demonstrate your MQ environment is helping your business comply with GDPR regulations:

  • As well as using authentication and authorization to secure your MQ system, end-to-end encryption is available as part of MQ Advanced and MQ Appliance to supplement this
    • Using end-to-end encryption could be the only way to protect personal data wherever in the organization it moves, as it moves as it reduces the need to ensure control of all intermediate systems to protect the data.
    • End-to-end encryption can help to demonstrate privacy by design as part of your compliance verification process.
  • IBM MQ can be configured to log all messages and accesses which can be used to track all movement of data, and who had access to it.
    • Making use of the tools available with IBM MQ to monitor and report on message movement can be an essential part of good data governance
    • Building new tools using the REST API for MQ admin to offer a custom view of MQ configuration and operation could be a critical aspect of generating reports on data movement and protection.
  • Holding personal data in files is widespread in many businesses
    • Holding and moving those files around your business could add further vulnerabilities.
    • File data needs to be handled and managed with the same care as application data as it is just as likely to be personal data that needs to be protected.
    • As part of the MQ Advanced and MQ Appliance entitlement, businesses can move file data securely, and with monitoring and tracking, through the MQ network, helping to meet GDPR compliance without additional complexity

 

As mentioned at the start, there is no single solution that can address all the aspects of GDPR within your business. Ensuring your MQ environment is configured to securely move data with end to end encryption, with comprehensive logging and reporting of messaging access and movement can be a critical step in the wider compliance task.

 

Work with your IBM representative to ensure you hear more about the benefits of MQ Advanced entitlement to allow your business to move file data and to encrypt messages and data end to end, and thus reduce the risk of data being exposed in a security breach. Or review whether the MQ Appliance would be a good fit for your business, providing the same benefits in end to end encryption and file data movement.

 

Read more about MQ Advanced here: https://www-03.ibm.com/software/products/en/ibm-mq#othertab3

Read more about the MQ Appliance here: https://www-03.ibm.com/software/products/en/ibm-mq#othertab4

For additional information about how IBM can help you with GDPR see here: https://www.ibm.com/analytics/us/en/technology/general-data-protection-regulation/

GDPR robot

Space is big. Really Big. If it was data it could be transferred with IBM Aspera.

July 4, 2017

The genius that was Douglas Adams wrote: “Space is big. Really big. You just won’t believe how vastly hugely mindbogglingly big it is. I mean you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space.”

The same could be said for some of the data files being created in businesses today. These could be anything from medical images to video files, collections of ERP data or data replication. I remember just a few years ago, a file that was more than a few MB was considered large. But these days it can be quite common to think nothing of files being hundreds of MB or even multiple GB.

To a great extent, with fast internal networks and lots of storage, file size is almost irrelevant these days. Except for one key area: the movement of large files, or large amounts of data over long distances. Even if you think you have a large capacity high speed network, once you start to move large amounts of data over a significant distance, you hits problems that are a combination of physics and network transfer protocols. The distance, and the need to send responses as part of the protocol means that the larger the amount of data being sent and the longer the distance, the transfer time starts to get exponentially longer.

person-pushing-rock-up-hill

Moving a large file is no problem on a high speed LAN, or even across a fast WAN link over a short distance, once you start moving across potentially thousands of miles, your transfer times become unworkable. This is the problem that IBM Aspera offerings solve by using the FASP protocol to move data at high predictable speed even over long distances.

While we are all now used to our business transactions being essentially instantaneous, this hasn’t been the case in moving large data files for business use as these might have had to be scheduled out of process time due to the time taken instead of being moved as part of a business process as would happen with transactional data.

world network links.jpg

A typical business might be using MQ to move transactional data between applications and systems. They might also be using MQ Managed File Transfer to move file contents. But with MQ’s message size limit of 100MB per message, these wouldn’t be large enough to see a problem even when moving large distances. But supposing MQ is also looking to trigger the movement of much larger files, say between the office in New York and the office in Singapore. A design file of 10GB would be impractical to send over FTP as it would take too long, but using Aspera, then depending on the network link speed it could be sent in minutes, if not seconds. This transfer then becomes possible to be just another standard part of the business process, and can be viewed in this way as an extension to IBM MQ and IBM MQ MFT.

Aspera speeds

To that point IBM recently announced a new set of parts to make it easier for customers using IBM MQ or other integration products to start using Aspera offerings to meet this need. See the announcement letter for IBM Aspera High Speed Transfer for MQ here and start to accelerate your data transfer needs.

MQ Aspera

Buried Treasure – embedding IBM MQ clients and MFT Agents into applications

June 13, 2017

treasure

I haven’t been doing this blog so long that I am going to repeat myself. Or at least not yet. But last year I did a blog on why you would use MQ – and that is broadly the topic of this entry as well but it comes from a specific use-case perspective. Plus – warning – it is longer than usual – sorry. Why do businesses, in their thousands, use IBM MQ – and its many different yet critical functions? Sadly, and I say this as the Offering/Product Manager for MQ, no one wakes up in the morning and decides they want to buy more IBM MQ – but they do so because of the benefit using MQ provides for the applications that run their business.

 

IBM MQ enables the exchange of data between applications, systems, services and files with reliability and security. It does this with scalability and simplicity. It has proved itself in doing this over the last 20+ years that much of the modern online business world takes IBM MQ, and its capabilities for granted.

 

The IT infrastructure is evolving rapidly – as it always is. As such there is both growth in new applications and existing applications are being updated and enhanced. Today’s applications typically have to be more resilient than ever, but also more portable – to be deployed pretty much anywhere. In most businesses applications will be extended out to business partners as the wider ecosystem is more tightly integrated than ever before.

 

These changes drive a greater need for seamless connectivity throughout the infrastructure and it makes it more important that all business data can be simply and quickly moved inside and outside the business. So how has IBM been working on IBM MQ to enable this? And will IBM MQ be able to help all customers – whether they are trying to connect and exchange data between applications, systems, services and files – not just the latest and greatest APIs?

 

IBM MQ allows for connectivity and exchange of data through MQ Clients and MQ MFT Agents and to make it easier for these to be used in many different use cases, IBM has been making changes to the packaging and licensing of these.

MFT Agents

One of the key changes was at the end of 2015, there was an update to the license documentation to allow for the redistribution of MQ Clients. IBM makes the MQ client libraries available for free download. These are then built into the MQ enabled applications to allow these applications to send and receive MQ messages. There is no cost for the MQ Clients – as they require a licensed MQ Queue Managers in order to function. However, until late 2015, the license prevent redistribution of these MQ Client files. This meant that if a business built the MQ Clients into an application, it wasn’t permitted to then distribute this application outside the business – i.e. it couldn’t share it with a business partner to allow that partner to work closely as an integrated partner. To allow this under the terms, the partner would need to either install the MQ Client library themselves or agree licensing terms to redistribute the MQ Client with IBM. This restriction was not helpful to these businesses or to the IBM MQ business and therefore it was changed to allow redistribution.

 

Now let’s look at a scenario – Company A uses MQ to exchange information throughout its business. It has suppliers (Company B and Company C) and it wants to streamline the manufacturing processes to enable them to get production statistics and thus help to plan for more efficient resupplies to their factories and warehouses. To do this it wants to provide them with a copy of their own in-house written application that uses MQ. Now that IBM allows for redistribution of the MQ Clients, Company A can simply provide their application to the partner companies to enable them to communicate seamlessly with no need to even be aware of the MQ Client embedded within the application. MQ messages can flow securely between the companies – and as only Company A has a MQ Queue Manager, they are the only ones licensed for MQ – and there is no additional MQ cost for this configuration. Note that companies exchanging MQ messages like this might want to make use of the MQ Internet Pass-thru feature to simplify passing messaging through their firewalls.

 

Now let’s imagine Company D. They are also part of the supply chain ecosystem for company A, and also many other businesses. But the stock control and distribution management systems are built mainly on files and file data. They keep these files updated with stock quantities and prices, but they find it simpler to keep using this method rather than online application updates and exchanges. They are used to sending these files to their customers using FTP but they always have a number of issues around FTP failures, reliability issues, and having to spend time diagnosing the problems inherent in these transfers.

 

Company A have a solution – the Managed File Transfer capability that is a part of IBM MQ Advanced. In place of regular FTP, the data inside the files can be sent as MQ messages from Company D to Company A, taking advantage of MQ’s reliability, security and management of data. And best of all Company D don’t need to change the way they handle data as they can still focus on keeping the file contents updated, but Company A can provide a program that can also embed the MQ MFT Agent which can run and extract the contents of the file and send it as MQ Messages to Company A. Just as with the MQ Client, the MQ MFT Agent is designed for easy embedding in an application, and benefits from also being redistributable under the license. The key difference is that MQ MFT Agents are free but only when they connect to MQ Queue Managers that benefit from the MQ Advanced license entitlement or are in the MQ Appliance. In providing this application making use of the MFT Agent to Company D, Company A is taking advantage of the recent change to make the Agent license redistributable, as well as the fact there is now no cost to embed MFT Agents and distribute them anywhere, as long as they connect to their MQ Advanced Queue Managers. Also, the packaging changed to ensure the MFT Agent was available as a standalone zip file for easier embedding.

 

As a business, your buried treasure may be hidden in your data. You owe it to yourself to ensure it is used as widely as possible and as timely as possible. But to do this you need buried treasure in your applications as well – and this time the buried treasure is the MQ Clients and MQ MFT Agents you can now embed in those applications. Hidden in your code, but providing value every day – maybe not buried treasure, but the goose that lays golden eggs?

Goose Golden Egg

Not too much of a good thing: MQ V9.0.3

June 6, 2017

After a gap of a few months I blogged earlier today about deploying MQ Queue Managers in a DMZ so it might seem a bit much to be blogging again so soon. However I will try to keep it short and snappy so you find these entries like a Japanese meal – small portions, but so many courses! And of course, delicious.

japanese meal

So it wasn’t long ago – just March – when I blogged about MQ V9.0.2 on MQ and MQ Advanced on distributed platforms and MQ V9.0.2 on the MQ Appliance. Remember that IBM is delivering MQ V9 as a continuous delivery release. This means that we deliver smallish amounts of hopefully easily consumable and usable function. And these functions, on the whole, will build incrementally to deliver eventually a substantial piece of new function.

 

One of these ongoing deliverables, that has been building over the last few releases is the growing REST API for administration of MQ. New capabilities in this release include read and update of the queue manager configuration, plus querying of the status.

 

Also, on top of the enhancements made to MQ Managed File Transfer, available with MQ Advanced or MQ Appliance, delivered in MQ V9.0.0, V9.0.1 and V9.0.2, there are even more usability enhancements in this release, focusing on problem determination when there may have been an issue in the completion of a file transfer. This is in addition to the license changes made recently that makes this far more attractive for deploying MQ MFT Agents widely through the business.

 

And for the MQ Appliance there was an update to allow an easier transition for some configurations to move to use the end to end encryption provided by MQ AMS when some MQ Clients may not support it, by doing the encryption on the MQ Appliance rather than the MQ Client side.

 

There are now announcement letters for MQ V9.0.3 and MQ Appliance V9.0.3 updates published but perhaps some of the most interesting updates of the MQ V9.0.3 releases was on the z/OS offering. There is already an announcement  letter about this – but this update specifically targeted the MQ Advanced for z/OS Value Unit Edition offering with a set of unique extensions for this delivered as a connector pack on top of the core MQ Advanced for z/OS VUE offering.

This connector pack included a Bridge to Blockchain, allowing MQ Advanced for z/OS VUE to query information on the Blockchain. Also there are changes to the licensing and deployment model of MQ Managed File Transfer components on z/OS. And support for MQ Advanced for z/OS VUE to publish information to the IBM Cloud Product Insights service.

 

There are some additional details on our development blog on MQ V9.0.3 here.

 

So that was a quick run through of the updates in IBM MQ V9.0.3. All you need now is some green tea to wash it down.

japan green tea

Power is nothing without control – IBM MQ V9.0.1

November 15, 2016

power-control-image

As mentioned on this blog before, much of the modern world runs on, and depends on IBM MQ. Whether sending credit card details from a retail store, tracking a shipping delivery, coordinating stock levels at a factory or processing an insurance claim, IBM MQ is at the heart of thousands of leading businesses, keeping their business running, always available and secure.

So, IBM MQ runs at the heart of your business, helping to keep your applications simple, yet connecting them with reliability and security, highly scalable and highly available, whether processing 1 transaction per day, or 10 billion messages per day. But however it runs in your businesses, it doesn’t run itself. Half of the value of IBM MQ is not just the assured once and once only delivery but that your business has the visibility that the message was delivery successfully or not. And to help your business understand that, IBM MQ can provide lots of information about MQ itself and its activity.

For years IBM MQ has offered a number of different ways in which to configure, manage and control itself. There has been MQ Explorer, MQSC, PCF.

With the release of MQ V9.0.1 – which is the first ‘Continuous Delivery’ release to build on top of MQ V9, IBM has brought new ways for the customer to control and manage their IBM MQ infrastructure.

The first new tool is the MQ Console – this is a browser based tool to configure and manage your MQ infrastructure. You may be aware we have had the MQ Console available on the MQ Appliance since its release in 2015, but this is the first time you can use the MQ Console with MQ software installs as well.

mqconsole1

The MQ Console adds some key benefits for customers. One is that you can use a browser and so you don’t need to install anything – unlike the MQ Explorer. Another is that you can customize the MQ Console to show you just what you are interested in – with widgets to show you activity on a queue, or what queues are defined. IBM will continue to update it, but it shouldn’t be seen as a replacement for the existing tools such as MQ Explorer, but rather an additional option.

The MQ Console is built on top of a REST API – and another feature in MQ V9.0.1 is to begin to expose an external REST API. While initially in this release there are only a couple of verbs published, the goal is to continue to add to these in future releases such as 9.0.2, 9.0.3 etc. Many customers will be looking to take the REST API and build new custom tooling to meet their specific needs. It is possible that various 3rd parties such as the numerous skilled IBM MQ business partners may use this new API to build their own offerings, or even custom offerings for individual customers.

Additionally, there have been some updates to the IBM MQ Managed File Transfer function, available in IBM MQ Advanced. As well as limiting the number of retries when there are failures in using the FTP Protocol Bridge, there is a packaging and install change for the MQ MFT Agent. Previously this was only available as a part of the overall IBM MQ install package – and it required a full install procedure. In the IBM MQ V9.0.1 release, the MQ MFT Agent is now available separately as a zip file. This means that it is now much easier to embed this capability as part of a customer created solution which can then be distributed without the need for the full MQ package or a dedicated install just for the MQ MFT Agent component.

As you can see all of these new features, in the first of these Continuous Delivery releases for IBM MQ V9 provide new value that is specifically aimed at making it easier to take advantage of the IBM MQ capabilities – and take back control of your systems.

Don’t forget you can get a free 90 trial of MQ and see for yourself. And you should also be able to download the MQ Advanced for Developers offering for free development and test.

UPDATE: An excellent blog on the MQDev developerWorks expanding on this topic

Maybe you can now see why it reminded me of the famous series of adverts from Pirelli from maybe a decade ago – Power is nothing without control. IBM MQ gives your business the power of reliable and secure movement of data. Now we are making it easier for you to take control.

mqconsole2

[Having used Ed Moses in a previous blog, it is nice to continue the theme of great athletes with Carl Lewis in this one]

IBM MQ V9 – A fast, secure, reliable and more agile MQ

April 19, 2016

edwin-moses-getty_2129850b

Some of you reading this blog may recall the great athlete Ed Moses – who had a record 122 race winning streak in just about the hardest event – the 400M Hurdles. You need to be strong, fast, and agile just to compete, and to keep winning you need to be reliable. Well, this is how we view IBM MQ, especially with the latest release – IBM MQ V9. You may have seen a recent blogpost on here that had a Statement of Direction talking about a new way of delivering IBM MQ – one that provided a Long Term Support release, and a Continuous Delivery release. The aim of this model is to give customers more choice to select either highly stable releases with just fixes, or releases that benefitted from additional function in the fixpacks.

TRY IT: Click here to get a free trial of MQ

UPDATE: There is a FAQ on the new support model. Read it here.

On April 19th, IBM announced MQ V9 which is the first release that moves to this new more agile delivery model. As such at the initial release it delivers a small set of additional capabilities that will be available to all customers. Then subsequent mod-level updates will deliver even more updates to customers choosing the continuous delivery stream, but all customers moving to V9 will get the benefit of the new capabilities being delivered in this release.

As with previous releases of IBM MQ, customers have a lot of choice in where and how they may want to deploy this version. IBM supports deployment of MQ – and MQ Advanced pretty much on every commercial IT environment where business critical applications may be exchanging data reliably, securely, and at scale. This could be on-premise, deployed in cloud environments like IBM Softlayer, Microsoft Azure or Amazon AWS. IBM also supports virtualization with many customers deploying in VM images, and also in Docker containers, which can be deployed anywhere, including in IBM’s Bluemix platform. This flexibility enables customers to make use of enterprise messaging to support deployments on-premise, on cloud or in hybrid environments.

So what are the key new features of MQ V9 being delivered in this release? Well there are a number of them that are called out in the announcement letters – so you can read the MQ V9 distributed announcement letter here. And the MQ V9 z/OS MLC announcement letter here. And you can read the MQ V9 One Time Charge announcement letter here. But below I will call out a few of the features that I think will be most important to customers.

One of the features likely to be most interesting is a change to the MQ Client Channel Definition Table (CCDT), which is needed by the MQ Client application to provide the channel definitions needed to connect to the MQ Queue Manager. This file is created automatically and prior to MQ V9 needed to be distributed to the client application prior to use. The big change from this new release is that the CCDT can be a web addressable file instead of needing to be distributed out to every client, and to then need to do that with every change. By having a web addressable CCDT accessed by URI, then there are much lower administration needs, and also the MQ infrastructure can be much more dynamic as changes can be made centrally and take effect quickly and without application disruption.

 

The second big change to the new release of MQ is in MQ Advanced Message Security (MQ AMS). This feature, which is a priced extension to MQ (available either separately or as a part of MQ Advanced) provides policy based encryption at rest of the MQ message contents. By using this capability, businesses can be assured that their message contents can only be unencrypted and read by the targeted application destination, and there is no risk of exposure should any security breach take place which provides access to the system or storage where the MQ Queue Manager holds its queues. This privacy and integrity has been assured by the generation of asymmetric keys for every exchange between client and queue manager, which provides an extremely high level of security, but can introduce a high overhead in terms of the processor cost of the asymmetric key generation.

MQ AMS performance

With MQ V9, a new mode of operation is added to MQ AMS, called ‘Confidentiality’. In this mode there is an initial asymmetric key exchange then subsequent exchanges can reuse (to an extent that can be configured) a symmetric key. This still provides a high level of security and protection for the message content, but with a dramatically lower level of overhead in terms of encryption workload cost. IBM expects that due to the increasing importance of security and protecting systems and data from breaches, that this new feature of MQ AMS will help more customers protect their message contents and therefore their business and customer data. IBM expects to produce performance data for the new AMS configuration around the time that MQ V9 is generally available. But the early testing shows considerable improvement.

 

A further change for MQ AMS is the support of non-IBM JREs for use with MQ AMS. Previously applications written in Java that relied on a non-IBM JRE wouldn’t work with MQ AMS. In MQ V9 this has now changed so that suitable non-IBM JREs can be used, as well as IBM JREs, extending the ability of more customers to use MQ AMS.

 

There are a number of other new functions and capabilities available in MQ V9, such as updates to MQ Managed File Transfer capabilities – which are described in the announcement letter, and with the movement to a Continuous Delivery model customers should expect to see more capabilities being delivered in mod levels on top of MQ V9 in the future.

 

With the recent announcement of the End of Support for MQ V7.1 – announced here – along with the related end of support of the older separate versions of MQ FTE and MQ AMS, this latest release of MQ V9, along with the recent announcement of the update to the MQ Appliance provides customers with a strong set of choices of how to take advantage of the latest new releases as they plan to move off the older releases of MQ they may be using, keeping their deployment of MQ up to date and supported.

When you are taking advantage of the benefits of IBM MQ, you may not need to have to work as hard as Ed Moses did to be #1.

UPDATE: Mark Taylor has provided one of his highly useful videos detailing more of the new function in MQ V9. Watch it here.