MQ Password/CHLAUTH research – Exec Summary

As of v8.0, MQ now can natively validate user IDs by checking the password against the Operating System or LDAP.  Checking against Pluggable Authentication Module (PAM) was added in v8.0.0.4.  Prior to v8.0 it was necessary to use a channel security exit to perform password-based authentication over SVRCONN channels.  With MQ v8.0 and later, password-based validation is natively supported and integrated with CHLAUTH rules.

This has been a widely anticipated feature so it came as no surprise that implementing it was among the requirements on each of my several most recent consulting engagements.   What was surprising however is that over time I noticed that techniques I’d used at one client for combining CHLAUTH with password based authentication didn’t seem to work at the next.  The first time I noticed this I wrote it off as having taken poor notes.  The second time though led me to undertake a comprehensive analysis on a per-version and per-fix-pack basis.

This post and accompanying materials are an executive overview of the findings and recommendations.  More detailed findings will be posted shortly.  My priority in this initial publication is to introduce the issues and outline the recommendations for safely using the new features.

Problem statement

The behavior of MQ password-based validation combined with CHLAUTH mapping rules has changed several times across the initial release of MQ v8.0 and subsequent Fix Packs.  Core functionality is affected, sometimes in ways that break applications, and other times in ways that silently over-authorize.  The best case is actually the one in which an outage occurs because at least there is some overt indication that the behavior has changed.  Silent over-authorization is worse due to an expectation of security that does not actually exist.

Secondary effects of these changes follow on in the form of increased administrative and testing overhead after each patch release, or else relaxation or abandonment of patch management.

Recommendations

These recommendations all apply when password-based authentication is enabled.  Unfortunately the proposed settings significantly restrict CHLAUTH functionality given the current MQ implementation, but there is no middle ground between this and what by most standards would be an unacceptable lack of security.

  1. If you are currently using an exit to perform password-based authentication, you might strongly consider sticking with it.

Assuming native MQ password authentication is going to be used, I consider the following to be mandatory:

  1. Enable the queue manager’s password protection policy and/or set up TLS on the client channels.  This protects credential exchange on the network up to a point.  It is still possible to send the ID and password in the clear but the queue manager will reject the connection.
  2. Use MQCSP authentication mode instead of Compatibility Mode. This is set at the client side.
  3. CHLAUTH must always be enabled.
  4. ADOPTCTX must always be set to YES.
  5. Use only USERMAP rules with USERSRC(CHANNEL). If address mapping is required, use the address function in the USERMAP rule in addition to specifying the CLNTUSER field.
  6. Under MQ v8.0.0.5 and MQ v9.0, enable ChlauthEearlyAdopt in all cases.
  7. Be prepared to spend more time testing Fix Packs, especially including negative cases that test for silent over-authorization failures.

As a strong proponent of timely patch management, the last recommendation pains me greatly to have to write but I don’t see an alternative.  Standard 30/60/90 day targets may be impossible to meet given the volatility observed and the substantial nature of the changes to key behavior of these security controls across maintenance releases.

  1. If you are an auditor or QSA, consider that the changes over maintenance levels seen so far potentially require code changes and/or modifications to the security model significant enough to preclude timely patch management. I’m not suggesting abandoning the requirement altogether, just that some Fix Packs (v8.0.0.1 was one example) may need to be skipped altogether and some (v8.0.0.5 for example) may impose an exceptional resource impact requiring extended lead and implementation times.

Platforms and versions known to be affected

I tested Red Hat and SUSE Linux and Windows 10 against MQ Advanced for Developers.  For Linux I have tested every version and Fix Pack between MQ v8.0 and MQ v9.0, inclusive.  I am still testing Windows but have completed tests at MQ v9.0.

The tests in all cases used OS-based ID and password checking over SVRCONN channels with CHKCLNT(REQUIRED) set, using the installed MQ Explorer as the client.  Variations included CHLAUTH enabled and disabled, channel Compatibility Mode on and off, and ADOPTCTX at both possible settings.

Behavior affected

MQ has always had granular authorization provided by the Object Authority Manager.  The problem introduced with client connections was that there was no authentication mechanism.  Designing complex granular security models was of limited use when the client could assert the mqm User ID and the queue manager blindly accepted the assertion.  Later CHLAUTH rules allowed the MQ administrator to map a source address, a certificate Distinguished Name, or an asserted User ID to some locally meaningful value.  The certificate mapping was the closest any of these came to authentication.

The introduction of native password-based authentication to MQ finally changed all that.  It is now possible to send an ID and password to MQ and have these authenticated against the local Operating System, against the Pluggable Authentication Module (PAM) in *nix platforms, or against an LDAP repository.  The only problem is that by default there is no connection between the authentication and the authorization.  The ID that was validated using the client-supplied password is not necessarily the ID the channel runs under and the configuration that decides that run-time behavior is at the client side of the connection.

The ADOPTCTX(YES) setting causes MQ to use the ID that goes with the password as the one the channel runs under.  It binds the authentication to the authorization and gives most of the control back to the MQ admin.  Unfortunately, it also removes all mapping functionality of the CHLAUTH rules.  It is no longer possible to delegate authority by mapping an address, User ID, or certificate Distinguished Name to a target User ID.

Finally, the ID evaluated by the queue manager to match CHLAUTH rules changes across Fix Packs.  In some cases it is the ID in the MQ Connection Data sent with the initial connection request.  In some cases, it is the MQ Connection Security Parameters data sent later in the handshake.  When these two data structures contain different user IDs the CHLAUTH behavior changes from Fix Pack to Fix Pack.

For complete details, please refer to the Google Sheet or PDF linked below.

Findings

The results were collected into a matrix.  It is available as a Google Sheet, a PDF and an Excel spreadsheet.  These are all downloadable from the Links page.

 Tools

The scripts used in this research are published on GitHub.  Originally I did not expect this project scope to explode the way it did so the testing is based on manually exercising of MQ Explorer against a local queue manager with a known set of channels and CHLAUTH rules.  Likewise, monitoring of event messages is performed through manual inspection.  The scripts switch the queue manager between combinations of ADOPTCTX, CHLAUTH rule types, and USERSRC(NOACCESS) for the different IDs presented by the client connections.

Over time I would like to generalize the scripts to eliminate hard-coded queue manager names and other dependencies, as well as automate the analysis of results. Pull requests are welcomed.  Despite the lack of such features, the current implementation of these tools removes much of the heavy lifting from the testing process and provides consistency, repeatability, and speed.

About

My services are available directly through IoPT Consulting, or subcontracting through any of several resellers. I have over 20 years of experience focused exclusively in MQ.  Although I’m focused strongly on security, I also cover architecture design, troubleshooting, clustering, high availability, and more, both as an instructor and on implementation/assessment projects.

I have made it my mission in life to improve the implementation of MQ both in the product itself and as it is practiced in the field.  If you’d like to help me fulfill that mission, chances are I can help you achieve your MQ mission at the same time.  Contact me by email or phone, on LinkedIn or as @tdotrob on Twitter.

This entry was posted in IBMMQ, Security, WMQ Security and tagged , , , , , . Bookmark the permalink.

9 Responses to MQ Password/CHLAUTH research – Exec Summary

  1. gisly says:

    Hi! I have got a problem with XMS connections and I am curious if you have come across it.
    The authentication for the Channel is set as Required, and the passwords for the connecting user are different between the MQ OS and the client OS. However, the user can successfully connect via the channel.

    • T.Rob says:

      Usually this means the CHLAUTH rule isn’t firing. Is there a backstop rule that would refuse connections if all the other rules fail to match?

  2. Pingback: MQGem Monthly (October 2016) | MQGem Software

  3. Ruud van Zundert says:

    Am I the only one who shrank back when the notion of using passwords with the MQ product was initially introduced? It reminded me of the old MQ product (was it v5?). I wonder how many MQ Admins are actually using passwords? However, following this article I shall refresh my knowledge. Thx … Ruud

    • T.Rob says:

      Not the only one and not in a minority, I suspect. But the number of shops that enabled password-based authentication when it became available seems to be substantial, based on my unscientific sample of personal observations.

      • Arun R says:

        I am curious about this exchange between Ruud and you. Is the suggestion that it is a bad thing? What is wrong with enabling password-based authentication? My thinking (albeit with incomplete understanding), is that this is much better than what existed before for client connections. Granted that it is not as good as certificates, but I have always been shocked at the possibility of user-spoofing and IP-spoofing that MQ seemed to allow before native password authentication.

        Please do provide some detail on your thoughts on this. And am I correct in understanding from Ruud’s comment, that the capability existed earlier, and was removed? Why?!

        • T.Rob says:

          > Is the suggestion that it is a bad thing?
          Not in and of itself. But a credential exchange needs to be protected and not just with obfuscation using a predefined key or deterministic algorithm. In MQ terms, that means a TLS connection is still required and once we do that we are already halfway to mutual certificate auth so passwords do not buy as much convenience as it might seem at first glance. There’s also the case that MQ is a transport for applications and the ability to use password credentials has resulted, entirely predictably, in many shops using non-expiring passwords and putting those credentials into parameter files, usually in plaintext. This just swaps one threat for another.

          But the thing that shocked you about MQ security is the main concern for me, and I believe Ruud as well. IBM’s historical track record with MQ security is…I’ll say “spotty” to be polite. This feature was no exception, as my post explains. Authorization must be in the context of an authenticated identity to be useful and in the initial releases of this feature IBM neglected to enforce a connection between the two. My research demonstrated the ability to authenticate as one user and assert a completely different one for authorization. That’s a fundamental cornerstone of password authentication yet it was completely overlooked in the design or testing, or both. Then the behavior changed frequently through multiple versions until it was finally stabilized and functioning. In the time between release and repair, indeed for some time after repair for folks who run back levels, users of this function were and are at more risk then not using it at all. At least when we know something is unsecure we are careful to apply other mitigating controls. When we think we’ve secured something but haven’t, the risk usually is net higher due to a false sense of security and relaxed vigilance.

          > And am I correct in understanding from Ruud’s comment, that the capability existed earlier, and was removed?
          No. The situation was that IBM provided the means for the client to send an ID and password and for a user-written exit to validate the password at the server side of the channel. MQ Explorer exposed the ID and password fields so shops with folks on staff capable of writing system level, reentrant, thread-safe, C code integrations between MQ and their Enterprise Account Management system could validate passwords. Since most shops didn’t write that exit (or buy it from Capitalware) password validation was not performed.

          Unfortunately, many (most?) people who saw the password field in MQ Explorer assumed it would not be there if MQ was ignoring it so, lots and lots of people happily put their credentials in there and broadcast them in plaintext. And since most shops only test whether access is granted and never test failure scenarios, most folks didn’t try to connect with any password value other than the real one. The way most users came to realize it wasn’t being validated was when it dawned on them that even though they had changed their password every 90 days, they had NEVER changed it in Explorer, yet it still worked. Oops.

          I suspect that part of the motivation for IBM to add password validation was that doing so would have been easier (and possibly less embarrassing to all involved) than telling the world-wide MQ user base that they’d been using unauthenticated identities and sending credentials mostly in plaintext for all those years. Easier to quietly back-fill the functionality users always assumed was there and enable it by default so the discussion becomes “the new version broke my authentication” rather than “holy crap, whaddaya mean my MQ has been wide open all these years?!?!?!”

          Hope that clarifies things a bit. Sorry for the delay, it seems my notifications are turned off or going to the spam folder. I see there are other comments I haven’t responded to so it’s been broke for a while. Sorry! My bad.

  4. Morag says:

    Compatibility mode only applies to Java clients of course.

    • T.Rob says:

      Good point, Morag! As a user of the client, yes Compatibility Mode is limited to Java-based clients. (What about XMS? Does that still model JMS feature-for-feature?)

      But as an admin at the server I have to assume an attacker would exploit this either by using a Java client where I’m expecting something else, or by bypassing MQ client code entirely. Recent Wireshark libraries have a very complete mapping of the MQ wire formats so the lack of public documentation isn’t that big an issue if a MITM attacker just wants to manipulate the MQCD values.

      So as a user trying to do the right thing, look for Compatibility Mode in Java-based clients. As an Admin who doesn’t have any legitimate Java clients, best to defend against differing values in the MQCSP and MQCD anyway. I will make that more clear in subsequent publications.

Leave a Reply to gislyCancel reply