We’re gonna need a bigger crowd

What not to crowdsource: Specialty training

What not to crowdsource: Specialty training

Crowdsourcing: Collaboration based on the idea that given a sufficiently large pool of talent, it is possible to create a specific deliverable of high quality and in a timely fashion, using donated excess capacity of the crowd.

Some things just don’t crowdsource easily.

At its core, crowdsourcing is a slight twist on that old saw about “You can have it fast, cheap or right.  Pick any two and let me know.”  The twist is that “cheap” has been stipulated as a given and replaced with “crowdsourced.”

You can have it crowdsourced.
You can have it fast.
You can have it right.
Pick any two and get back to me.

But crowdsourcing works great, right?  There’s Wikipedia, Linux, FreeDB and many other examples one can point to of high quality products built by crowds.  But what I’d invite you to do is to consider the one thing where crowdsourcing tends to fall down.

Specialty training.

Remember, the premise of crowdsourcing is that somewhere dormant within the crowd lies sufficient expertise and capacity to produce the desired content.  But if the desired content happens to be training material, then there’s an inverse relation between the availability of skilled resources versus the magnitude of the need for the courseware.  By definition, the more you need high quality training the less likely you are to find the skills to produce it.

So it’s no surprise that when it comes to a niche specialty like security, or even sub-specialties such as certificate management, the “prevailing wisdom” is long on prevailing and short on wisdom.  The pool of talent contributing to the body of free knowledge isn’t sufficiently large to produce consistently high quality materials.  Or, more to the point, the pool of relatively unskilled talent producing security content is much larger.  Worse, the nature of security is that you often don’t know how good or bad it is until after the breach.  Despite the prevalence of breaches in the news, they are rare enough that security that is purely a facade and completely ineffective appears to the buyer to be as just as good as truly effective security.  Except that the security facade is invariably cheaper, easier to use, or both, so there’s an in-built incentive to favor crappy security content.

A case in point is the idea of a company running their own internal Certificate Authority.  If you spend 30 seconds on Google you can find dozens of tutorials on how to set up and run your own internal CA.  They explain in great detail all the commands necessary to generate a CA certificate and use it to sign application and user certificates.  I have yet to find one tutorial that makes even a half-hearted attempt to explain why this might be a bad idea, except of course for the tutorials from the commercial CAs.  The problem is that the role of a CA is distilled in these tutorials down to the mechanics of signing certificates and all the other aspects of a robust CA are ignored.

What are those aspects, you ask?  Consider that asymmetric security was invented in part to solve the key distribution problem.  With X.509 certificates and public/private keys it is possible to eliminate the requirement to securely distribute the secret key to both sides.

But once we solved that problem, distributing the public key was just too much work so we invented the Certificate Authority.  Rather than trust individual self-signed certificates, we decided to trust a CA, and by extension every certificate that CA has ever or will ever sign.  Essentially, the burden of tight security was shifted to the CA.  When described in those terms, it seems obvious that concentrating all that trust in a CA requires it to be a lot more secure than its customers.  To meet that requirement, CAs usually take what would be considered extraordinary measures in an average IT shop.  For example a robust commercial CA will manage a root certificate using dual-knowledge protocols so that no one person ever has exclusive access to it.  A robust commercial CA will ensure that no two certificates issued will have the same name or serial number.  A robust commercial CA runs a secure and highly available revocation responder.

These are just a few examples of how a commercial CA provides a robust and trustworthy service.  There is actually an intricate mesh of intrusion prevention, intrusion detection, auditing, access control and access revocation going on behind the scenes.  Is it foolproof?  No.  Is it something we can afford to abandon to move to our own internal CA?  Again, no.

Unfortunately, we are so far past the original goals of asymmetric encryption that we’ve forgotten why we needed it, and the CA system, in the first place.  When we want to replace the commercial CA, instead of falling back to what we had before, we start with the CA concept and seek to bring it in-house.

But if your internal CA…

  • doesn’t use intermediate signers…
  • doesn’t have a revocation server, or the revocation server update process is insecure…
  • doesn’t enforce uniqueness of names or serial numbers…
  • doesn’t have separate roots for production versus non-production…
  • doesn’t have a real-time inventory of all certs issued, their owners and their current status…
  • doesn’t audit every access to the root or intermediate signer…
  • doesn’t run on its own dedicated, built-to-purpose, hardened servers…

…then you might want to stick with a commercial CA or consider self-signed certificates.  One of the assumptions underlying the CA system is that the CA is extraordinarily more secure than its customers.  If you run an internal CA without making that extraordinary investment in not just configuration but also process controls and deep skills, then you haven’t replaced the CA.  You’ve built a security facade.  Unfortunately, this is what most of the literature available tells you to do.

Of course, this CA example is just that – an example.  The bigger point here is that the more specialized the skill set, the less likely that free content will be quality content.  When it comes to security, paying does not guarantee high quality, but not paying almost always guarantees poor quality.

The exceptions are curated and peer-reviewed content from authoritative sources such as OWASP and The SANS Institute.  Second-tier sources are reputable companies such as IBM X-Force, Verizon’s security team, Ponemon Institute and so on.  Third tier are individual but highly reputable security professionals and researchers.  These are people whose primary skill set and training is security.  Fourth tier are people like me.  I am an MQ administrator who acquired a secondary interest in security.  I’m pretty good at what I do, even approaching authoritative on a very few topics, but realistically I’m still down there with Kathy Griffin on the D List.

Somewhere way below that…

…at the 5th, 6th or 7th tier…

…is where all the free security advice seems to originate.

How else do we explain that “broken account management” has held a place in the OWASP Top 10 Web App Vulnerabilities list continuously since the list was started a over a decade ago?

Or that companies are falling all over themselves to replace commercial CAs with internal CAs run on a thumb drive under some admin’s desk?

Or that the Gold Standard for security in asynchronous messaging, a transport that by definition involves 3rd party escrow of transient data, is to secure the connection and not the data itself?

Or that “security” is usually managed and budgeted as a one-time configuration task rather than a core discipline undertaken as an ongoing practice consisting mostly of procedural controls and continuing education?

Get a mix of sources for your security advice.  Make sure some of them are authoritative sources.   If it’s a specialty like MQ, or something new like Personal Clouds or Internet of Things, consider paying a specialist. Get a second opinion.

Crowdsource anything else.  Hell, crowdsource everything else.  But please, if you value your business, don’t crowdsource your security training.  Just because it’s free doesn’t mean you can afford it.

This entry was posted in Security. Bookmark the permalink.

4 Responses to We’re gonna need a bigger crowd

  1. Long says:

    Just by chance that I found out about your where about! I have rarely met anyone who’s so articulate about the topics (WMQ and WMQ security in particular) as you. Looking forward to chances working with you again. As a matter of fact, I took the liberty to mention your name and to reference your blogs/articles in a session today with a WMQ’s customer. It’s a beautiful day here in Pittsburgh.

    • T.Rob says:

      Thanks, Long! I had a rather busy last week at IBM and didn’t get around to setting up an out-of-office or sending out a massive email which was unfortunate. (Actually spent the last couple of days performing military-grade wipe of laptop and was offline.) Thanks for the referrals and the kind words. I am on-boarded through ISSW and can work through them, through other partners or independently so there’s a good chance we’ll bump into one another sooner or later. Definitely see you if you can make it to IMPACT next year.

  2. T.Rob says:

    Shhhhhhh! That part is supposed to be subliminal!

  3. george carey says:

    Well written article and even better subtle soft sell of ones skill set!

Leave a Reply