[Gluster-devel] Securing GlusterD management

[Jeff Darcy]

As some of you might already have noticed, GlusterD has been notably insecure ever since it was written.  Unlike our I/O path, which does check access control on each request, anyone who can craft a CLI RPC request and send it to GlusterD's well known TCP port can do anything that the CLI itself can do.  TLS support was added for GlusterD a while ago, but it has always been a bit problematic and as far as I know hasn't been used much.  It's a bit of a chicken-and-egg problem.  Nobody wants to use a buggy or incomplete feature, but as long as nobody's using it there's little incentive to improve it.

Recently, there have been some efforts to add features which would turn the existing security problem into a full-fledged "arbitrary code execution" vulnerability (as the security folks would call it).  These efforts have been blocked, but they have also highlighted the fact that we're *long* past the point where we should have tried to make GlusterD more secure.  To that end, I've submitted the following patch to make TLS mandatory for all GlusterD communication, with some very basic authorization for CLI commands.

   http://review.gluster.org/#/c/14866/

The technical details are in the commit message, but the salient point is that it requires *zero configuration* to get basic authentication and encryption.  This is equivalent to putting a lock on the door.  Sure, maybe everybody knows the default combination, but *at least there's a lock* and people who want to secure their systems can change the combination to whatever they want.  That's better than the door hanging open, without even a solid attachment point for a lock, and it's essential infrastructure for anything else we might do.  The patch also fixes some bugs that affect even today's optional TLS implementation.

One significant downside of this change has to do with rolling upgrades.  While it might be possible for those who are already using TLS to do a rolling upgrade, it would still require some manual steps.  The vast majority of users who haven't enabled TLS will be unable to upgrade without "stopping the world" (as is already the case for enabling TLS).

I'd appreciate feedback from users on both the positive and negative aspects of this change.  Should it go into 3.9?  Should it be backported to 3.8?  Or should it wait until 4.0?  Feedback from developers is also appreciated, though at this point I think any problems with the patch itself have already been resolved to the point where GlusterFS with the patch is more stable than GlusterFS without it.  I'm just fighting through some NetBSD testing issues at this point, hoping to make that situation better as well.