[Gluster-users] Transparent encryption in GlusterFS: Implications on manageability

Edward Shishkin edward at redhat.com
Wed Aug 13 17:35:42 UTC 2014

Hello everyone,

In this thread we'll consider in details all issues related to
the management aspects of the feature "Transparent encryption in

We assume that the reader is familiar with the basic concepts of the
feature "Transparent encryption in GlusterFS". See, for example,
slides 1-7 of the design document [2].

Comments, questions, any your experience are welcome.


             Transparent encryption in GlusterFS:
                Implications on manageability


              I. Managing master volume keys

Any user of the feature "Transparent encryption in GlusterFS" will
need to manage master volume keys (see man pages [1], section 6.2).

            I.1 Generating the master volume key

Master volume key should be generated by user on the trusted machine.
Recommendations on master key generation provided at section 6.2
of the manpages [1]. Generating of master volume key is in user's

           I.2 Location of the master volume key
                  when mounting a volume

At mount time the crypt translator searches for a master volume key
on the client machine at the location specified by the respective
translator option. If there is no any key at the specified location,
or the key at specified location is in improper format, then mount
will fail. Otherwise, the crypt translator loads the key to its
private memory data structures.

Location of the master volume key can be specified at volume creation
time (see option "master-key", section 6.7 of the man pages [1]).
However, this option can be overridden by user at mount time to specify
another location, see section 7 of manpages [1], steps 6, 7, 8.

             I.3 Retention of master volume keys
                    between mount sessions

After successful mount the file with master volume key can be removed
from the client machine: the crypt translator don't need it for _this_
mount session anymore: it will use the key installed in the private
memory data structures of the crypt translator. IMPORTANT: next mount
session will require the master volume key again, so if you remove the
file with master volume key after the mount command, make sure you
have stored it in the safe place for the next mount session.

Master volume key is a private secret key, so it is highly important
to provide its proper storage (on trusted machines, trusted media,
trusted network, etc). Otherwise it can be easily compromised.

Keeping the master volume key between mount sessions is in user's
competence. If user has more than one encrypted volume, he should
maintain the mapping volume <-> master-volume-key. Maintenance of
such mapping is in user's competence.

          I.4 Don't use the same master volume key
               for different Gluster volumes!

Different volumes should be encrypted with different master volume
keys. Using the same master volume key for different volumes is
highly not recommended because of security reasons.

   I.5 Using different master volume keys for the same volume

It is possible to mount a volume with a different master volume key
(see FAQ #4, section 11 of manpages [1]). In such mount session
content of files created with different master volume key will be
inaccessible. In particular, the crypt translator will refuse to open
files encrypted with different master volume key. However, it will be
possible to create new files on this volume and access them in the
mount sessions with respective master volume key.

In this case instead of simple mapping volume <-> master-volume-key
user will need to maintain mapping file <-> master-volume-key, which
is more complicated. Maintenance of this mapping is in user's

            I.6 Compromised master volume key

If your master volume key has been compromised because of some
reasons, or you suspect, that it has been compromised, then the whole
volume should be re-encrypted with the newly generated master volume
key. The common way is to create a new empty encrypted GlusterFS
volume, mount it with the new master volume key (see the manpages [1],
section 7), copy the old volume, mounted with compromised key to the
new one, mounted with the new key (for example, using cp(1) command.
After successful copy overwrite the content of all regular files of
your old volume with zeros. After this delete the old volume as usual.
Don't use the compromised master volume key anymore.

               II. Check graph of translators
             on your client machine after mount!

During mount your client machine receives configuration info from
the non-trusted server. In particular, this info contains the graph of
translators, which can be subjected to tampering, so that encryption
won't be invoked for your volume at all. So it is highly important to
verify this graph. After successful mount make sure that the graph of
translators contains the crypt translator with proper options (see
FAQ#1, section 11 of the manpages [1]).

         III. Changing the encryption status of the volume

Encryption status (enable/disable encryption) can be changed only
for empty volumes (with no files or directories). Changing encryption
status for not empty volumes is unacceptable: it will lead to IO
errors, data corruption and security problems. We highly recommend
to decide once and forever at volume creation time, whether your
volume has to be encrypted.


More information about the Gluster-users mailing list