[Gluster-devel] Fwd: Change in glusterfs[master]: Transparent data encryption and metadata authentication...
Edward Shishkin
edward at redhat.com
Thu Mar 14 17:03:52 UTC 2013
On Wed, 13 Mar 2013 16:04:14 -0700
Anand Avati <anand.avati at gmail.com> wrote:
> On Wed, Mar 13, 2013 at 3:05 PM, Edward Shishkin <edward at redhat.com>
> wrote:
>
> >
> > Dear all,
> >
> > This requires openssl of version >= 1.0.1c
> >
> > Is it possible to upgrade the openssl on the machines, which
> > perform smoke tests? Any ideas? I can provide the instructions..
> >
> > I hope this feature will be useful and popular.
> >
> > Thanks,
> > Edward.
> >
> >
Hi Avati,
Actually, 1.0.1 (without 'c') will also work, but, I guess, it won't
make us more happy..
> Thanks for posting the patches! Can you give a quick summary of what
> 1.0.1c features are being used in the translator?
The following features are used (absent in 1.0.0):
1) XTS: cipher mode for data encryption;
2) GCM: cipher mode for metadata encryption and authentication;
3) CMAC: mode for authentication of non-encrypted part of metadata;
> Is it possible to
> re-implement some code to depend on openssl-1.0.0?
Perhaps, we can find a replacement for (3) in the 1.0.0 (using HMACs
instead of CMACs). However, absence of (1) and (2) seems to be a real
problem: 1.0.0 doesn't have algorithms suitable for our purposes.
Existing ones do have different issues. For example, the popular mode
CBC doesn't sustain overwrites with the same key, etc.
We can copypaste the absent code to the crypt translator. However, it
would mean that this code will be beyond the reach of openssl people,
which is not good..
> The build system
> is CentOS 6.3 (~= RHEL 6.3) - which is a very popular "base" for
> deploying glusterfs. I suspect depending on 1.0.1c openssl will be
> more than just a "build server dependency" issue.
Maybe it makes sense to ifdef the crypt translator and the related
code (depending on the openssl version, or having the respective
header files, say <openssl/cmac.h>)? I can see similar tricks in
the current Gluster code (HAVE_BD_XLATOR and friends)..
Thanks,
Edward.
>
> Avati
>
>
>
> >
> > On Wed, 13 Mar 2013 17:25:41 -0400 (EDT)
> > John Mark Walker <johnmark at redhat.com> wrote:
> >
> > > This marks an interesting development in GlusterFS. If you've been
> > > looking for data encryption, you may want to try this patch.
> > >
> > > -JM
> > >
> > >
> > > -------- Original Message --------
> > > Subject: Change in glusterfs[master]: Transparent data
> > > encryption and metadata authentication... From: "Edward Shishkin
> > > (Code Review)" <root at dev.gluster.com> To:
> > > CC:
> > >
> > >
> > >
> > >
> > > Edward Shishkin has uploaded a new change for review.
> > >
> > > Change subject: Transparent data encryption and metadata
> > > authentication in the systems with non-trusted
> > > server
> > ......................................................................
> > >
> > > Transparent data encryption and metadata authentication
> > > in the systems with non-trusted server
> > >
> > > This new functionality can be useful in various cloud
> > > technologies. It is implemented via a pair of the following
> > > interacting translators:
> > >
> > > . encryption/crypt, which works on client side and performs
> > > encryption and authentication;
> > > . features/oplock, which works on server side and resolves
> > > "conflicts"
> > >
> > > 1. The class of algorithms for data encryption,
> > > that can be supported by this pair of translators
> > >
> > > The mentioned pair of translators can support any atomic symmetric
> > > block cipher algorithms (which require to pad plain/cipher text
> > > before performing encryption/decryption transform (see glossary
> > > in atom.c for definitions). In particular, it can support
> > > algorithms with the EOF issue (which require to pad the end of
> > > file by extra-data).
> > >
> > > In most cases crypt translator translates the pair (offset, count)
> > > passed by user to different values, and resolves individual
> > > ->write() ->truncate(), etc. file operations to read-modify-write
> > > sequences.
> > >
> > > A volume can contain files encrypted by different algorithms. For
> > > newly created files one can specify desirable algorithm at mount
> > > time via a respective option of crypt translator.
> > >
> > > Currently only one algorithm is supported: AES_XTS.
> > >
> > > Example of algorithms, which can not be supported by this pair of
> > > translators:
> > >
> > > 1. Asymmetric block cipher algorithms, which inflate data, e.g.
> > > RSA; 2. Symmetric block cipher algorithms with inline MACs for
> > > data authentication.
> > >
> > > 2. Implementation notes.
> > >
> > > a) Atomic algorithms
> > >
> > > Since any process in a stackable file system manipulates with
> > > local data (which can be obsoleted by local data of another
> > > process), atomic cipher algorithms without proper support can
> > > lead to non-POSIX behavior. To resolve the "collisions" we
> > > introduce a special helper translator (features/oplock), which
> > > works on the server and manages requests (queues, grants access)
> > > of read/write issued by the clients. When an exclusive access is
> > > granted to client, the last one performs cipher transform and
> > > proceeds the stack. After all the client's work is done on the
> > > server, the oplock translator drops the access, and wakes up the
> > > next request in the queue (if any). Our implementation guarantees
> > > that an access will be granted to every concurrent process, which
> > > accesses the same file (i.e. the process won't hang).
> > >
> > > b) Algorithms with EOF issue
> > >
> > > Such algorithms require to pad the end of file with some
> > > extra-data. Without proper support this will result in losing
> > > information about real file size. Keeping a track of real file
> > > size is a responsibility of the mentioned features/oplock
> > > translator. When writing/truncating a file, the oplock translator
> > > cuts the padding and stores the last one as a special extended
> > > attribute with the key "trusted.ceof". When reading a file,
> > > oplock translator appends the respective padding. So, in the
> > > bricks every file has its real size.
> > >
> > > Comment. This makes transparent encryption incompatible with
> > > GlusterFS striping and replication translators, which spawn
> > > extra-writes to stripe/replica files without due interaction with
> > > the oplock translator.
> > >
> > > 3. Non-trusted servers and
> > > Metadata authentication
> > >
> > > We assume that server, where user's data is stored on is
> > > non-trusted. It means that the server can be subjected to various
> > > attacks directed to reveal user's encrypted personal data. We
> > > provide protection against such attacks.
> > >
> > > Every encrypted file has specific private attributes (cipher
> > > algorithm id, atom size, trusted object id, etc), which are
> > > packed to a string (so-called "format string") and stored as a
> > > special extended attribute with the key "trusted.cfmt". We
> > > protect the string from tampering. This protection is mandatory,
> > > hardcoded and is always on. Without such protection various
> > > attacks (based on extending the scope of per-file secret keys)
> > > are possible.
> > >
> > > Our authentication method has been developed in tight
> > > collaboration with Red Hat security team and is implemented as
> > > "metadata loader of version 1" (see file metadata.c). This method
> > > is NIST-compliant and is based on checking 8-byte per-link MACs
> > > created(updated) by FOP->create(), FOP->link(), FOP->unlink(),
> > > FOP->rename() by the following unique entities:
> > >
> > > . link name;
> > > . trusted file's uuid, specially created on the (trusted) client
> > > side
> > >
> > > Every time, before manipulating with a file, we check it's MACs at
> > > FOP->open() time. Some FOPs don't require a file to be opened
> > > (e.g. FOP->truncate()). In such cases the crypt translator opens
> > > the file mandatory.
> > >
> > > 4. Generating keys
> > >
> > > Unique per-file keys are derived by NIST-compliant methods (file
> > > keys.c) from the
> > >
> > > a) parent key;
> > > b) unique trusted object-id of the file;
> > >
> > > Per-volume master key, provided by user at mount time is in the
> > > root of this "tree of keys".
> > >
> > > Those keys are used to:
> > >
> > > 1) encrypt/decrypt file data;
> > > 2) encrypt/decrypt file metadata;
> > > 3) create per-file and per-link MACs for metadata authentication.
> > >
> > > 5. Instructions
> > > how to use the new functionality
> > >
> > > 1) Specify an option "encrypt" when creating a volume.
> > >
> > > Example:
> > > # gluster volume create myvol encrypt pepelac:/root/exp8
> > >
> > > 2) On the client side, when mounting a volume, specify the
> > > absolute name of the file, which contains per-volume master key,
> > > overriding the option "key" of the crypt translator. This file
> > > should contain 256-bit AES key in the hex form, i.e. 64 symbols.
> > > Crypt translator accepts the first 64 symbols of the specified
> > > file. Other extra-symbols are ignored.
> > > After successful mount the file with master key may be removed.
> > >
> > > Example:
> > > # glusterfs --xlator-option=myvol-crypt.key=/home/edward/mykey
> > > \ --volfile-id=myvol --volfile-server=pepelac /mnt/gluster
> > >
> > > WARNING! Losing the master key means losing the data of the whole
> > > volume without any chances to recovery.
> > >
> > > 6. Options of the crypt translator
> > >
> > > . "key" (specifies names of the file which contains per-volume
> > > master key); . "kbits" (specifies size of per-file key for data
> > > encryption), possible values:
> > > . "256" default value
> > > . "512"
> > > . "blocksize" (specifies the atom size), possible values:
> > > . "512"
> > > . "1024"
> > > . "2048"
> > > . "4096" default value;
> > > . id of algorithm for data encryption (hidden option);
> > > . id of metadata loader (hidden option);
> > >
> > > 7. Test cases
> > >
> > > Any workload, which involves the following file operations:
> > >
> > > ->create();
> > > ->open();
> > > ->readv();
> > > ->writev();
> > > ->truncate();
> > > ->ftruncate();
> > > ->link();
> > > ->unlink();
> > > ->rename();
> > >
> > > 8. TODOs:
> > >
> > > 1) Currently iov_len coincides with atom_size (4K by default). We
> > > can introduce larger units for IOs to improve performance.
> > >
> > > 2) Show encryption status (on/off) of the volume in gluster volume
> > > info.
> > >
> > > Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee
> > > Signed-off-by: Edward Shishkin <edward at redhat.com>
> > > ---
> > > M cli/src/cli-cmd-parser.c
> > > M configure.ac
> > > M doc/gluster.8
> > > M xlators/encryption/Makefile.am
> > > A xlators/encryption/crypt/Makefile.am
> > > A xlators/encryption/crypt/src/Makefile.am
> > > A xlators/encryption/crypt/src/atom.c
> > > A xlators/encryption/crypt/src/crypt-common.h
> > > A xlators/encryption/crypt/src/crypt-mem-types.h
> > > A xlators/encryption/crypt/src/crypt.c
> > > A xlators/encryption/crypt/src/crypt.h
> > > A xlators/encryption/crypt/src/data.c
> > > A xlators/encryption/crypt/src/keys.c
> > > A xlators/encryption/crypt/src/metadata.c
> > > A xlators/encryption/crypt/src/metadata.h
> > > M xlators/features/Makefile.am
> > > A xlators/features/oplock/Makefile.am
> > > A xlators/features/oplock/src/Makefile.am
> > > A xlators/features/oplock/src/oplock-mem-types.h
> > > A xlators/features/oplock/src/oplock.c
> > > A xlators/features/oplock/src/oplock.h
> > > M xlators/mgmt/glusterd/src/glusterd-volgen.c
> > > M xlators/mgmt/glusterd/src/glusterd-volume-ops.c
> > > M xlators/mgmt/glusterd/src/glusterd.h
> > > 24 files changed, 10,159 insertions(+), 19 deletions(-)
> > >
> > >
> > > git pull ssh://git.gluster.org/glusterfs refs/changes/67/4667/1
> > > --
> > > To view, visit http://review.gluster.org/4667
> > > To unsubscribe, visit http://review.gluster.org/settings
> > >
> > > Gerrit-MessageType: newchange
> > > Gerrit-Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee
> > > Gerrit-PatchSet: 1
> > > Gerrit-Project: glusterfs
> > > Gerrit-Branch: master
> > > Gerrit-Owner: Edward Shishkin <edward.shishkin at gmail.com>
> >
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/gluster-devel
> >
More information about the Gluster-devel
mailing list