[Gluster-devel] Change in glusterfs[master]: Transparent data encryption and metadata authentication in t...

[Kaleb KEITHLEY]

1.0.1e is in Fedora 18+, but Fedora policy is to not offer packages in 
EPEL that are also (already) in RHEL.

We could build the 1.0.1e src.rpm from Fedora on build.gluster.org. I 
have some small amount of confidence that it would be compatible with 
the 1.0.0 that's on there.

Thinking about the future though, it would certainly be a problem to 
require 1.0.1 for Gluster on CentOS and RHEL. We would probably need to 
disable encryption on platforms that don't have the required version of 
openssl. We should probably survey Ubuntu and Debian to see which of 
those have the right version. We're okay for RHEL 7.

Are we really using a feature that's only available in 1.0.1? I presume 
the answer is yes; I'm just doing due diligence by asking.

On 10/14/2013 05:27 PM, Anand Avati wrote:
> Edward,
> It looks like this patch requires a higher version of openssl (I recall
> you have mentioned before that that dependency was on version 1.0.1c? I
> checked yum update on the build server and the latest available version
> is 1.0.0-27. Is there a "clean" way to get the right version of openssl
> to a RHEL/CENTOS-6.x server?
>
> Also note that the previous submission of the patch was at
> http://review.gluster.org/4667. The recent on
> (http://review.gluster.org/6086) has a different Change-Id: in the
> commit log. It will be good if you can re-submit the patch with the old
> Change-Id (and abandon #6086) so that we can maintain the history of
> resubmission and the old work on records.
>
> Thanks!
> Avati
>
> On 10/14/2013 07:26 AM, Edward Shishkin (Code Review) wrote:
>> Edward Shishkin has uploaded a new change for review.
>>
>>    http://review.gluster.org/6086
>>
>>
>> Change subject: Transparent data encryption and metadata
>> authentication in the systems with non-trusted server (take II)
>> ......................................................................
>>
>> Transparent data encryption and metadata authentication
>> in the systems with non-trusted server (take II)
>>
>> This new functionality can be useful in various cloud technologies.
>> It is implemented via a special encryption/crypt translator, which
>> works on the client side and performs encryption and authentication;
>>
>>                1. Class of supported algorithms
>>
>> The crypt translator can support any atomic symmetric block cipher
>> algorithms (which require to pad plain/cipher text before performing
>> encryption/decryption transform (see glossary in atom.c for
>> definitions). In particular, it can support algorithms with the EOF
>> issue (which require to pad the end of file by extra-data).
>>
>> Crypt translator performs translations
>> user -> (offset, size) -> (aligned-offset, padded-size) ->server
>> (and backward), and resolves individual FOPs (write(), truncate(),
>> etc) to read-modify-write sequences.
>>
>> A volume can contain files encrypted by different algorithms of the
>> mentioned class. To change some option value just reconfigure the
>> volume.
>>
>> Currently only one algorithm is supported: AES_XTS.
>>
>> Example of algorithms, which can not be supported by the crypt
>> translator:
>>
>> 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA;
>> 2. Symmetric block cipher algorithms with inline MACs for data
>>     authentication.
>>
>>                     2. Implementation notes.
>>
>> a) Atomic algorithms
>>
>> Since any process in a stackable file system manipulates with local
>> data (which can be obsoleted by local data of another process), any
>> atomic cipher algorithm without proper support can lead to non-POSIX
>> behavior. To resolve the "collisions" we introduce locks: before
>> performing FOP->read(), FOP->write(), etc. the process should first
>> lock the file.
>>
>> b) Algorithms with EOF issue
>>
>> Such algorithms require to pad the end of file with some extra-data.
>> Without proper support this will result in losing information about
>> real file size. Keeping a track of real file size is a responsibility
>> of the crypt translator. A special extended attribute with the name
>> "trusted.glusterfs.crypt.att.size" is used for this purpose. All files
>> contained in bricks of encrypted volume do have "padded" sizes.
>>
>>                    3. Non-trusted servers and
>>                       Metadata authentication
>>
>> We assume that server, where user's data is stored on is non-trusted.
>> It means that the server can be subjected to various attacks directed
>> to reveal user's encrypted personal data. We provide protection
>> against such attacks.
>>
>> Every encrypted file has specific private attributes (cipher algorithm
>> id, atom size, etc), which are packed to a string (so-called "format
>> string") and stored as a special extended attribute with the name
>> "trusted.glusterfs.crypt.att.cfmt". We protect the string from
>> tampering. This protection is mandatory, hardcoded and is always on.
>> Without such protection various attacks (based on extending the scope
>> of per-file secret keys) are possible.
>>
>> Our authentication method has been developed in tight collaboration
>> with Red Hat security team and is implemented as "metadata loader of
>> version 1" (see file metadata.c). This method is NIST-compliant and is
>> based on checking 8-byte per-hardlink MACs created(updated) by
>> FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the
>> following unique entities:
>>
>> . file (hardlink) name;
>> . verified file's object id (gfid).
>>
>> Every time, before manipulating with a file, we check it's MACs at
>> FOP->open() time. Some FOPs don't require a file to be opened (e.g.
>> FOP->truncate()). In such cases the crypt translator opens the file
>> mandatory.
>>
>>                          4. Generating keys
>>
>> Unique per-file keys are derived by NIST-compliant methods from the
>>
>> a) parent key;
>> b) unique verified object-id of the file (gfid);
>> Per-volume master key, provided by user at mount time is in the root
>> of this "tree of keys".
>>
>> Those keys are used to:
>>
>> 1) encrypt/decrypt file data;
>> 2) encrypt/decrypt file metadata;
>> 3) create per-file and per-link MACs for metadata authentication.
>>
>>                            5. Instructions
>>                   Getting started with crypt translator
>>
>> Example:
>>
>> 1) Create a volume "myvol" by specifying the option "encrypt":
>>
>>     # gluster volume create myvol encrypt pepelac:/vols/xvol
>>
>> 2) Set location (absolute pathname) of your master key:
>>
>>     # gluster volume set myvol encryption.master-key /home/me/mykey
>>
>> 3) Set other options to override default options, if needed.
>>
>> 4) On the client side make sure that the file /home/me/mykey exists
>>     and contains proper per-volume master key (that is 256-bit AES
>>     key). This key has to be in hex form, i.e. should be represented
>>     by 64 symbols from the set  {'0', ..., '9', 'a', ..., 'f'}.
>>     The key should start at the beginning of the file. All symbols at
>>     offsets >= 64 are ignored.
>>
>> 5) Mount the volume "myvol" on the client side:
>>
>>     # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt
>>
>>     After successful mount the file which contains master key may be
>>     removed. NOTE: Keeping the master key between mount sessions is in
>>     user's competence.
>>
>> **********************************************************************
>>
>> WARNING! Losing the master key will make content of all regular files
>> inaccessible. Mount with improper master key allows to access content
>> of directories: file names are not encrypted.
>>
>> **********************************************************************
>>
>>                 6. Options of crypt translator
>>
>> 1) "master-key": specifies location (absolute pathname) of the file
>>     which contains per-volume master key. There is no default location
>>     for master key.
>>
>> 2) "data-key-size": specifies size of per-file key for data encryption
>>     Possible values:
>>     . "256" default value
>>     . "512"
>>
>> 3) "block-size": specifies atom size. Possible values:
>>     . "512"
>>     . "1024"
>>     . "2048"
>>     . "4096" default value;
>>
>>                         7. Test cases
>>
>> Any workload, which involves the following file operations:
>>
>> ->create();
>> ->open();
>> ->readv();
>> ->writev();
>> ->truncate();
>> ->ftruncate();
>> ->link();
>> ->unlink();
>> ->rename();
>> ->readdirp().
>>
>>                          8. TODOs:
>>
>> 1) Currently size of IOs issued by crypt translator is restricted
>>     by block_size (4K by default). We can use larger IOs to improve
>>     performance.
>>
>> Change-Id: Idc3523a8752888d3dd10f4fe71aa38dfc53d9ded
>> Signed-off-by: Edward Shishkin <edward at redhat.com>
>> ---
>> M cli/src/cli-cmd-parser.c
>> M configure.ac
>> M doc/gluster.8
>> M libglusterfs/src/gf-dirent.c
>> M libglusterfs/src/gf-dirent.h
>> M xlators/cluster/dht/src/dht-common.c
>> M xlators/encryption/Makefile.am
>> A xlators/encryption/crypt/Makefile.am
>> A xlators/encryption/crypt/src/Makefile.am
>> A xlators/encryption/crypt/src/atom.c
>> A xlators/encryption/crypt/src/crypt-common.h
>> A xlators/encryption/crypt/src/crypt-mem-types.h
>> A xlators/encryption/crypt/src/crypt.c
>> A xlators/encryption/crypt/src/crypt.h
>> A xlators/encryption/crypt/src/data.c
>> A xlators/encryption/crypt/src/keys.c
>> A xlators/encryption/crypt/src/metadata.c
>> A xlators/encryption/crypt/src/metadata.h
>> M xlators/mgmt/glusterd/src/glusterd-store.c
>> M xlators/mgmt/glusterd/src/glusterd-store.h
>> M xlators/mgmt/glusterd/src/glusterd-volgen.c
>> M xlators/mgmt/glusterd/src/glusterd-volume-ops.c
>> M xlators/mgmt/glusterd/src/glusterd-volume-set.c
>> M xlators/mgmt/glusterd/src/glusterd.h
>> M xlators/protocol/client/src/client-rpc-fops.c
>> 25 files changed, 8,733 insertions(+), 28 deletions(-)
>>
>>
>>    git pull ssh://git.gluster.org/glusterfs refs/changes/86/6086/1
>>
>