[Gluster-users] xattr and ZFS
Vijay Bellur
vbellur at redhat.com
Tue Jan 14 18:21:36 UTC 2014
On 01/14/2014 05:34 AM, Franco Broi wrote:
> On Mon, 2014-01-13 at 23:48 +0530, Vijay Bellur wrote:
>> On 01/13/2014 01:54 PM, Franco Broi wrote:
>>> Hi all
>>>
>>> I have about 400TB of storage running dht on ZFS and want to change the
>>> way the extended attributes are stored, assuming it's safe to do this in
>>> ZFS version 0.6.2? I discovered by experimentation that if I remove the
>>> attributes on the server, Gluster will remake them when a file is first
>>> accessed. So the question is, would it be safe to simply delete all the
>>> attributes from all files and just let Gluster remake them?
>>>
>>
>> GlusterFS does maintain state in extended attributes and manually
>> altering this state information has to be done very carefully.Removing
>> extended attributes for directories is not recommended as this can
>> potentially result in gfid split brains.
>>
>> As far as files go in a distributed cluster (with no replication), the
>> gfid attribute can be removed and it gets re-created on access. What
>> other attributes do you intend removing?
>
> I thought I might need to remove trusted.glusterfs.dht as well.
>
> Which attributes get access when reading a directory? Does a stat on a
> file need to read attributes?
Reading a directory does result in extended attributes used by md-cache
translator and others being read.A lookup operation that can precede a
stat does read extended attributes.
>
> I just did a test where I made 1000 directories over 8 bricks. I removed
> trusted.gfid and trusted.glusterfs.dht and gluster remade them when I
> did an ls -l from the client.
A lookup from a client can heal these extended attributes. In the past
(primarily in 3.2.x versions), we had problems when directories missed
extended attributes and multiple clients tried to access & set these
attributes, we would end up with different gfids for the same
directories in different bricks. We have added resiliency to prevent
such occurrences beyond 3.2 but it is best to attempt this operation
when your production storage system is not heavily loaded.
Is this change to make use of SA based extended attributes in zfs?
-Vijay
More information about the Gluster-users
mailing list