replicate/distribute oddities in 2.0.0 Was Re: [Gluster-devel] rc8
Gordan Bobic
gordan at bobich.net
Tue May 12 07:07:11 UTC 2009
Liam Slusser wrote:
>
> Even with manually fixing (adding or removing) the extended attributes i
> was never able to get Gluster to see the missing files. So i ended up
> writing a quick program that searched the raw bricks filesystem and then
> checked to make sure the file existed in the Gluster cluster and if it
> didn't it would tag the file. Once that job was done i shut down
> Gluster, moved all the missing files off the raw bricks into temp
> storage, and then i restarted Gluster and copied all the files back into
> each directory. That fixed the missing file problems.
>
> Id still like to find out why Gluster would ignore certain files without
> the correct attributes. Even removing all the file attributes wouldn't
> fix the problem. I also tried manually coping a file into a brick which
> it still wouldn't find. It would be nice to be able to manual copy
> files into a brick, then set an extended attribute flag which would
> cause gluster to see the new file(s) and copy them to all bricks after a
> ls -alR was done. Or even better just do it automatically when new
> files without attributes are found in a brick.
It sounds like you are experiencing this known yet dangerous bug:
http://gluster.org/docs/index.php/Understanding_AFR_Translator#Known_Issues
Quote:
Self-heal of a file that does not exist on the first subvolume:
If a file does not exist on the first subvolume but exists on some other
subvolume, it will not show up in the output of 'ls'. This is because
the replicate translator fetches the directory listing only from the
first subvolume. Thus, the file that does not exist on the first
subvolume is never seen and never healed. However, if you know the name
of the file and do a 'stat' on the file or try to access it in any other
way, the file will be properly healed and created on the first subvolume.
So, either the directory listing should be fetched from the
read-subvolume, or better, fetched from all nodes (but that gets slow).
At least if it was fetched from the read-subvolume, you could run a cron
job on each server that ls -laR, which would force the files into sync
(since each server probably has itself as the read-subvolume, so the
missing files will be found). But that's not how it seems to work at the
moment.
Gordan
More information about the Gluster-devel
mailing list