[Gluster-devel] GlusterFS projects page for GSOC and the likes

[Xavier Hernandez]

Hi Shishir,

I'm trying to build a tool able to restore full integrity to any kind of 
gluster volume. The repairing will start at brick level and go up to the 
full volume.

Currently, the main idea is to begin with a local repair of each brick, 
which means to check that every single file and directory have the 
appropriate xattrs (basically gfid) and are correctly linked into 
.glusterfs directory. In this stage any inconsistency, duplicated gfid, 
bad structure inside .glusterfs and some other errors are tested and 
reported. When possible, these inconsistencies will be corrected.

After that, there will be more stages depending on the kind of volume to 
repair. I plan to handle replicated and distributed volumes. On each of 
these stages there will be two main operations:

1. Analyze the xattrs and file information collected during the previous 
repairing stage and perform all possible checks and repairs.
2. Cross check data from previous stages to help repair unsolved 
problems on those stages (for example use information from one brick of 
a replica to repair brick-level inconsistencies on the other brick, like 
missing or inconsistent gfid, duplications, ...).

It will also report as warnings things that are not problems by 
themselves but can have a performance impact, like linked files between 
bricks on a distributed volume. Optionally these things will be 
corrected by gfsck.

As an optional feature, it will be possible to take a standard volume 
with data and "glusterize" it to get a healthy brick (adding xattrs, 
.glusterfs structure, ...).

I have implemented some of these tests as independent small programs (no 
automatic correcting yet). I need to integrate and optimize them to be 
really useful, and implement some correction strategies. I'm also 
planning to use some kind of user interface to solve problems where it 
is hard or impossible to automatically choose the best alternative.

I use the libglusterfs.so to scan the volume files, and to use other 
helper functions (like the hashing used in dht). However I don't use all 
the xlator infrastructure to do the healing.

The gfsck could run with the volume mounted but it won't make any 
correction, only checks. To do any corrections it will need to run in 
offline mode.

If needed I can post some details about the type of errors detected on 
each stage.

Best regards,

Xavi

Al 24/04/13 07:55, En/na Shishir Gowda ha escrit:
> Hi Xavi,
>
> I would be interested in knowing what the gfsck tool would try to accomplish.
>
> I have certain scenarios from distribute xlator, which would be ideal candidates to be handled in fsck.
>
> Please let me know the scope of gfsck, so that I could share the ideas with you.
>
> With regards,
> Shishir
>
> ----- Original Message -----
> From: "Xavier Hernandez" <xhernandez at datalab.es>
> To: "Krishnan Parthasarathi" <kparthas at redhat.com>
> Cc: gluster-devel at nongnu.org
> Sent: Monday, April 22, 2013 7:20:36 PM
> Subject: Re: [Gluster-devel] GlusterFS projects page for GSOC and the likes
>
> I've just added 'gfsck', a tool to check file system integrity and
> repair any detected error.
>
> I'm already working on it.
>
> Xavi
>
> Al 22/04/13 15:03, En/na Krishnan Parthasarathi ha escrit:
>> Hi All,
>>
>> I am trying to collect all GlusterFS project ideas into a single page
>> in the wiki, here:
>> http://www.gluster.org/community/documentation/index.php/Projects
>>
>> I have added the first entry. It is about building a diagnostic tool
>> like nfsiostat for GlusterFS mounts. I volunteer to mentor anyone
>> interested in this.
>>
>> I hope to see more entries and volunteers :-)
>>
>> cheers,
>> krish
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> https://lists.nongnu.org/mailman/listinfo/gluster-devel