[Gluster-users] Targeted fix-layout?

[Jeff Darcy]

On 01/15/2013 01:10 PM, Dan Bretherton wrote:
> I am running a fix-layout operation on a volume after seeing errors mentioning
> "anomalies" and "holes" in the logs.  There is a particular directory that is
> giving trouble and I would like to be able to run the layout fix on that
> first.  Users are experiencing various I/O errors including "invalid argument"
> and "Unknown error 526", but after running for a week the volume wide
> fix-layout doesn't seem to have reached this particular directory yet.
> Fix-layout takes a long time because there are millions of files in the volume
> and the CPU load is consistently very high on all the servers while it is
> running, sometimes over 20. Therefore I really need to find a way to target
> particular directories or speed up the volume wide fix-layout.

You should be able to do the following command on a client to fix the layout 
for just one directory (it's the same xattr used by the rebalance command).

	setfattr -n distribute.fix.layout -v "anything" /bad/directory

> I have no idea what caused these errors but it could be related to the previous
> fix-layout operation, which I started following the addition of a new pair of
> bricks, not having completed successfully.  The problem is that the rebalance
> operation on one or more servers often fails before completing and there is no
> way (that I know of) to restart or resume the process on one server.  Every
> time this happens I stop the fix-layout and start it again, but it has never
> completed successfully on every server despite sometimes running for several
> weeks.
>
> One other possible cause I can think of is my recent policy of using XFS for
> new bricks instead of ext4.  The reason I think this might be causing the
> problem is that none of the other volumes have any XFS bricks yet and they
> aren't experiencing any I/O errors.  Are there any special mount options
> required for XFS, and is there any reason why a volume shouldn't contain a
> mixture of ext4 and XFS bricks?

It doesn't seem like that should be a problem, but maybe someone else knows 
something about ext4/XFS differences that could shed some light.