[Gluster-users] strange error hangs any access to gluster mount

[Burnash, James]

Hi Jeff - sorry for the delay. Crazy busy here ... as usual. I'm actually fighting for the continued use of this platform. The internal client has (understandably) gotten a bit nervous about continued use after all of the issues I ran into while doing the migration from 3.0.4 to 3.1.1.

I did modify the layout xattr directly as suggested by Amar - which means that I actually deleted the existing on brick g03. I did not (that I recall) to a rebalance, but the correct attributes seem to have appeared by the next time I looked (approx the next day).

I did not have to do a full migration of the weird sub volume, though I was prepping to do so, and going forward I am going to try to have a "spare" brick available on each storage server in case this needs to be done in the future. Seems like it would make things much easier.

Hope this helps. As for the IRC - it's blocked here, but I can get it on my Android, so I will hook that up to a power source here at work and run the client so that I can participate. Thanks for the tip.

One other recent discovery that I literally stumbled onto. It appears that during the migration I missed a dozen or better version 3.0.4 clients because they had not accessed the bricks that I used to generate my client list. Sloppy of me, and something I will endeavor not to repeat. They may have contributed to the instability of the problem volume(s).

Amar / Gluster Devs - if old incompatible clients are accessing the storage servers, I imagine that they know that at some level - however I saw nothing in the logs that led me to detect them. Could you point me towards what patterns in the logs I should be looking for / unique error messages, or configuration settings that I should enable to find this issue?

Onward and upward ... Gluster HO!

James Burnash, Unix Engineering

-----Original Message-----
From: Jeff Darcy [mailto:jdarcy at redhat.com] 
Sent: Monday, April 18, 2011 9:39 AM
To: Burnash, James
Cc: 'Amar Tumballi'; gluster-users at gluster.org
Subject: Re: [Gluster-users] strange error hangs any access to gluster mount

On 04/18/2011 09:23 AM, Burnash, James wrote:
> After taking the actions suggested by Jeff and confirmed by Amar, it
> looks like all of my maps are clean (thought I am still having problems,
> which may or may not be related).
> 
>  
> 
> Here are the maps after all the corrections were done:
> 
>  ...
> 
> How do these look to you? I don’t see a gap/hole, and since mirrored
> pairs are claiming (only) the same range for each mirrored brick, all
> looks ok to me.  Can  one / both of you just confirm that my layout now
> looks clean?

Those look much healthier to me, and I'm glad to hear that at least some
issues seem to have been resolved.  Just for future reference, did the
suggestion to modify the layout xattr directly work, or did you have to
do a full manual migration of the files on the "weird" subvolume?  It
seems like a couple of other users on IRC have run into layout issues in
3.1.3 that look disturbingly similar, so it would be nice to know which
fixes work and which don't.  Thanks!


DISCLAIMER: 
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. 
NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com