[Gluster-devel] Permissions and ownership ...

[Raghavendra G]

Hi Gareth ,

Can you please send:
1. glusterfs client and server logs. (run both client and server with
options -l logfile -L DEBUG)
2. glusterfs client and server configuration files.

Also can you check whether the following commands complete successfully on a
glusterfs mount:
1. chmod
2. chown

Which is the backend file system you are using?

regards,

On Dec 28, 2007 3:30 AM, Gareth Bult <gareth at encryptec.net> wrote:

> Hi,
>
> I've run into another issue trying to run applications on a glusterfs ..
> in particular something to down with the setting of permissions / ownership.
>
> Here's a sample from a "Zimbra" install log;
>
> (Reading database ... 13490 files and directories currently installed.)
> Unpacking zimbra-core (from
> .../zimbra-core_5.0.0_GA_1869.UBUNTU6_i386.deb) ...
> dpkg: error processing
> ./packages/zimbra-core_5.0.0_GA_1869.UBUNTU6_i386.deb (--install):
>  error setting ownership of `./opt/zimbra/db/create_database.sql':
> Function not implemented
> dpkg-deb: subprocess paste killed by signal (Broken pipe)
> Errors were encountered while processing:
>  ./packages/zimbra-core_5.0.0_GA_1869.UBUNTU6_i386.deb
>
> Typically I see lots of errors if I try;
>
> root at zimbra:/opt# cp -rav /bin /mnt/cluster/mail/
> `/bin' -> `/mnt/cluster/mail/bin'
> `/bin/dash' -> `/mnt/cluster/mail/bin/dash'
> cp: failed to preserve ownership for `/mnt/cluster/mail/bin/dash':
> Function not implemented
> `/bin/which' -> `/mnt/cluster/mail/bin/which'
> cp: failed to preserve ownership for `/mnt/cluster/mail/bin/which':
> Function not implemented
>
> ... anyone any idea what causes this ?
> ... what am I doing wrong?
>
> "cp -rv" works fine - no problems at all.
>
> Is there a solution .. this far the only "application" I've successfully
> managed to make work running off a live glusterfs are xen virtual filesystem
> instances ..
>
> tia
> Gareth.
>
> ----- Original Message -----
> From: "Gareth Bult" <gareth at encryptec.net>
> To: "Kevan Benson" <kbenson at a-1networks.com>
> Cc: "gluster-devel" <gluster-devel at nongnu.org>, "Gareth Bult" <
> gareth at encryptec.net>
> Sent: Thursday, December 27, 2007 11:00:48 PM (GMT) Europe/London
> Subject: Re: [Gluster-devel] Choice of Translator question
>
> This could be the problem.
>
> When I do this on a 1G file, I have 1 file in each stripe partition of
> size ~ 1G.
>
> I don't get (n) files where n=1G/chunk size ... (!)
>
> If I did, I could see how it would work .. but I don't ..
>
> Are you saying I "definitely should" see files broken down into multiple
> sub files, or were you assuming this is how it worked?
>
> Gareth.
>
>
> ----- Original Message -----
> From: "Kevan Benson" <kbenson at a-1networks.com>
> To: "Gareth Bult" <gareth at encryptec.net>
> Cc: "gluster-devel" <gluster-devel at nongnu.org>
> Sent: Thursday, December 27, 2007 8:16:53 PM (GMT) Europe/London
> Subject: Re: [Gluster-devel] Choice of Translator question
>
> Gareth Bult wrote:
> >> Agreed, which is why I just showed the single file self-heal
> >> method, since in your case targeted self heal (maybe before a full
> >> filesystem self heal) might be more useful.
> >
> > Sorry, I was mixing moans .. on the one hand there's no log hence no
> > automatic detection of out of date files (which means you need a
> > manual scan), and secondly, doing a full self-heal on a large
> > file-system "can" be prohibitively "expensive" ...
> >
> > I'm vaguely wondering if it would be possible to have a "log"
> > translator that wrote changes to a namespace volume for quick
> > recovery following a node restart. (as an option of course)
>
> An interesting thought.  Possibly something that keeps a filename and
> timestamp so other AFR members could connect and request changed file
> AFR versions since X timestamp.
>
> Automatic self-heal is supposed to be on the way, so I suspect they are
> already doing (or planning) something like this.
>
> >> I don't see how the AFR could even be aware the chunks belong to
> >> the same file, so how it would know to replicate all the chunks of
> >> a file is a bit of a mystery to me.  I will admit I haven't done
> >> much with the stripe translator though, so my understanding of it's
> >> operation may wrong.
> >
> > Mmm, trouble is there's nothing definitive in the documentation
> > either way .. I'm wondering whether it's a known critical omission
> > which is why it's not been documented (!) At the moment stripe is
> > pretty useless without self-heal (i.e. AFR). AFR is pretty useless
> > without stripe for anyone with large files. (which I'm guessing is
> > why stripe was implemented after all the "stripe is bad"
> > documentation) If the the two don't play well and a self-heal on a
> > large file means a 1TB network data transfer - this would strike me
> > as a show stopper.
>
> I think the original docs said it was implemented because it was easy,
> but there wasn't a whole lot to be gained by using it.  Since then, I've
> seen people post numbers that seemed to indicate it gave a somewhat
> sizable boost, but the extra complexity in introduced never made it
> attractive to me.
>
> The possibility it could be used to greatly speed up self-heal on large
> files seems like a real good reason to use it though, so hopefully we
> can find a way to make it work.
>
> >> Understood.  I'll have to actually try this when I have some time,
> >> instead of just doing some armchair theorizing.
> >
> > Sure .. I think my tests were "proper" .. although I might try them
> > on TLA just to make sure.
> >
> > Just thinking logically for a second, for AFR to do chunk level
> > self-heal, there must be a chunk level signature store somewhere. ...
> > where would this be ?
>
> Well, to AFR each chunk should just look like another file, it shouldn't
> care that it's part of a whole.
>
> I assume the stripe translator uses another extended attribute to tell
> what file it's part of.  Perhaps the AFR translator is stripe aware and
> that's causing the problem?
>
> >> Was this on AFR over stripe or stripe over AFR?
> >
> > Logic told me it must be AFR over stipe, but I tries it both ways
> > round ..
>
> Let get rid of the over/under terminology (which I always seem to think
> of reverse from other people), and use a representation that's more
> absolute:
>
> client -> XLATOR(stripe) -> XLATOR(AFR) -> diskVol(1..N)
>
> Throw in your network connections wherever you want, but this should be
> testable on a single box with two different directories exported as
> volumes.
>
> The client writes to the stripe translator, which splits up the large
> file, which is then sent to the AFR translator so each chunk is stored
> redundantly in each disk volume supplied.
>
> If the AFR and stripe are reversed, it will have to pull all stripe
> chunks to do a self heal (unless AFR is stripe aware), which isn't what
> we are aiming for.
>
> Is that similar to what you tested?
>
> --
>
> -Kevan Benson
> -A-1 Networks
>
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>



-- 
Raghavendra G

A centipede was happy quite, until a toad in fun,
Said, "Prey, which leg comes after which?",
This raised his doubts to such a pitch,
He fell flat into the ditch,
Not knowing how to run.
-Anonymous