[Gluster-users] Client and server file "view", different results?! Client can't see the right file.
Pranith Kumar. Karampuri
pranithk at gluster.com
Thu May 19 11:40:27 UTC 2011
hi Martin,
Manually copying the file may lead to same files with different gfids, which will again lead to the same problem.
Make sure that the "trusted.gfid" attribute values are same on this file on both the replicas.
Pranith
----- Original Message -----
From: "Martin Schenker" <martin.schenker at profitbricks.com>
To: "Pranith Kumar. Karampuri" <pranithk at gluster.com>
Cc: gluster-users at gluster.org
Sent: Tuesday, May 17, 2011 3:49:30 PM
Subject: RE: [Gluster-users] Client and server file "view", different results?! Client can't see the right file.
It version 3.1.3 (we tried 3.2.0 for about 10h and rolled back)
Unfortunateltly the file view was "repaired" already by brutally copying
manually from the correct /mnt (server) mountpoint to the /opt (client)
mount which fixed the situation for now. We needed the files accessible
ASAP.
Best, Martin
> -----Original Message-----
> From: Pranith Kumar. Karampuri [mailto:pranithk at gluster.com]
> Sent: Tuesday, May 17, 2011 10:41 AM
> To: Martin Schenker
> Cc: gluster-users at gluster.org
> Subject: Re: [Gluster-users] Client and server file "view",
> different results?! Client can't see the right file.
>
>
> hi Martin,
> Could you please gather the following outputs so that we
> can debug as to what is happening:
> 1) whats the version of the gluster.
> 2) backend "ls -l" of files in question on all bricks that
> file is replicated on.
> 3) 'ls -l" o/p from mnt point for that file.
>
> Thanks
> Pranith
> ----- Original Message -----
> From: "Martin Schenker" <martin.schenker at profitbricks.com>
> To: "Pranith Kumar. Karampuri" <pranithk at gluster.com>
> Cc: gluster-users at gluster.org
> Sent: Tuesday, May 17, 2011 11:13:32 AM
> Subject: RE: [Gluster-users] Client and server file "view",
> different results?! Client can't see the right file.
>
> Yes, it is!
>
> Here's the volfile:
>
> cat /mnt/gluster/brick0/config/vols/storage0/storage0-fuse.vol:
>
> volume storage0-client-0
> type protocol/client
> option remote-host de-dc1-c1-pserver3
> option remote-subvolume /mnt/gluster/brick0/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-1
> type protocol/client
> option remote-host de-dc1-c1-pserver5
> option remote-subvolume /mnt/gluster/brick0/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-2
> type protocol/client
> option remote-host de-dc1-c1-pserver3
> option remote-subvolume /mnt/gluster/brick1/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-3
> type protocol/client
> option remote-host de-dc1-c1-pserver5
> option remote-subvolume /mnt/gluster/brick1/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-4
> type protocol/client
> option remote-host de-dc1-c1-pserver12
> option remote-subvolume /mnt/gluster/brick0/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-5
> type protocol/client
> option remote-host de-dc1-c1-pserver13
> option remote-subvolume /mnt/gluster/brick0/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-6
> type protocol/client
> option remote-host de-dc1-c1-pserver12
> option remote-subvolume /mnt/gluster/brick1/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-client-7
> type protocol/client
> option remote-host de-dc1-c1-pserver13
> option remote-subvolume /mnt/gluster/brick1/storage
> option transport-type rdma
> option ping-timeout 5
> end-volume
>
> volume storage0-replicate-0
> type cluster/replicate
> subvolumes storage0-client-0 storage0-client-1
> end-volume
>
> volume storage0-replicate-1
> type cluster/replicate
> subvolumes storage0-client-2 storage0-client-3
> end-volume
>
> volume storage0-replicate-2
> type cluster/replicate
> subvolumes storage0-client-4 storage0-client-5
> end-volume
>
> volume storage0-replicate-3
> type cluster/replicate
> subvolumes storage0-client-6 storage0-client-7
> end-volume
>
> volume storage0-dht
> type cluster/distribute
> subvolumes storage0-replicate-0 storage0-replicate-1
> storage0-replicate-2 storage0-replicate-3 end-volume
>
> volume storage0-write-behind
> type performance/write-behind
> subvolumes storage0-dht
> end-volume
>
> volume storage0-read-ahead
> type performance/read-ahead
> subvolumes storage0-write-behind
> end-volume
>
> volume storage0-io-cache
> type performance/io-cache
> option cache-size 4096MB
> subvolumes storage0-read-ahead
> end-volume
>
> volume storage0-quick-read
> type performance/quick-read
> option cache-size 4096MB
> subvolumes storage0-io-cache
> end-volume
>
> volume storage0-stat-prefetch
> type performance/stat-prefetch
> subvolumes storage0-quick-read
> end-volume
>
> volume storage0
> type debug/io-stats
> subvolumes storage0-stat-prefetch
> end-volume
>
>
> > -----Original Message-----
> > From: Pranith Kumar. Karampuri [mailto:pranithk at gluster.com]
> > Sent: Tuesday, May 17, 2011 7:16 AM
> > To: Martin Schenker
> > Cc: gluster-users at gluster.org
> > Subject: Re: [Gluster-users] Client and server file "view",
> > different results?! Client can't see the right file.
> >
> >
> > Martin,
> > Is this a distributed-replicate setup?. Could you
> > attach the vol-file of the client.
> >
> > Pranith
> > ----- Original Message -----
> > From: "Martin Schenker" <martin.schenker at profitbricks.com>
> > To: gluster-users at gluster.org
> > Sent: Monday, May 16, 2011 2:49:29 PM
> > Subject: [Gluster-users] Client and server file "view",
> > different results?! Client can't see the right file.
> >
> >
> > Client and server file "view", different results?! Client
> > can't see the right file.
> >
> > Hi all!
> >
> > Here we have another mismatch between the client "view" and
> > the server mounts:
> >
> > From the server site everything seems well, the 20G file is
> > visible and the attributes seem to match:
> >
> > 0 root at pserver5:~ # getfattr -R -d -e hex -m "trusted.afr."
> > /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8
> > f-8542864da6ef/hdd-images/
> >
> > # file:
> > mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f
> > -8542864da6ef/hdd-images//20964
> > trusted.afr.storage0-client-2=0x000000000000000000000000
> > trusted.afr.storage0-client-3=0x000000000000000000000000
> >
> > 0 root at pserver5:~ # find /mnt/gluster/ -name 20964 | xargs -i
> > ls -al {}
> > -rwxrwx--- 1 libvirt-qemu vcb 21474836480 May 13 11:21
> > /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8
> > f-8542864da6ef/hdd-images/20964
> >
> > But the client view shows 2!! files with 0 byte size!! And
> > these aren't any link files created by Gluster. ( with the T
> > on the end)
> >
> > 0 root at pserver5:~ # find /opt/profitbricks/storage/ -name
> > 20964 | xargs -i ls -al {}
> > -rwxrwx--- 1 libvirt-qemu kvm 0 May 13 11:24
> > /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-
> > 8542864da6ef/hdd-images/20964
> >
> > -rwxrwx--- 1 libvirt-qemu kvm 0 May 13 11:24
> > /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-
> > 8542864da6ef/hdd-images/20964
> >
> > I'm a bit stumped that we seem to have so many weird errors
> > cropping up. Any ideas? I've checked the ext4 filesystem on
> > all boxes, no real problems. We run a distributed cluster
> > with 4 servers offering 2 bricks each.
> >
> > Best, Martin
> >
> >
> >
> >
> > > -----Original Message-----
> > > From: Mohit Anchlia [ mailto:mohitanchlia at gmail.com ]
> > > Sent: Monday, May 16, 2011 2:24 AM
> > > To: Martin Schenker
> > > Cc: gluster-users at gluster.org
> > > Subject: Re: [Gluster-users] Brick pair file mismatch,
> > > self-heal problems?
> > >
> > >
> > > Try this to trigger self heal:
> > >
> > > find <gluster-mount> -noleaf -print0 -name <file name>|
> xargs --null
> > > stat >/dev/null
> > >
> > >
> > >
> > > On Sun, May 15, 2011 at 11:20 AM, Martin Schenker
> > > <martin.schenker at profitbricks.com> wrote:
> > > > Can someone enlighten me what's going on here? We have a
> > two peers,
> > > > the file 21313 is shown through the client mountpoint as
> > > "1Jan1970",
> > > > attribs on server pserver3 don't match but NO self-heal or
> > > repair can
> > > > be triggered through "ls -alR"?!?
> > > >
> > > > Checking the files through the server mounts show that
> > two versions
> > > > are on the system. But the wrong one (as with the
> > > "1Jan1970") seems to
> > > > be the preferred one by the client?!?
> > > >
> > > > Do I need to use setattr or what in order to get the client
> > > to see the
> > > > RIGHT version?!? This is not the ONLY file displaying this
> > > problematic
> > > > behaviour!
> > > >
> > > > Thanks for any feedback.
> > > >
> > > > Martin
> > > >
> > > > pserver5:
> > > >
> > > > 0 root at pserver5:~ # ls -al
> > > >
> > >
> >
> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-854286
> > > > 4da6ef
> > > > /hdd-images
> > > >
> > > > -rwxrwx--- 1 libvirt-qemu vcb 483183820800 May 13 13:41 21313
> > > >
> > > > 0 root at pserver5:~ # getfattr -R -d -e hex -m "trusted.afr."
> > > >
> > >
> >
> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-854286
> > > > 4da6ef
> > > > /hdd-images/21313
> > > > getfattr: Removing leading '/' from absolute path names
> > > > # file:
> > > >
> > > mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f
> > > -8542864da6ef/
> > > > hdd-images/21313
> > > > trusted.afr.storage0-client-2=0x000000000000000000000000
> > > > trusted.afr.storage0-client-3=0x000000000000000000000000
> > > >
> > > > 0 root at pserver5:~ # ls -alR
> > > >
> > >
> >
> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864d
> > > > a6ef/h
> > > > dd-images/21313
> > > > -rwxrwx--- 1 libvirt-qemu kvm 483183820800 Jan 1 1970
> > > >
> > > /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-
> > > 8542864da6ef/h
> > > > dd-images/21313
> > > >
> > > > pserver3:
> > > >
> > > > 0 root at pserver3:~ # ls -al
> > > >
> > >
> >
> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-854286
> > > > 4da6ef
> > > > /hdd-images
> > > >
> > > > -rwxrwx--- 1 libvirt-qemu kvm 483183820800 Jan 1 1970 21313
> > > >
> > > > 0 root at pserver3:~ # ls -alR
> > > >
> > >
> >
> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864d
> > > > a6ef/h
> > > > dd-images/21313
> > > > -rwxrwx--- 1 libvirt-qemu kvm 483183820800 Jan 1 1970
> > > >
> > > /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-
> > > 8542864da6ef/h
> > > > dd-images/21313
> > > >
> > > > 0 root at pserver3:~ # getfattr -R -d -e hex -m "trusted.afr."
> > > > /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-
> > > > ad8f-8542864da6ef/hdd-images/21313
> > > > getfattr: Removing leading '/' from absolute path names
> > > > # file:
> > > >
> > >
> >
> mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864
> > > > da6ef/
> > > > hdd-images/21313
> > > > trusted.afr.storage0-client-2=0x000000000000000000000000
> > > > trusted.afr.storage0-client-3=0x0b0000090900000000000000
> > > <- mismatch,
> > > > should be targeted for self-heal/repair? Why is there a
> > > difference in the
> > > > views?
> > > >
> > > >
> > > > From the volfile:
> > > >
> > > > volume storage0-client-2
> > > > type protocol/client
> > > > option remote-host de-dc1-c1-pserver3
> > > > option remote-subvolume /mnt/gluster/brick1/storage
> > > > option transport-type rdma
> > > > option ping-timeout 5
> > > > end-volume
> > > >
> > > > volume storage0-client-3
> > > > type protocol/client
> > > > option remote-host de-dc1-c1-pserver5
> > > > option remote-subvolume /mnt/gluster/brick1/storage
> > > > option transport-type rdma
> > > > option ping-timeout 5
> > > > end-volume
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Gluster-users mailing list
> > > > Gluster-users at gluster.org
> > > > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> > > >
> > >
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> >
>
More information about the Gluster-users
mailing list