[Gluster-devel] unify: other clients cannot read files

Thu Feb 14 11:56:38 UTC 2008

David,
I might se a problem in your config. In the 'volume server' you define
subvolumes twice, in two lines. Try in one line - like:
'subvolumes storage manespace'
I'm running that without problems.

Cheers,
Einar

On Thu, 2008-02-14 at 11:39 +0100, David Mayr - Hetzner Online AG wrote:
> Hi all,
> 
> I got the solution to the problem and am a bit confused.
> 
> The working config can be found at  http://gluster.pastebin.org/19526
> I'm now using srv9 only as namespace, not as storage any more.
> 
> I used one of the bricks (srv9) as storage brick *and* as namespace brick 
> before. After having created a file from a client, I noticed that the file 
> was on one of the storage nodes and in the *storage* directory of the 
> namespace node (srv9) with zero bytes (and not in the namespace directory).
> 
> I seems that it is not possible any more to define / use more than one volume 
> per server or rather it works not as expected.
> 
> David
> 
> 
> Am Freitag 08 Februar 2008 13:12:25 schrieb David Mayr - Hetzner Online AG:
> > Hi Avati,
> >
> > >  Can you upgrade to patch-650 and start with an empty backend? Suspecting
> > > that you were effected by the bug ~640 which might have created 0 byte
> > > files in the backend (which can cause such a behaviour). We are curious
> > > to know if patch-650 (with an empty backend) works for you..
> >
> > I just updated to patch-650 on all servers and clients, removed all my
> > storage and namespace dirs on the servers, restarted servers and clients
> > (which recreated the storage/namespace dirs automatically) and
> > unfortunately have still the same problem:
> > files created by client-1 can not be read by client-2.
> >
> > client-2 logfile (level 'warning') says now while trying to read a file
> > created by client-1:
> >
> > E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 76: /testfile => -1
> > (116) E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 77: /testfile =>
> > -1 (116) E [unify.c:849:unify_open] unify: /testfile: entry_count is 3
> > E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 78: /testfile => -1 (5)
> >
> > David
> >
> > > 2008/2/8, David Mayr - Hetzner Online AG <david.mayr at hetzner.de>:
> > > > Hi Avati,
> > > >
> > > > > Is the client spec file same on both clients?
> > > >
> > > > Yes, the client specs are the same on both clients.
> > > >
> > > > > can you show the client log of client-2?
> > > >
> > > > with loglevel DEBUG I see the following on client-2 while reading (with
> > > > dd) a
> > > > file written by client-1 (timestamps removed):
> > > >
> > > > D [fuse-bridge.c:492:fuse_lookup] glusterfs-fuse: 2: LOOKUP /testfile-2
> > > > D [fuse-bridge.c:354:fuse_entry_cbk] glusterfs-fuse: 2: /testfile-2 =>
> > > > 1972
> > > > D [inode.c:559:__create_inode] fuse/inode: create inode(1972)
> > > > D [inode.c:351:__active_inode] fuse/inode: activating inode(1972),
> > > > lru=0/1024
> > > > D [inode.c:381:__passive_inode] fuse/inode: passivating inode(1972),
> > > > lru=1/1024
> > > > D [inode.c:308:__destroy_inode] fuse/inode: destroy inode(0)
> > > > [@0x80a5720] D [inode.c:351:__active_inode] fuse/inode: activating
> > > > inode(1972), lru=0/1024
> > > > D [fuse-bridge.c:1470:fuse_open] glusterfs-fuse: 3: OPEN /testfile-2
> > > > E [unify.c:875:unify_open] unify: /testfile-2: entry_count is 3
> > > > E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 3: /testfile-2 => -1
> > > > (5) D [inode.c:381:__passive_inode] fuse/inode: passivating
> > > > inode(1972), lru=1/1024
> > > >
> > > >
> > > > when I only do an ls /glusterfs/filename I see the following on
> > > > client-2:
> > > >
> > > > D [inode.c:351:__active_inode] fuse/inode: activating inode(1972),
> > > > lru=0/1024
> > > > D [fuse-bridge.c:500:fuse_lookup] glusterfs-fuse: 4: LOOKUP
> > > > /testfile-2(1972)
> > > > E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 4: /testfile-2 =>
> > > > -1 (116)
> > > > D [fuse-bridge.c:437:fuse_entry_cbk] glusterfs-fuse: unlinking stale
> > > > dentry
> > > > for `/testfile-2'
> > > > D [fuse-bridge.c:354:fuse_entry_cbk] glusterfs-fuse: 4: /testfile-2 =>
> > > > 1972
> > > > D [inode.c:308:__destroy_inode] fuse/inode: destroy inode(0)
> > > > [@0xb6b05248] D [inode.c:381:__passive_inode] fuse/inode: passivating
> > > > inode(1972), lru=1/1024
> > > >
> > > > > also which version of glusterfs/fuse are you using?
> > > >
> > > > - debian etch
> > > > - kernel 2.6.18-5-686-bigmem
> > > > - glusterfs tla 643  (debian packages from
> > > > http://projects.nlogy.cz/debian )
> > > > - fuse-2.7.2glfs8 from
> > > >
> > > > http://ftp.zresearch.com/pub/gluster/glusterfs/fuse/fuse-2.7.2glfs8.tar
> > > >.g z
> > > >
> > > >
> > > > David
> > > >
> > > > > 2008/1/29, David Mayr - Hetzner Online AG <david.mayr at hetzner.de>:
> > > > > > Hi Avati,
> > > > > >
> > > > > > > > Hi GlusterFS-Team,
> > > > > > > >
> > > > > > > > first thank you for this great piece of software!
> > > > > > > >
> > > > > > > > I try glusterfs for testing at the moment on 9 servers and two
> > > > > > > > clients with
> > > > > > > > unify - for config see
> > > > > > > > http://glusterfs.pastebin.com/m7a74bd29(client) and
> > > > > > > > http://glusterfs.pastebin.com/m206785bc (server).
> > > > > > > >
> > > > > > > > I'm using TLA 643 of the packages from
> > > > > > > > http://projects.nlogy.cz/debian (camel1cz builds them nightly),
> > > >
> > > > but
> > > >
> > > > > > > > saw the following problem already last week with earlier patch
> > > > > > > > versions.
> > > > > > > >
> > > > > > > > When I write files with client-1, I can stat and read these
> > > > > > > > files from client-1 without problems. But I get wrong results
> > > > > > > > doing an
> > > >
> > > > "ls
> > > >
> > > > > > > > -l"
> > > > > >
> > > > > > (zero
> > > > > >
> > > > > > > > bytes) on client-2 and cannot read the file on client-2.
> > > > > > > >
> > > > > > > > See the following example:
> > > > > > > >
> > > > > > > >
> > > > > > > > ===> on client-1:
> > > > > > > >
> > > > > > > > root at client-1 ~ # dd if=/dev/zero of=/glusterfs/testfile bs=1M
> > > > > >
> > > > > > count=100
> > > > > >
> > > > > > > > 104857600 bytes (105 MB) copied, 1.9773 seconds, 53.0 MB/s
> > > > > > > >
> > > > > > > > root at client-1 ~ # ls -lah /glusterfs/testfile
> > > > > > > > -rw-r--r-- 1 root root 100M 2008-01-28 12:48
> > > > > > > > /glusterfs/testfile
> > > > > > > >
> > > > > > > > root at client-1 ~ # dd if=/glusterfs/testfile of=/dev/null
> > > > > > > > 104857600 bytes (105 MB) copied, 2.55219 seconds, 41.1 MB/s
> > > > > > > >
> > > > > > > >
> > > > > > > > ===> on client-2:
> > > > > > > >
> > > > > > > > root at client-2 ~ # ls -lah /glusterfs/testfile
> > > > > > > > -rw-r--r-- 1 root root 0 2008-01-28 12:47 /glusterfs/testfile
> > > > > > > > (=> zero byte size)
> > > > > > > >
> > > > > > > > root at client-2 ~ # dd if=/glusterfs/testfile of=/dev/null
> > > > > > > > dd: opening `/glusterfs/testfile': Input/output error
> > > > > > > >
> > > > > > > > David
> > > > > > >
> > > > > > > In your client spec I do not see 'ns9' being declared as a
> > > > > >
> > > > > > protocol/client.
> > > > > >
> > > > > > > Please declare the global namespace as it is necssary for shared
> > > > > > > access. avati
> > > > > >
> > > > > > Oops, I'm sorry - forgot that while cleaning up (removing comments
> > > > > > and unnecessary) my config files.
> > > > > > In my original files the ns9-entry exists.
> > > > > >
> > > > > > I re-posted my config files here:
> > > > > > server:   http://glusterfs.pastebin.com/m96b2583
> > > > > > client:    http://glusterfs.pastebin.com/m6c8472aa
> > > > > >
> > > > > > Any ideas why clients cannot read files of other clients?
> > > > > >
> > > > > > --
> > > > > > David
> > > >
> > > > _______________________________________________
> > > > Gluster-devel mailing list
> > > > Gluster-devel at nongnu.org
> > > > http://lists.nongnu.org/mailman/listinfo/gluster-devel
> 
> 
> 
-- 
Einar Gautun                     einar.gautun at statkart.no

Statens kartverk            | Norwegian Mapping Authority
3507 Hønefoss               |    NO-3507 Hønefoss, Norway

Ph +47 32118372   Fax +47 32118101       Mob +47 92692662