[Gluster-users] starting 4th node in 4 node dht cluster fails

[Amar Tumballi (bulde)]

Hi Craig,
 As you are using 'distribute' (client side) over 'distribute' (server
side), this will not be working right now. To get it working right now, you
can have 4 export volumes from each server exported, and in client have 4x4
client protocol volumes, which you can aggregate with a single
'cluster/distribute' (which will have 16 subvolumes).

To get the below mentioned configuration working as is, you need to wait for
a week more IMO.

Regards,
Amar

2009/2/6 Craig Flockhart <craigflockhart at yahoo.com>

> Using dht translator to cluster together 4 nodes each with 4 disks.
> Starting glusterfs on the 4th causes "Structure needs cleaning" when ls-ing
> the mount point on any of them. It's fine with 3 nodes started.
> Using fuse-2.7.4
> GlusterFS 2.0.0rc1
> Linux 2.6.18-53.el5 kernel
>
> Errors from the log:
>
>
> 2009-02-06 15:23:51 E [dht-layout.c:460:dht_layout_normalize] dist1: found
> anomalies in /. holes=1 overlaps=3
> 2009-02-06 15:23:51 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing
> assignment on /
> 2009-02-06 15:23:51 E [dht-selfheal.c:422:dht_selfheal_directory] dist1:
> the directory is not a virgin
> 2009-02-06 15:23:51 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge:
> revalidate of / failed (Structure needs cleaning)
> 2009-02-06 15:23:51 E [dht-layout.c:460:dht_layout_normalize] dist1: found
> anomalies in /. holes=1 overlaps=3
> 2009-02-06 15:23:51 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing
> assignment on /
> 2009-02-06 15:23:51 E [dht-selfheal.c:422:dht_selfheal_directory] dist1:
> the directory is not a virgin
> 2009-02-06 15:23:51 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 2:
> LOOKUP() / => -1 (Structure needs cleaning)
>
> Config for one of the machines:
>
> volume posix-d1
>  type storage/posix
>  option directory /mnt/chard1/export
> end-volume
>
> volume locks1
>   type features/locks
>   subvolumes posix-d1
> end-volume
>
>
> volume posix-d2
>  type storage/posix
>  option directory /mnt/chard2/export
> end-volume
>
>
> volume locks2
>   type features/locks
>   subvolumes posix-d2
> end-volume
>
>
> volume posix-d3
>  type storage/posix
>  option directory /mnt/chard3/export
> end-volume
>
> volume locks3
>   type features/locks
>   subvolumes posix-d3
> end-volume
>
>
> volume posix-d4
>  type storage/posix
>  option directory /mnt/chard4/export
> end-volume
>
> volume locks4
>   type features/locks
>   subvolumes posix-d4
> end-volume
>
> volume home-ns
>  type storage/posix
>  option directory /var/local/glusterfs/namespace1
> end-volume
>
> volume home
>  type cluster/distribute
>  subvolumes locks1 locks2 locks3 locks4
> end-volume
>
> volume server
>  type protocol/server
>  option transport-type tcp
>  subvolumes home
>  option auth.addr.home.allow *
> end-volume
>
>
> volume zwei
>       type protocol/client
>       option transport-type tcp
>       option remote-host zwei
>       option remote-subvolume home
> end-volume
>
> volume char
>       type protocol/client
>       option transport-type tcp
>       option remote-host char
>       option remote-subvolume home
> end-volume
>
> volume pente
>       type protocol/client
>       option transport-type tcp
>       option remote-host pente
>       option remote-subvolume home
> end-volume
>
> volume tres
>       type protocol/client
>       option transport-type tcp
>       option remote-host tres
>       option remote-subvolume home
> end-volume
>
> volume dist1
>     type cluster/distribute
>     subvolumes pente char tres zwei
> end-volume
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
>


-- 
Amar Tumballi
Gluster/GlusterFS Hacker
[bulde on #gluster/irc.gnu.org]
http://www.zresearch.com - Commoditizing Super Storage!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090206/ef563d87/attachment.html>