[Gluster-devel] OS X client wrong disk size in combination with DHT

Ate Poorthuis atepoorthuis at gmail.com
Fri Jul 3 12:45:57 UTC 2009


Hi,

Using gluster 2.0.2 client os OS X 10.5. Disk size is not outputted
correctly, only the size of the first brick in dht is counted. Used disk
space is calculated correctly.

OS X client: /Users/ate/glusterfs_ip.vol  5.4Ti  1.1Ti  3.2Ti    26%
/mnt/gluster
Debian client: /etc/glusterfs/client_st.vol  22T  1.1T   20T   6%
/mnt/gluster

There are also some errors from dht-diskusage in the log about full disks.
The % differs each time I mount (ranging from 96-100%).

================================================================================
Version      : glusterfs 2.0.2 built on Jun 30 2009 14:14:33
TLA Revision : 07019da2e16534d527215a91904298ede09bb798
Starting Time: 2009-07-03 14:40:34
Command line : /usr/local/gluster/sbin/glusterfs --debug -f
/Users/ate/glusterfs_ip.vol /mnt/gluster/
PID          : 97785
System name  : Darwin
Nodename     : ate-poorthuiss-macbook-2.local
Kernel Release : 9.7.0
Hardware Identifier: i386

Given volfile:
+------------------------------------------------------------------------------+
  1: ### file: client-volume.vol
  2:
  3: #####################################
  4: ###  GlusterFS Client Volume File  ##
  5: #####################################
  6:
  7: #### CONFIG FILE RULES:
  8: ### "#" is comment character.
  9: ### - Config file is case sensitive
 10: ### - Options within a volume block can be in any order.
 11: ### - Spaces or tabs are used as delimitter within a line.
 12: ### - Each option should end within a line.
 13: ### - Missing or commented fields will assume default values.
 14: ### - Blank/commented lines are allowed.
 15: ### - Sub-volumes should already be defined above before referring.
 16:
 17: ### Add client feature and attach to remote subvolume
 18: volume gfs-001-afr1
 19:   type protocol/client
 20:   option transport-type tcp
 21:   option remote-host 10.0.0.30         # IP address of the remote brick
 22:   option remote-subvolume afr1        # name of the remote volume
 23: #  option ping-timeout 5
 24: end-volume
 25: volume gfs-001-afr2
 26:   type protocol/client
 27:   option transport-type tcp
 28:   option remote-host 10.0.0.30         # IP address of the remote brick
 29:   option remote-subvolume afr2        # name of the remote volume
 30: #  option ping-timeout 5
 31: end-volume
 32: volume gfs-002-afr1
 33:   type protocol/client
 34:   option transport-type tcp
 35:   option remote-host 10.0.0.32         # IP address of the remote brick
 36:   option remote-subvolume afr1        # name of the remote volume
 37: #  option ping-timeout 5
 38: end-volume
 39: volume gfs-002-afr2
 40:   type protocol/client
 41:   option transport-type tcp
 42:   option remote-host 10.0.0.32         # IP address of the remote brick
 43:   option remote-subvolume afr2        # name of the remote volume
 44: #  option ping-timeout 5
 45: end-volume
 46:
 47: volume bricks
 48:   type cluster/distribute
 49: #  option lookup-unhashed yes
 50:   option min-free-disk 5%
 51:   subvolumes gfs-001-afr1 gfs-001-afr2 gfs-002-afr1 gfs-002-afr2
 52: end-volume
 53:
 54: ### Add readahead feature
 55: volume readahead
 56:   type performance/read-ahead
 57:   option page-count 8       # cache per file  = (page-count x
page-size)
 58:   subvolumes bricks
 59: end-volume
 60:
 61: ### Add IO-Cache feature
 62: volume iocache
 63:   type performance/io-cache
 64:   option cache-size 256MB
 65:   subvolumes readahead
 66: end-volume
 67: volume iothreads
 68:   type performance/io-threads
 69:   option thread-count 32  # default is 1
 70:   subvolumes iocache
 71: end-volume
 72: ### Add writeback feature
 73: volume writeback
 74:   type performance/write-behind
 75:   option cache-size 2MB
 76:   option flush-behind off
 77:   subvolumes iothreads
 78: end-volume
 79:

+------------------------------------------------------------------------------+
[2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify)
on dlsym(0x101730, notify): symbol not found -- neglecting
[2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify)
on dlsym(0x101a40, notify): symbol not found -- neglecting
[2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify)
on dlsym(0x101d50, notify): symbol not found -- neglecting
[2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify)
on dlsym(0x102060, notify): symbol not found -- neglecting
[2009-07-03 14:40:34] D [glusterfsd.c:256:_add_fuse_mount] glusterfs:
'direct-io-mode' in fuse causes data corruption if O_APPEND is used.
disabling 'direct-io-mode'
[2009-07-03 14:40:34] D [glusterfsd.c:1179:main] glusterfs: running in pid
97785
[2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-001-afr1:
defaulting frame-timeout to 30mins
[2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-001-afr1:
defaulting ping-timeout to 10
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-001-afr2:
defaulting frame-timeout to 30mins
[2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-001-afr2:
defaulting ping-timeout to 10
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-002-afr1:
defaulting frame-timeout to 30mins
[2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-002-afr1:
defaulting ping-timeout to 10
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-002-afr2:
defaulting frame-timeout to 30mins
[2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-002-afr2:
defaulting ping-timeout to 10
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt
to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [read-ahead.c:786:init] readahead: Using
conf->page_count = 8
[2009-07-03 14:40:34] D [io-threads.c:2280:init] iothreads: io-threads:
Autoscaling: off, min_threads: 32, max_threads: 32
[2009-07-03 14:40:34] D [write-behind.c:1859:init] writeback: disabling
write-behind for first 1 bytes
[2009-07-03 14:40:34] D [dict.c:297:dict_get] dict: @this=0x0 @key=0x22dd0b
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got
GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] N [glusterfsd.c:1198:main] glusterfs: Successfully
started
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr1: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr1: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr2: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr2: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr1: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr1: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr2: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr2: got
GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-001-afr1: Connected to 10.0.0.30:6996, attached to remote volume 'afr1'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-001-afr1: Connected to 10.0.0.30:6996, attached to remote volume 'afr1'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-001-afr2: Connected to 10.0.0.30:6996, attached to remote volume 'afr2'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-002-afr1: Connected to 10.0.0.32:6996, attached to remote volume 'afr1'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-002-afr1: Connected to 10.0.0.32:6996, attached to remote volume 'afr1'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-002-afr2: Connected to 10.0.0.32:6996, attached to remote volume 'afr2'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-002-afr2: Connected to 10.0.0.32:6996, attached to remote volume 'afr2'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk]
gfs-001-afr2: Connected to 10.0.0.30:6996, attached to remote volume 'afr2'.
[2009-07-03 14:40:35] D [dht-common.c:1405:dht_err_cbk] bricks: subvolume
gfs-001-afr1 returned -1 (Invalid argument)
[2009-07-03 14:40:51] C [dht-diskusage.c:197:dht_is_subvol_filled] bricks:
disk space on subvolume 'gfs-001-afr1' is getting full (97.00 %), consider
adding more nodes
[2009-07-03 14:40:51] W [dht-diskusage.c:231:dht_free_disk_available_subvol]
bricks: No subvolume has enough free space to create
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20090703/f110028d/attachment-0003.html>


More information about the Gluster-devel mailing list