[Gluster-users] Fwd: to mailing list gluster.org

Pranith Kumar Karampuri pkarampu at redhat.com
Wed Jan 28 06:43:41 UTC 2015


Vijaikumar, Sachin will work on this quota issue.

Pranith
On 01/22/2015 12:46 AM, Pranith Kumar Karampuri wrote:
> + quota devs
>
> Pranith
> On 01/07/2015 07:37 PM, Bekk, Klaus (IKP) wrote:
>>
>>> Hi all,
>>>
>>> we are running 4 server with Ubuntu 14.04 as glusterfs server with 
>>> 3.5.2-ubuntu1~trusty1.
>>>
>>
>>> When starting quotas on any of the volumes the brick will be not 
>>> available after some time. So we have to disable quota on this 
>>> volume and stop and start this volume.
>>>
>>> We have the following configuration
>>>
>>> root at ikpsrv01:/var/log/glusterfs# gluster volume info
>>>
>>> Volume Name: gltestn
>>>
>>> Type: Distribute
>>>
>>> Volume ID: e50bd30d-dbc8-4279-ac9d-2e557328a643
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 1
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv01:/gltest2/gl
>>>
>>> Volume Name: glwww1
>>>
>>> Type: Replicate
>>>
>>> Volume ID: 811e4c2a-4520-4e61-bb84-955afbdb9dff
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 1 x 2 = 2
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv03:/glwwwn/gl
>>>
>>> Brick2: ikpsrv01:/glwwwn2/gl
>>>
>>> Options Reconfigured:
>>>
>>> features.quota: on
>>>
>>> Volume Name: corsika
>>>
>>> Type: Distribute
>>>
>>> Volume ID: d6e07f3e-4b35-4736-9b71-afc3224f0f29
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 4
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv03:/glcors3-1/gl
>>>
>>> Brick2: ikpsrv03:/glcors3-2/gl
>>>
>>> Brick3: ikpsrv02:/glcors2-1/gl
>>>
>>> Brick4: ikpsrv02:/glcors2-2/gl
>>>
>>> Volume Name: gldata
>>>
>>> Type: Distribute
>>>
>>> Volume ID: 82485d03-9ef4-4151-a2cc-881072e871a1
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 2
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv02:/gldata2-1/gl
>>>
>>> Brick2: ikpsrv01:/gldata1-2/gl
>>>
>>> Options Reconfigured:
>>>
>>> features.quota: on
>>>
>>> Volume Name: gltest
>>>
>>> Type: Distribute
>>>
>>> Volume ID: 9e69afcc-d421-41c9-824b-7d9a31240072
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 2
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv01:/gltest3/gl
>>>
>>> Brick2: ikpsrv01:/gltest4/gl
>>>
>>> Options Reconfigured:
>>>
>>> features.quota: off
>>>
>>> Volume Name: glusersold
>>>
>>> Type: Distribute
>>>
>>> Volume ID: 91a79e1a-4ff3-4191-b0ae-73357105774c
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 1
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv02:/glusers2-2/gl
>>>
>>> Volume Name: glusers
>>>
>>> Type: Distribute
>>>
>>> Volume ID: f48b1941-9c19-4311-bd9e-497660fbcc80
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 1
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: ikpsrv02:/glusers2-1/gl
>>>
>>> Options Reconfigured:
>>>
>>> diagnostics.client-log-level: CRITICAL
>>>
>>> diagnostics.brick-log-level: CRITICAL
>>>
>>> features.quota: off
>>>
>>> When starting quotas on any of the volumes the brick will be not 
>>> available after some time. So we have to disable quota on this 
>>> volume and stop and start this volume.
>>>
>>> The command
>>>
>>> Gluster volume quota gltestn enable
>>>
>>> gives the following output in quotad.log
>>>
>>> [2015-01-07 12:01:55.522229] W [glusterfsd.c:1095:cleanup_and_exit] 
>>> (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb455bd9fbd] 
>>> (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb455ead182] 
>>> (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5) 
>>> [0x7fb4569a7265]))) 0-: received signum (15), shutting down
>>>
>>> [2015-01-07 12:01:56.534931] I [glusterfsd.c:1959:main] 
>>> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 
>>> 3.5.2 (/usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad 
>>> -p /var/lib/glusterd/quotad/run/quotad.pid -l 
>>> /var/log/glusterfs/quotad.log -S 
>>> /var/run/fa869d5552bf340ec7506747e21b2841.socket --xlator-option 
>>> *replicate*.data-self-heal=off --xlator-option 
>>> *replicate*.metadata-self-heal=off --xlator-option 
>>> *replicate*.entry-self-heal=off)
>>>
>>> [2015-01-07 12:01:56.537061] I [socket.c:3561:socket_init] 
>>> 0-socket.glusterfsd: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.537134] I [socket.c:3576:socket_init] 
>>> 0-socket.glusterfsd: using system polling thread
>>>
>>> [2015-01-07 12:01:56.537355] I [socket.c:3561:socket_init] 
>>> 0-glusterfs: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.537383] I [socket.c:3576:socket_init] 
>>> 0-glusterfs: using system polling thread
>>>
>>> [2015-01-07 12:01:56.541480] I [graph.c:254:gf_add_cmdline_options] 
>>> 0-glwww1-replicate-0: adding option 'entry-self-heal' for volume 
>>> 'glwww1-replicate-0' with value 'off'
>>>
>>> [2015-01-07 12:01:56.541516] I [graph.c:254:gf_add_cmdline_options] 
>>> 0-glwww1-replicate-0: adding option 'metadata-self-heal' for volume 
>>> 'glwww1-replicate-0' with value 'off'
>>>
>>> [2015-01-07 12:01:56.541536] I [graph.c:254:gf_add_cmdline_options] 
>>> 0-glwww1-replicate-0: adding option 'data-self-heal' for volume 
>>> 'glwww1-replicate-0' with value 'off'
>>>
>>> [2015-01-07 12:01:56.543122] I [socket.c:3561:socket_init] 
>>> 0-socket.quotad: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.543157] I [socket.c:3576:socket_init] 
>>> 0-socket.quotad: using system polling thread
>>>
>>> [2015-01-07 12:01:56.543404] I [dht-shared.c:311:dht_init_regex] 
>>> 0-glwww1: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$
>>>
>>> [2015-01-07 12:01:56.547124] I [socket.c:3561:socket_init] 
>>> 0-glwww1-client-1: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.547158] I [socket.c:3576:socket_init] 
>>> 0-glwww1-client-1: using system polling thread
>>>
>>> [2015-01-07 12:01:56.547728] I [socket.c:3561:socket_init] 
>>> 0-glwww1-client-0: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.547754] I [socket.c:3576:socket_init] 
>>> 0-glwww1-client-0: using system polling thread
>>>
>>> [2015-01-07 12:01:56.547820] I [dht-shared.c:311:dht_init_regex] 
>>> 0-gltestn: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$
>>>
>>> [2015-01-07 12:01:56.549471] I [socket.c:3561:socket_init] 
>>> 0-gltestn-client-0: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.549501] I [socket.c:3576:socket_init] 
>>> 0-gltestn-client-0: using system polling thread
>>>
>>> [2015-01-07 12:01:56.549565] I [dht-shared.c:311:dht_init_regex] 
>>> 0-gldata: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$
>>>
>>> [2015-01-07 12:01:56.551155] I [socket.c:3561:socket_init] 
>>> 0-gldata-client-1: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.551184] I [socket.c:3576:socket_init] 
>>> 0-gldata-client-1: using system polling thread
>>>
>>> [2015-01-07 12:01:56.551744] I [socket.c:3561:socket_init] 
>>> 0-gldata-client-0: SSL support is NOT enabled
>>>
>>> [2015-01-07 12:01:56.551770] I [socket.c:3576:socket_init] 
>>> 0-gldata-client-0: using system polling thread
>>>
>>> [2015-01-07 12:01:56.551790] W [graph.c:329:_log_if_unknown_option] 
>>> 0-quotad: option 'rpc-auth.auth-glusterfs' is not recognized
>>>
>>> [2015-01-07 12:01:56.551819] W [graph.c:329:_log_if_unknown_option] 
>>> 0-quotad: option 'rpc-auth.auth-unix' is not recognized
>>>
>>> [2015-01-07 12:01:56.551838] W [graph.c:329:_log_if_unknown_option] 
>>> 0-quotad: option 'rpc-auth.auth-null' is not recognized
>>>
>>> [2015-01-07 12:01:56.551854] W [graph.c:329:_log_if_unknown_option] 
>>> 0-quotad: option 'glwww1.volume-id' is not recognized
>>>
>>> [2015-01-07 12:01:56.551870] W [graph.c:329:_log_if_unknown_option] 
>>> 0-quotad: option 'gltestn.volume-id' is not recognized
>>>
>>> [2015-01-07 12:01:56.551885] W [graph.c:329:_log_if_unknown_option] 
>>> 0-quotad: option 'gldata.volume-id' is not recognized
>>>
>>> [2015-01-07 12:01:56.551922] I [client.c:2294:notify] 
>>> 0-glwww1-client-0: parent translators are ready, attempting connect 
>>> on transport
>>>
>>> [2015-01-07 12:01:56.552579] I [client.c:2294:notify] 
>>> 0-glwww1-client-1: parent translators are ready, attempting connect 
>>> on transport
>>>
>>> [2015-01-07 12:01:56.553179] I [client.c:2294:notify] 
>>> 0-gltestn-client-0: parent translators are ready, attempting connect 
>>> on transport
>>>
>>> [2015-01-07 12:01:56.553757] I [client.c:2294:notify] 
>>> 0-gldata-client-0: parent translators are ready, attempting connect 
>>> on transport
>>>
>>> [2015-01-07 12:01:56.554372] I [client.c:2294:notify] 
>>> 0-gldata-client-1: parent translators are ready, attempting connect 
>>> on transport
>>>
>>> Final graph:
>>>
>>> +------------------------------------------------------------------------------+
>>>
>>>   1: volume glwww1-client-0
>>>
>>>   2:     type protocol/client
>>>
>>>   3:     option remote-host ikpsrv03
>>>
>>>   4:     option remote-subvolume /glwwwn/gl
>>>
>>>   5:     option transport-type socket
>>>
>>> 6: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481
>>>
>>> 7:     option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f
>>>
>>>   8: end-volume
>>>
>>>   9:
>>>
>>> 10: volume glwww1-client-1
>>>
>>> 11:     type protocol/client
>>>
>>> 12:     option remote-host ikpsrv01
>>>
>>> 13:     option remote-subvolume /glwwwn2/gl
>>>
>>> 14:     option transport-type socket
>>>
>>> 15: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481
>>>
>>> 16:     option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f
>>>
>>> 17: end-volume
>>>
>>> 18:
>>>
>>> 19: volume glwww1-replicate-0
>>>
>>> 20:     type cluster/replicate
>>>
>>> 21:     option data-self-heal off
>>>
>>> 22:     option metadata-self-heal off
>>>
>>> 23:     option entry-self-heal off
>>>
>>> 24: subvolumes glwww1-client-0 glwww1-client-1
>>>
>>> 25: end-volume
>>>
>>> 26:
>>>
>>> 27: volume glwww1
>>>
>>> 28:     type cluster/distribute
>>>
>>> 29: subvolumes glwww1-replicate-0
>>>
>>> 30: end-volume
>>>
>>> 31:
>>>
>>> 32: volume gltestn-client-0
>>>
>>> 33:     type protocol/client
>>>
>>> 34:     option remote-host ikpsrv01
>>>
>>> 35:     option remote-subvolume /gltest2/gl
>>>
>>> 36:     option transport-type socket
>>>
>>> 37:     option username 76c8bf85-a5c5-445d-8e04-e596b6eb173b
>>>
>>> 38:     option password ceec81fd-5afd-4d6b-a413-00b2688f4c62
>>>
>>> 39: end-volume
>>>
>>> 40:
>>>
>>> 41: volume gltestn
>>>
>>> 42:     type cluster/distribute
>>>
>>> 43: subvolumes gltestn-client-0
>>>
>>> 44: end-volume
>>>
>>> 45:
>>>
>>> 46: volume gldata-client-0
>>>
>>> 47:     type protocol/client
>>>
>>> 48:     option remote-host ikpsrv02
>>>
>>> 49:     option remote-subvolume /gldata2-1/gl
>>>
>>> 50:     option transport-type socket
>>>
>>> 51: option username e9dcdbec-1110-4ab2-b667-63625126b246
>>>
>>> 52:     option password 05a9e37d-67a0-476b-88df-93d612a4e08e
>>>
>>> 53: end-volume
>>>
>>> 54:
>>>
>>> 55: volume gldata-client-1
>>>
>>> 56:     type protocol/client
>>>
>>> 57:     option remote-host ikpsrv01
>>>
>>> 58:     option remote-subvolume /gldata1-2/gl
>>>
>>> 59:     option transport-type socket
>>>
>>> 60: option username e9dcdbec-1110-4ab2-b667-63625126b246
>>>
>>> 61:     option password 05a9e37d-67a0-476b-88df-93d612a4e08e
>>>
>>> 62: end-volume
>>>
>>> 63:
>>>
>>> 64: volume gldata
>>>
>>> 65:     type cluster/distribute
>>>
>>> 66: subvolumes gldata-client-0 gldata-client-1
>>>
>>> 67: end-volume
>>>
>>> 68:
>>>
>>> 69: volume quotad
>>>
>>> 70:     type features/quotad
>>>
>>> 71:     option rpc-auth.auth-glusterfs on
>>>
>>> 72:     option rpc-auth.auth-unix on
>>>
>>> 73:     option rpc-auth.auth-null on
>>>
>>> 74:     option transport.socket.listen-path /tmp/quotad.socket
>>>
>>> 75:     option transport-type socket
>>>
>>> 76:     option transport.address-family unix
>>>
>>> 77:     option glwww1.volume-id glwww1
>>>
>>> 78:     option gltestn.volume-id gltestn
>>>
>>> 79:     option gldata.volume-id gldata
>>>
>>> 80: subvolumes glwww1 gltestn gldata
>>>
>>> 81: end-volume
>>>
>>> 82:
>>>
>>> +------------------------------------------------------------------------------+
>>>
>>> [2015-01-07 12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 
>>> 0-gldata-client-0: changing port to 49160 (from 0)
>>>
>>> [2015-01-07 12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 
>>> 0-glwww1-client-1: changing port to 49165 (from 0)
>>>
>>> [2015-01-07 12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 
>>> 0-gltestn-client-0: changing port to 49166 (from 0)
>>>
>>> [2015-01-07 12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 
>>> 0-gldata-client-1: changing port to 49178 (from 0)
>>>
>>> [2015-01-07 12:01:56.558382] W 
>>> [client-callback.c:31:client_cbk_fetchspec] 0-glwww1-client-0: this 
>>> function should not be called
>>>
>>> [2015-01-07 12:01:56.558484] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 
>>> 0-mgmt: Volume file changed
>>>
>>> [2015-01-07 12:01:56.558601] I 
>>> [client-handshake.c:1677:select_server_supported_programs] 
>>> 0-gldata-client-0: Using Program GlusterFS 3.3, Num (1298437), 
>>> Version (330)
>>>
>>> [2015-01-07 12:01:56.559594] I 
>>> [client-handshake.c:1677:select_server_supported_programs] 
>>> 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), 
>>> Version (330)
>>>
>>> [2015-01-07 12:01:56.559686] I 
>>> [client-handshake.c:1677:select_server_supported_programs] 
>>> 0-glwww1-client-1: Using Program GlusterFS 3.3, Num (1298437), 
>>> Version (330)
>>>
>>> [2015-01-07 12:01:56.559768] I 
>>> [client-handshake.c:1677:select_server_supported_programs] 
>>> 0-gldata-client-1: Using Program GlusterFS 3.3, Num (1298437), 
>>> Version (330)
>>>
>>> [2015-01-07 12:01:56.559852] I 
>>> [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-0: 
>>> Connected to 141.52.67.86:49160, attached to remote volume 
>>> '/gldata2-1/gl'.
>>>
>>> [2015-01-07 12:01:56.559873] I 
>>> [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-0: 
>>> Server and Client lk-version numbers are not same, reopening the fds
>>>
>>> [2015-01-07 12:01:56.559996] I 
>>> [client-handshake.c:1462:client_setvolume_cbk] 0-gltestn-client-0: 
>>> Connected to 141.52.67.85:49166, attached to remote volume 
>>> '/gltest2/gl'.
>>>
>>> [2015-01-07 12:01:56.560017] I 
>>> [client-handshake.c:1474:client_setvolume_cbk] 0-gltestn-client-0: 
>>> Server and Client lk-version numbers are not same, reopening the fds
>>>
>>> [2015-01-07 12:01:56.560089] I 
>>> [client-handshake.c:450:client_set_lk_version_cbk] 
>>> 0-gldata-client-0: Server lk version = 1
>>>
>>> [2015-01-07 12:01:56.560139] I 
>>> [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-1: 
>>> Connected to 141.52.67.85:49165, attached to remote volume 
>>> '/glwwwn2/gl'.
>>>
>>> [2015-01-07 12:01:56.560158] I 
>>> [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-1: 
>>> Server and Client lk-version numbers are not same, reopening the fds
>>>
>>> [2015-01-07 12:01:56.560204] I [afr-common.c:4131:afr_notify] 
>>> 0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came back up; 
>>> going online.
>>>
>>> [2015-01-07 12:01:56.560262] I 
>>> [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-1: 
>>> Connected to 141.52.67.85:49178, attached to remote volume 
>>> '/gldata1-2/gl'.
>>>
>>> [2015-01-07 12:01:56.560301] I 
>>> [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-1: 
>>> Server and Client lk-version numbers are not same, reopening the fds
>>>
>>> [2015-01-07 12:01:56.560385] I 
>>> [client-handshake.c:450:client_set_lk_version_cbk] 
>>> 0-gltestn-client-0: Server lk version = 1
>>>
>>> [2015-01-07 12:01:56.560467] I 
>>> [client-handshake.c:450:client_set_lk_version_cbk] 
>>> 0-glwww1-client-1: Server lk version = 1
>>>
>>> [2015-01-07 12:01:56.560546] I 
>>> [client-handshake.c:450:client_set_lk_version_cbk] 
>>> 0-gldata-client-1: Server lk version = 1
>>>
>>> [2015-01-07 12:01:57.255911] I 
>>> [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs: No change in 
>>> volfile, continuing
>>>
>>> [2015-01-07 12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 
>>> 0-glwww1-client-0: changing port to 49153 (from 0)
>>>
>>> [2015-01-07 12:01:57.557924] I 
>>> [client-handshake.c:1677:select_server_supported_programs] 
>>> 0-glwww1-client-0: Using Program GlusterFS 3.3, Num (1298437), 
>>> Version (330)
>>>
>>> [2015-01-07 12:01:57.558360] I 
>>> [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-0: 
>>> Connected to 141.52.67.96:49153, attached to remote volume '/glwwwn/gl'.
>>>
>>> [2015-01-07 12:01:57.558405] I 
>>> [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-0: 
>>> Server and Client lk-version numbers are not same, reopening the fds
>>>
>>> [2015-01-07 12:01:57.558706] I 
>>> [client-handshake.c:450:client_set_lk_version_cbk] 
>>> 0-glwww1-client-0: Server lk version = 1
>>>
>>> what means
>>>
>>> 2015-01-07 12:01:56.559594] I 
>>> [client-handshake.c:1677:select_server_supported_programs] 
>>> 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), 
>>> Version (330)
>>>
>>> In the log output.
>>>
>>> Our glusterfs clients all have 3.5.2 glusterfs version.
>>>
>>> Can someone help?
>>>
>>> kind regards
>>>
>>> Klaus Bekk
>>>
>>> ----------------------------------------------------------
>>>
>>> Karlsruher Institut of Technologie (KIT)
>>>
>>> Institut für Kernphysik (IKP)
>>>
>>> Dr. Klaus Bekk
>>>
>>> Hermann-von-Helmholtz-Platz 1
>>>
>>> Campus Nord Gebäude 425
>>>
>>> 76344 Eggenstein-Leopoldshafen
>>>
>>> Telefon: +49 721 608-23382
>>>
>>> Fax: +49 721 608-23321
>>>
>>> E-Mail: Klaus.Bekk at kit.edu <mailto:Klaus.Bekk at kit.edu>
>>>
>>> http://www.kit.edu/
>>>
>>> KIT -- University of the State of Baden-Wuerttemberg and
>>>
>>> National Research Center of the Helmholtz Association
>>>
>>> ----------------------------------------------------------
>>>
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150128/0978be1a/attachment.html>


More information about the Gluster-users mailing list