[Gluster-users] Quota trouble
Avra Sengupta
asengupt at redhat.com
Tue Apr 21 09:17:20 UTC 2015
In the logs I see, glusterd_lock() being used. This api is called only
in older versions of gluster or if you have a cluster version is less
then 30600. So along with the version of glusterfs used, could you also
let us know what is the cluster version. You can check it as
"operating-version" in /var/lib/glusterd/glusterd.info file.
Regards,
Avra
On 04/21/2015 02:34 PM, Avra Sengupta wrote:
> Hi Kondo,
>
> Can u also mention the version of gluster you are using.
>
> +Adding gluster-users
>
> Regards,
> Avra
> On 04/21/2015 02:27 PM, Avra Sengupta wrote:
>> Hi Kondo,
>>
>> I went through the gluster13 logs you had sent. Seems like something
>> on that machine is holding the lock and is not releasing it. There
>> are ways in which the system might end up in this scenario. I will
>> try and explain the same with an example.
>>
>> Let's say I have gluster 11, gluster12, and gluster 13 in my cluster.
>> I initiate a command from gluster11. Now the first thing that command
>> does is, it holds a lock on all the nodes in the cluster on behalf of
>> gluster11. Once the command does what's intended, it's last act
>> before ending is to unlock all the nodes in the cluster. Now, only
>> the node that has issued the lock, can issue the unlock.
>>
>> In your case what has happened is some command after successfully
>> acquired the lock on gluster13. Now the node which initiated the
>> command, went down or glusterd on that node went down before it could
>> complete the command and it never got to send the unlock to gluster13.
>>
>> There's a workaround to it. You can restart glusterd on gluster13 and
>> it should work fine.
>>
>> Regards,
>> Avra
>>
>> On 04/20/2015 06:55 PM, kenji kondo wrote:
>>> Hello Vijay,
>>> Maybe this is very rare case. But is there any idea?
>>>
>>> Thanks,
>>> Kondo
>>>
>>> 2015-04-15 9:47 GMT+09:00 Vijaikumar M <vmallika at redhat.com
>>> <mailto:vmallika at redhat.com>>:
>>>
>>> Adding Avra...
>>>
>>> Thanks,
>>> Vijay
>>>
>>>
>>> -------- Forwarded Message --------
>>> Subject: Re: [Gluster-users] Quota trouble
>>> Date: Wed, 15 Apr 2015 00:27:26 +0900
>>> From: kenji kondo <kkay.jp at gmail.com> <mailto:kkay.jp at gmail.com>
>>> To: Vijaikumar M <vmallika at redhat.com>
>>> <mailto:vmallika at redhat.com>
>>>
>>>
>>>
>>> Hi Vijay,
>>>
>>> Thanks for your comments.
>>>
>>>
>>> The lock error occurs at one server it's called "gluster13".
>>>
>>> In the gluster13, I tried to create new volume and start quota.
>>> But it failed as below,
>>>
>>>
>>> In both host gluster10 and gluster13, ran below
>>>
>>> $ sudo mkdir /export11/testbrick1
>>>
>>> $ sudo mkdir /export11/testbrick2
>>>
>>> In gluster13, ran below
>>>
>>> $ sudo /usr/sbin/gluster volume create testvol2
>>> gluster13:/export11/testbrick1 gluster13:/export11/testbrick2
>>>
>>> volume create: testvol2: failed: Locking failed on gluster13.
>>> Please check log file for details.
>>>
>>> $ sudo /usr/sbin/gluster volume create testvol2
>>> gluster10:/export11/testbrick1 gluster10:/export11/testbrick2
>>>
>>> volume create: testvol2: failed: Locking failed on gluster13.
>>> Please check log file for details.
>>>
>>> But I recived error messages above.
>>>
>>> On the other hand, in gluster10, it was success.
>>>
>>> Again, in gluster13, I tried to run quota, but it failed as below.
>>>
>>> $ sudo /usr/sbin/gluster volume quota testvol2 enable
>>>
>>> quota command failed : Locking failed on gluster13. Please check
>>> log file for details.
>>>
>>>
>>> Could you find attached?
>>>
>>> We can find error messages in the log of gluster13.
>>>
>>>
>>> Best regards,
>>>
>>> Kondo
>>>
>>>
>>>
>>> 2015-04-13 19:38 GMT+09:00 Vijaikumar M <vmallika at redhat.com
>>> <mailto:vmallika at redhat.com>>:
>>>
>>> Hi Kondo,
>>>
>>> The lock error you mentioned is because, another operation
>>> is still running on the volume and hence not able to acquire
>>> the lock.
>>> This is bug of not displaying proper error message, we are
>>> working on fixing this issue.
>>>
>>> I was not able to find any clue on why quotad is not running.
>>>
>>> I wanted to check, if we can manually start quotad something
>>> like below:
>>>
>>> # /usr/local/sbin/glusterfs -s localhost --volfile-id
>>> gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l
>>> /var/log/glusterfs/quotad.log -S
>>> /var/run/gluster/myquotad.socket --xlator-option
>>> *replicate*.data-self-heal=off --xlator-option
>>> *replicate*.metadata-self-heal=off --xlator-option
>>> *replicate*.entry-self-heal=off
>>>
>>> or
>>>
>>> create a new temporary volume, and enable quota on this
>>> volume. (quotad will be same for all the volume which has
>>> quota enabled)
>>>
>>>
>>> Thanks,
>>> Vijay
>>>
>>>
>>> On Sunday 12 April 2015 07:05 PM, kenji kondo wrote:
>>>> Hi Vijay,
>>>>
>>>> Thank you for your suggestion. But I'm sorry, it's
>>>> difficult to access from outside because my glusterfs
>>>> system is closed.
>>>> I will give up if there is no clue information in attached log.
>>>>
>>>> Best regards,
>>>> Kondo
>>>>
>>>>
>>>> 2015-04-09 15:40 GMT+09:00 Vijaikumar M
>>>> <vmallika at redhat.com <mailto:vmallika at redhat.com>>:
>>>>
>>>>
>>>>
>>>> On Thursday 09 April 2015 11:58 AM, Vijaikumar M wrote:
>>>>>
>>>>>
>>>>> On Wednesday 08 April 2015 09:57 PM, kenji kondo wrote:
>>>>>> Hi Vijay,
>>>>>>
>>>>>> I checked the all of the setting.
>>>>>> The all are 'features.quota=on' when I set quota
>>>>>> enable and the all are 'features.quota=off' when I
>>>>>> set quota disable.
>>>>>>
>>>>>> But I could find new issue.
>>>>>> When I checked a volume status for all server, in one
>>>>>> of the servers I received the error message as below.
>>>>>>
>>>>>> $ sudo /usr/sbin/gluster volume status testvol
>>>>>> Locking failed on gluster13. Please check log file
>>>>>> for details.
>>>>>>
>>>>>> In etc-glusterfs-glusterd.vol.log of problem server,
>>>>>> I found error messages as below.
>>>>>> [2015-04-08 08:40:04.782644] I
>>>>>> [mem-pool.c:545:mem_pool_destroy] 0-management:
>>>>>> size=588 max=0 total=0
>>>>>> [2015-04-08 08:40:04.782685] I
>>>>>> [mem-pool.c:545:mem_pool_destroy] 0-management:
>>>>>> size=124 max=0 total=0
>>>>>> [2015-04-08 08:40:04.782848] W
>>>>>> [socket.c:611:__socket_rwv] 0-management: readv on
>>>>>> /var/run/14b05cd492843e6e288e290c2d63093c.socket
>>>>>> failed (Invalid arguments)
>>>>>> [2015-04-08 08:40:04.805407] I [MSGID: 106006]
>>>>>> [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify]
>>>>>> 0-management: nfs has disconnected from glusterd.
>>>>>> [2015-04-08 08:43:02.439001] I
>>>>>> [glusterd-handler.c:3803:__glusterd_handle_status_volume]
>>>>>> 0-management: Received status volume req for volume
>>>>>> testvol
>>>>>> [2015-04-08 08:43:02.460581] E
>>>>>> [glusterd-utils.c:148:glusterd_lock] 0-management:
>>>>>> Unable to get lock for uuid:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9, lock held by:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9
>>>>>> [2015-04-08 08:43:02.460632] E
>>>>>> [glusterd-op-sm.c:6584:glusterd_op_sm] 0-management:
>>>>>> handler returned: -1
>>>>>> [2015-04-08 08:43:02.460654] E
>>>>>> [glusterd-syncop.c:105:gd_collate_errors] 0-: Locking
>>>>>> failed on gluster13. Please check log file for details.
>>>>>> [2015-04-08 08:43:02.461409] E
>>>>>> [glusterd-syncop.c:1602:gd_sync_task_begin]
>>>>>> 0-management: Locking Peers Failed.
>>>>>> [2015-04-08 08:43:43.698168] I
>>>>>> [glusterd-handler.c:3803:__glusterd_handle_status_volume]
>>>>>> 0-management: Received status volume req for volume
>>>>>> testvol
>>>>>> [2015-04-08 08:43:43.698813] E
>>>>>> [glusterd-utils.c:148:glusterd_lock] 0-management:
>>>>>> Unable to get lock for uuid:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9, lock held by:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9
>>>>>> [2015-04-08 08:43:43.698898] E
>>>>>> [glusterd-op-sm.c:6584:glusterd_op_sm] 0-management:
>>>>>> handler returned: -1
>>>>>> [2015-04-08 08:43:43.698994] E
>>>>>> [glusterd-syncop.c:105:gd_collate_errors] 0-: Locking
>>>>>> failed on gluster13. Please check log file for details.
>>>>>> [2015-04-08 08:43:43.702126] E
>>>>>> [glusterd-syncop.c:1602:gd_sync_task_begin]
>>>>>> 0-management: Locking Peers Failed.
>>>>>> [2015-04-08 08:44:01.277139] I
>>>>>> [glusterd-handler.c:3803:__glusterd_handle_status_volume]
>>>>>> 0-management: Received status volume req for volume
>>>>>> testvol
>>>>>> [2015-04-08 08:44:01.277560] E
>>>>>> [glusterd-utils.c:148:glusterd_lock] 0-management:
>>>>>> Unable to get lock for uuid:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9, lock held by:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9
>>>>>> [2015-04-08 08:44:01.277639] E
>>>>>> [glusterd-op-sm.c:6584:glusterd_op_sm] 0-management:
>>>>>> handler returned: -1
>>>>>> [2015-04-08 08:44:01.277676] E
>>>>>> [glusterd-syncop.c:105:gd_collate_errors] 0-: Locking
>>>>>> failed on gluster13. Please check log file for details.
>>>>>> [2015-04-08 08:44:01.281514] E
>>>>>> [glusterd-syncop.c:1602:gd_sync_task_begin]
>>>>>> 0-management: Locking Peers Failed.
>>>>>> [2015-04-08 08:45:42.599796] I
>>>>>> [glusterd-handler.c:3803:__glusterd_handle_status_volume]
>>>>>> 0-management: Received status volume req for volume
>>>>>> testvol
>>>>>> [2015-04-08 08:45:42.600343] E
>>>>>> [glusterd-utils.c:148:glusterd_lock] 0-management:
>>>>>> Unable to get lock for uuid:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9, lock held by:
>>>>>> 03a32bce-ec63-4dc3-a287-4901a55dd8c9
>>>>>> [2015-04-08 08:45:42.600417] E
>>>>>> [glusterd-op-sm.c:6584:glusterd_op_sm] 0-management:
>>>>>> handler returned: -1
>>>>>> [2015-04-08 08:45:42.600482] E
>>>>>> [glusterd-syncop.c:105:gd_collate_errors] 0-: Locking
>>>>>> failed on gluster13. Please check log file for details.
>>>>>> [2015-04-08 08:45:42.601039] E
>>>>>> [glusterd-syncop.c:1602:gd_sync_task_begin]
>>>>>> 0-management: Locking Peers Failed.
>>>>>>
>>>>>> Does this situation relate to my quota problems?
>>>>>>
>>>>>
>>>>> This is a glusterd different issue. Can we get the
>>>>> glusterd logs from gluster13?
>>>>> Can get access to these machines, so that we can debug
>>>>> live?
>>>>>
>>>>> Thanks,
>>>>> Vijay
>>>>>
>>>> Regarding quota issue, quota feature is enabled
>>>> successfully. I am wondering why quotad is not started.
>>>> If we get the access to the machine, it will be easier
>>>> to debug the issue.
>>>>
>>>> Thanks,
>>>> Vijay
>>>>
>>>>
>>>>>>
>>>>>> Best regards,
>>>>>> Kondo
>>>>>>
>>>>>>
>>>>>> 2015-04-08 15:14 GMT+09:00 Vijaikumar M
>>>>>> <vmallika at redhat.com <mailto:vmallika at redhat.com>>:
>>>>>>
>>>>>> Hi Kondo,
>>>>>>
>>>>>> I suspect, in one of the node quota feature is
>>>>>> not set for some reason and hence quotad is not
>>>>>> starting.
>>>>>>
>>>>>> On all the nodes can you check if below option is
>>>>>> set to 'on'
>>>>>>
>>>>>> # grep quota /var/lib/glusterd/vols/<volname>/info
>>>>>> features.quota=on
>>>>>>
>>>>>>
>>>>>> Also can I get brick logs from all the nodes?
>>>>>>
>>>>>> Also can you create a temporary volume and enable
>>>>>> the quota here and see if see quota works fine
>>>>>> with this volume?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Vijay
>>>>>>
>>>>>> On Tuesday 07 April 2015 08:34 PM, kenji kondo wrote:
>>>>>>> Hi Vijay,
>>>>>>>
>>>>>>> Could you find attached?
>>>>>>> I got logs of server and client.
>>>>>>> As same as before, I could not create a file
>>>>>>> after quota usage-limit setting.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Kondo
>>>>>>>
>>>>>>>
>>>>>>> 2015-04-07 18:34 GMT+09:00 Vijaikumar M
>>>>>>> <vmallika at redhat.com <mailto:vmallika at redhat.com>>:
>>>>>>>
>>>>>>> Hi Konda,
>>>>>>>
>>>>>>> Can we get all the log files?
>>>>>>>
>>>>>>> # gluster volume quota <volname> disable
>>>>>>> # gluster volume quota <volname> enable
>>>>>>>
>>>>>>>
>>>>>>> Now copy all the logs files.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Vijay
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tuesday 07 April 2015 12:39 PM, K.Kondo
>>>>>>> wrote:
>>>>>>>> Thank you very much ! Vijay
>>>>>>>> I want to use a quota because each volume
>>>>>>>> became too big.
>>>>>>>>
>>>>>>>> Best regard
>>>>>>>> Kondo
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2015/04/07 15:18、Vijaikumar M
>>>>>>>> <vmallika at redhat.com
>>>>>>>> <mailto:vmallika at redhat.com>> のメッセージ:
>>>>>>>>
>>>>>>>>> Hi Kondo,
>>>>>>>>>
>>>>>>>>> I couldn’t find clue from the logs. I will
>>>>>>>>> discuss about this issue with my
>>>>>>>>> colleagues today.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Vijay
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Monday 06 April 2015 10:56 PM, kenji
>>>>>>>>> kondo wrote:
>>>>>>>>>> Hello Vijay,
>>>>>>>>>> Is there something idea for this?
>>>>>>>>>> Best regards,
>>>>>>>>>> Kondo
>>>>>>>>>>
>>>>>>>>>> 2015-03-31 22:46 GMT+09:00 kenji kondo
>>>>>>>>>> <kkay.jp at gmail.com
>>>>>>>>>> <mailto:kkay.jp at gmail.com>>:
>>>>>>>>>>
>>>>>>>>>> Hi Vijay,
>>>>>>>>>>
>>>>>>>>>> I'm sorry for late reply.
>>>>>>>>>> I could get the debug mode log as
>>>>>>>>>> attached.
>>>>>>>>>> In this test, unfortunately the quota
>>>>>>>>>> did not work as same as before.
>>>>>>>>>>
>>>>>>>>>> Could you find the cause of my problem?
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Kondo
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2015-03-25 17:20 GMT+09:00 Vijaikumar
>>>>>>>>>> M <vmallika at redhat.com
>>>>>>>>>> <mailto:vmallika at redhat.com>>:
>>>>>>>>>>
>>>>>>>>>> Hi Kondo,
>>>>>>>>>>
>>>>>>>>>> For some reason quota enable was
>>>>>>>>>> not successful. We may have
>>>>>>>>>> re-try enabling quota.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Vijay
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tuesday 24 March 2015 07:08
>>>>>>>>>> PM, kenji kondo wrote:
>>>>>>>>>>> Hi Vijay,
>>>>>>>>>>> Thanks for your checking.
>>>>>>>>>>> Unfortunately, currently I can't
>>>>>>>>>>> stop the service because many
>>>>>>>>>>> users are using.
>>>>>>>>>>> But, I want to know this cause
>>>>>>>>>>> of this trouble, so I will plan
>>>>>>>>>>> to stop. Please wait to get the log.
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>> Kondo
>>>>>>>>>>>
>>>>>>>>>>> 2015-03-24 17:01 GMT+09:00
>>>>>>>>>>> Vijaikumar M
>>>>>>>>>>> <vmallika at redhat.com
>>>>>>>>>>> <mailto:vmallika at redhat.com>>:
>>>>>>>>>>>
>>>>>>>>>>> Hi Kondo,
>>>>>>>>>>>
>>>>>>>>>>> I couldn't find much clue in
>>>>>>>>>>> the glusterd logs, other
>>>>>>>>>>> than the error message you
>>>>>>>>>>> mentioned below.
>>>>>>>>>>> Can you try disabling and
>>>>>>>>>>> enabling the quota again and
>>>>>>>>>>> see if this start quotad?
>>>>>>>>>>>
>>>>>>>>>>> Try below command:
>>>>>>>>>>> # gluster volume quota
>>>>>>>>>>> <volname> disable
>>>>>>>>>>>
>>>>>>>>>>> wait for all quota process
>>>>>>>>>>> to terminate
>>>>>>>>>>> #ps -ef | quota
>>>>>>>>>>>
>>>>>>>>>>> # service glusterd stop
>>>>>>>>>>> # glusterd -LDEBUG
>>>>>>>>>>> # gluster volume quota
>>>>>>>>>>> <volname> enable
>>>>>>>>>>>
>>>>>>>>>>> Now verify if quotad is running
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Vijay
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Monday 23 March 2015
>>>>>>>>>>> 06:24 PM, kenji kondo wrote:
>>>>>>>>>>>> Hi Vijay,
>>>>>>>>>>>> As you pointed out, the
>>>>>>>>>>>> quotad is not running in
>>>>>>>>>>>> the all of server.
>>>>>>>>>>>> I checked the volume status
>>>>>>>>>>>> and got following log.
>>>>>>>>>>>>
>>>>>>>>>>>> Quota Daemon on
>>>>>>>>>>>> gluster25N/ANN/A
>>>>>>>>>>>>
>>>>>>>>>>>> So, I attached requested
>>>>>>>>>>>> log
>>>>>>>>>>>> 'etc-glusterfs-glusterd.vol.log'.
>>>>>>>>>>>> The error messages can be
>>>>>>>>>>>> found in the log.
>>>>>>>>>>>>
>>>>>>>>>>>> [2015-03-19
>>>>>>>>>>>> 11:51:07.457697] E
>>>>>>>>>>>> [glusterd-quota.c:1467:glusterd_op_stage_quota]
>>>>>>>>>>>> 0-management: Quota is
>>>>>>>>>>>> disabled, please enable quota
>>>>>>>>>>>>
>>>>>>>>>>>> If you want more some
>>>>>>>>>>>> information to solve this
>>>>>>>>>>>> problems, please ask me.
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Kondo
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2015-03-23 16:04 GMT+09:00
>>>>>>>>>>>> Vijaikumar M
>>>>>>>>>>>> <vmallika at redhat.com
>>>>>>>>>>>> <mailto:vmallika at redhat.com>>:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Kondo,
>>>>>>>>>>>>
>>>>>>>>>>>> Can you please verify
>>>>>>>>>>>> if quotad is running?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> root at rh1:~ *# gluster
>>>>>>>>>>>> volume status*
>>>>>>>>>>>> Status of volume: vol1
>>>>>>>>>>>> Gluster process TCP
>>>>>>>>>>>> Port RDMA Port Online Pid
>>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>>> Brick
>>>>>>>>>>>> rh1:/var/opt/gluster/bricks/b1/dir
>>>>>>>>>>>> 49152 0 Y 1858
>>>>>>>>>>>> NFS Server on localhost
>>>>>>>>>>>> 2049 0 Y 1879
>>>>>>>>>>>> *Quota Daemon on
>>>>>>>>>>>> localhost N/A N/A
>>>>>>>>>>>> Y 1914 **
>>>>>>>>>>>> *
>>>>>>>>>>>> Task Status of Volume vol1
>>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>>> There are no active
>>>>>>>>>>>> volume tasks
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> root at rh1:~ # *ps -ef |
>>>>>>>>>>>> grep quotad*
>>>>>>>>>>>> root 1914 1 0
>>>>>>>>>>>> 12:29 ? 00:00:00
>>>>>>>>>>>> /usr/local/sbin/glusterfs
>>>>>>>>>>>> -s localhost
>>>>>>>>>>>> --volfile-id
>>>>>>>>>>>> gluster/quotad -p
>>>>>>>>>>>> /var/lib/glusterd/quotad/run/quotad.pid
>>>>>>>>>>>> -l
>>>>>>>>>>>> */var/log/glusterfs/quotad.log*-S
>>>>>>>>>>>> /var/run/gluster/bb6ab82f70f555fd5c0e188fa4e09584.socket
>>>>>>>>>>>> --xlator-option
>>>>>>>>>>>> *replicate*.data-self-heal=off
>>>>>>>>>>>> --xlator-option
>>>>>>>>>>>> *replicate*.metadata-self-heal=off
>>>>>>>>>>>> --xlator-option
>>>>>>>>>>>> *replicate*.entry-self-heal=off
>>>>>>>>>>>> root 1970 1511 0
>>>>>>>>>>>> 12:31 pts/1 00:00:00
>>>>>>>>>>>> grep quotad
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> root at rh1:~ # *gluster
>>>>>>>>>>>> volume info*
>>>>>>>>>>>> Volume Name: vol1
>>>>>>>>>>>> Type: Distribute
>>>>>>>>>>>> Volume ID:
>>>>>>>>>>>> a55519ec-65d1-4741-9ad3-f94020fc9b21
>>>>>>>>>>>> Status: Started
>>>>>>>>>>>> Number of Bricks: 1
>>>>>>>>>>>> Transport-type: tcp
>>>>>>>>>>>> Bricks:
>>>>>>>>>>>> Brick1:
>>>>>>>>>>>> rh1:/var/opt/gluster/bricks/b1/dir
>>>>>>>>>>>> Options Reconfigured:
>>>>>>>>>>>> *features.quota: on**
>>>>>>>>>>>> *
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> If quotad is not
>>>>>>>>>>>> running, can you please
>>>>>>>>>>>> provide glusterd logs
>>>>>>>>>>>> 'usr-local-etc-glusterfs-glusterd.vol.log'.
>>>>>>>>>>>> I will check is there
>>>>>>>>>>>> are any issues starting
>>>>>>>>>>>> quotad.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Vihay
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Monday 23 March 2015
>>>>>>>>>>>> 11:54 AM, K.Kondo wrote:
>>>>>>>>>>>>> Hi Vijay,
>>>>>>>>>>>>> I could not find
>>>>>>>>>>>>> the"quotad.log" in
>>>>>>>>>>>>> directory
>>>>>>>>>>>>> /var/log/glusterfs in
>>>>>>>>>>>>> both servers and
>>>>>>>>>>>>> client. But other test
>>>>>>>>>>>>> server has the log.
>>>>>>>>>>>>> Do you know why there
>>>>>>>>>>>>> is no the file?
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Kondo
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2015/03/23 13:41、
>>>>>>>>>>>>> Vijaikumar M
>>>>>>>>>>>>> <vmallika at redhat.com
>>>>>>>>>>>>> <mailto:vmallika at redhat.com>>
>>>>>>>>>>>>> のメッセージ:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Kondo,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> log file 'quotad.log'
>>>>>>>>>>>>>> is missing in the
>>>>>>>>>>>>>> attachment.Can you
>>>>>>>>>>>>>> provide this log file
>>>>>>>>>>>>>> as well?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Vijay
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Monday 23 March
>>>>>>>>>>>>>> 2015 09:50 AM, kenji
>>>>>>>>>>>>>> kondo wrote:
>>>>>>>>>>>>>>> Hi Vijay,
>>>>>>>>>>>>>>> Could you find the
>>>>>>>>>>>>>>> attached?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>> Kondo
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2015-03-23 12:53
>>>>>>>>>>>>>>> GMT+09:00 Vijaikumar
>>>>>>>>>>>>>>> M
>>>>>>>>>>>>>>> <vmallika at redhat.com
>>>>>>>>>>>>>>> <mailto:vmallika at redhat.com>>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Kondo,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you please
>>>>>>>>>>>>>>> provide below
>>>>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>> gluterfs logs?
>>>>>>>>>>>>>>> client logs
>>>>>>>>>>>>>>> (name of this
>>>>>>>>>>>>>>> log will be
>>>>>>>>>>>>>>> prefixed with
>>>>>>>>>>>>>>> mount-point dirname)
>>>>>>>>>>>>>>> brick logs
>>>>>>>>>>>>>>> quotad logs
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Vijay
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Friday 20
>>>>>>>>>>>>>>> March 2015 06:31
>>>>>>>>>>>>>>> PM, kenji kondo
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hi, Vijay and
>>>>>>>>>>>>>>>> Peter
>>>>>>>>>>>>>>>> Thanks for your
>>>>>>>>>>>>>>>> reply.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I create new
>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>> "testvol" with
>>>>>>>>>>>>>>>> two bricks and
>>>>>>>>>>>>>>>> set quota to
>>>>>>>>>>>>>>>> simplify this
>>>>>>>>>>>>>>>> problem.
>>>>>>>>>>>>>>>> I got the
>>>>>>>>>>>>>>>> glusterfs log
>>>>>>>>>>>>>>>> as following
>>>>>>>>>>>>>>>> after try to
>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>> directory and file.
>>>>>>>>>>>>>>>> BTW, my
>>>>>>>>>>>>>>>> glusterd was
>>>>>>>>>>>>>>>> upgraded from
>>>>>>>>>>>>>>>> older version,
>>>>>>>>>>>>>>>> although I
>>>>>>>>>>>>>>>> don't know
>>>>>>>>>>>>>>>> related to it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>> Kondo
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.931016] I
>>>>>>>>>>>>>>>> [MSGID: 100030]
>>>>>>>>>>>>>>>> [glusterfsd.c:1998:main]
>>>>>>>>>>>>>>>> 0-/usr/sbin/glusterfs:
>>>>>>>>>>>>>>>> Started running
>>>>>>>>>>>>>>>> /usr/sbin/glusterfs
>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> 3.6.0.29 (args:
>>>>>>>>>>>>>>>> /usr/sbin/glusterfs
>>>>>>>>>>>>>>>> --volfile-server=gluster10
>>>>>>>>>>>>>>>> --volfile-id=testvol
>>>>>>>>>>>>>>>> testvol)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.944850] I
>>>>>>>>>>>>>>>> [dht-shared.c:337:dht_init_regex]
>>>>>>>>>>>>>>>> 0-testvol-dht:
>>>>>>>>>>>>>>>> using regex
>>>>>>>>>>>>>>>> rsync-hash-regex =
>>>>>>>>>>>>>>>> ^\.(.+)\.[^.]+$
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.946256] I
>>>>>>>>>>>>>>>> [client.c:2280:notify]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> parent
>>>>>>>>>>>>>>>> translators are
>>>>>>>>>>>>>>>> ready,
>>>>>>>>>>>>>>>> attempting
>>>>>>>>>>>>>>>> connect on
>>>>>>>>>>>>>>>> transport
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.950674] I
>>>>>>>>>>>>>>>> [client.c:2280:notify]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> parent
>>>>>>>>>>>>>>>> translators are
>>>>>>>>>>>>>>>> ready,
>>>>>>>>>>>>>>>> attempting
>>>>>>>>>>>>>>>> connect on
>>>>>>>>>>>>>>>> transport
>>>>>>>>>>>>>>>> Final graph:
>>>>>>>>>>>>>>>> +------------------------------------------------------------------------------+
>>>>>>>>>>>>>>>> 1: volume
>>>>>>>>>>>>>>>> testvol-client-0
>>>>>>>>>>>>>>>> 2: type
>>>>>>>>>>>>>>>> protocol/client
>>>>>>>>>>>>>>>> 3: option
>>>>>>>>>>>>>>>> ping-timeout 42
>>>>>>>>>>>>>>>> 4: option
>>>>>>>>>>>>>>>> remote-host
>>>>>>>>>>>>>>>> gluster24
>>>>>>>>>>>>>>>> 5: option
>>>>>>>>>>>>>>>> remote-subvolume /export25/brick
>>>>>>>>>>>>>>>> 6: option
>>>>>>>>>>>>>>>> transport-type
>>>>>>>>>>>>>>>> socket
>>>>>>>>>>>>>>>> 7: option
>>>>>>>>>>>>>>>> send-gids true
>>>>>>>>>>>>>>>> 8: end-volume
>>>>>>>>>>>>>>>> 9:
>>>>>>>>>>>>>>>> 10: volume
>>>>>>>>>>>>>>>> testvol-client-1
>>>>>>>>>>>>>>>> 11: type
>>>>>>>>>>>>>>>> protocol/client
>>>>>>>>>>>>>>>> 12: option
>>>>>>>>>>>>>>>> ping-timeout 42
>>>>>>>>>>>>>>>> 13: option
>>>>>>>>>>>>>>>> remote-host
>>>>>>>>>>>>>>>> gluster25
>>>>>>>>>>>>>>>> 14: option
>>>>>>>>>>>>>>>> remote-subvolume /export25/brick
>>>>>>>>>>>>>>>> 15: option
>>>>>>>>>>>>>>>> transport-type
>>>>>>>>>>>>>>>> socket
>>>>>>>>>>>>>>>> 16: option
>>>>>>>>>>>>>>>> send-gids true
>>>>>>>>>>>>>>>> 17: end-volume
>>>>>>>>>>>>>>>> 18:
>>>>>>>>>>>>>>>> 19: volume
>>>>>>>>>>>>>>>> testvol-dht
>>>>>>>>>>>>>>>> 20: type
>>>>>>>>>>>>>>>> cluster/distribute
>>>>>>>>>>>>>>>> 21: subvolumes
>>>>>>>>>>>>>>>> testvol-client-0 testvol-client-1
>>>>>>>>>>>>>>>> 22: end-volume
>>>>>>>>>>>>>>>> 23:
>>>>>>>>>>>>>>>> 24: volume
>>>>>>>>>>>>>>>> testvol-write-behind
>>>>>>>>>>>>>>>> 25: type
>>>>>>>>>>>>>>>> performance/write-behind
>>>>>>>>>>>>>>>> 26: subvolumes
>>>>>>>>>>>>>>>> testvol-dht
>>>>>>>>>>>>>>>> 27: end-volume
>>>>>>>>>>>>>>>> 28:
>>>>>>>>>>>>>>>> 29: volume
>>>>>>>>>>>>>>>> testvol-read-ahead
>>>>>>>>>>>>>>>> 30: type
>>>>>>>>>>>>>>>> performance/read-ahead
>>>>>>>>>>>>>>>> 31: subvolumes
>>>>>>>>>>>>>>>> testvol-write-behind
>>>>>>>>>>>>>>>> 32: end-volume
>>>>>>>>>>>>>>>> 33:
>>>>>>>>>>>>>>>> 34: volume
>>>>>>>>>>>>>>>> testvol-io-cache
>>>>>>>>>>>>>>>> 35: type
>>>>>>>>>>>>>>>> performance/io-cache
>>>>>>>>>>>>>>>> 36: subvolumes
>>>>>>>>>>>>>>>> testvol-read-ahead
>>>>>>>>>>>>>>>> 37: end-volume
>>>>>>>>>>>>>>>> 38:
>>>>>>>>>>>>>>>> 39: volume
>>>>>>>>>>>>>>>> testvol-quick-read
>>>>>>>>>>>>>>>> 40: type
>>>>>>>>>>>>>>>> performance/quick-read
>>>>>>>>>>>>>>>> 41: subvolumes
>>>>>>>>>>>>>>>> testvol-io-cache
>>>>>>>>>>>>>>>> 42: end-volume
>>>>>>>>>>>>>>>> 43:
>>>>>>>>>>>>>>>> 44: volume
>>>>>>>>>>>>>>>> testvol-md-cache
>>>>>>>>>>>>>>>> 45: type
>>>>>>>>>>>>>>>> performance/md-cache
>>>>>>>>>>>>>>>> 46: subvolumes
>>>>>>>>>>>>>>>> testvol-quick-read
>>>>>>>>>>>>>>>> 47: end-volume
>>>>>>>>>>>>>>>> 48:
>>>>>>>>>>>>>>>> 49: volume testvol
>>>>>>>>>>>>>>>> 50: type
>>>>>>>>>>>>>>>> debug/io-stats
>>>>>>>>>>>>>>>> 51: option
>>>>>>>>>>>>>>>> latency-measurement
>>>>>>>>>>>>>>>> off
>>>>>>>>>>>>>>>> 52: option
>>>>>>>>>>>>>>>> count-fop-hits off
>>>>>>>>>>>>>>>> 53: subvolumes
>>>>>>>>>>>>>>>> testvol-md-cache
>>>>>>>>>>>>>>>> 54: end-volume
>>>>>>>>>>>>>>>> 55:
>>>>>>>>>>>>>>>> 56: volume
>>>>>>>>>>>>>>>> meta-autoload
>>>>>>>>>>>>>>>> 57: type meta
>>>>>>>>>>>>>>>> 58: subvolumes
>>>>>>>>>>>>>>>> testvol
>>>>>>>>>>>>>>>> 59: end-volume
>>>>>>>>>>>>>>>> 60:
>>>>>>>>>>>>>>>> +------------------------------------------------------------------------------+
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.955337] I
>>>>>>>>>>>>>>>> [rpc-clnt.c:1759:rpc_clnt_reconfig]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> changing port
>>>>>>>>>>>>>>>> to 49155 (from 0)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.957549] I
>>>>>>>>>>>>>>>> [rpc-clnt.c:1759:rpc_clnt_reconfig]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> changing port
>>>>>>>>>>>>>>>> to 49155 (from 0)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.959889] I
>>>>>>>>>>>>>>>> [client-handshake.c:1415:select_server_supported_programs]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> Using Program
>>>>>>>>>>>>>>>> GlusterFS 3.3,
>>>>>>>>>>>>>>>> Num (1298437),
>>>>>>>>>>>>>>>> Version (330)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.960090] I
>>>>>>>>>>>>>>>> [client-handshake.c:1415:select_server_supported_programs]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> Using Program
>>>>>>>>>>>>>>>> GlusterFS 3.3,
>>>>>>>>>>>>>>>> Num (1298437),
>>>>>>>>>>>>>>>> Version (330)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.960376] I
>>>>>>>>>>>>>>>> [client-handshake.c:1200:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> Connected to
>>>>>>>>>>>>>>>> testvol-client-0,
>>>>>>>>>>>>>>>> attached to
>>>>>>>>>>>>>>>> remote volume
>>>>>>>>>>>>>>>> '/export25/brick'.
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.960405] I
>>>>>>>>>>>>>>>> [client-handshake.c:1212:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> Server and
>>>>>>>>>>>>>>>> Client
>>>>>>>>>>>>>>>> lk-version
>>>>>>>>>>>>>>>> numbers are not
>>>>>>>>>>>>>>>> same, reopening
>>>>>>>>>>>>>>>> the fds
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.960471] I
>>>>>>>>>>>>>>>> [client-handshake.c:1200:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> Connected to
>>>>>>>>>>>>>>>> testvol-client-1,
>>>>>>>>>>>>>>>> attached to
>>>>>>>>>>>>>>>> remote volume
>>>>>>>>>>>>>>>> '/export25/brick'.
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.960478] I
>>>>>>>>>>>>>>>> [client-handshake.c:1212:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> Server and
>>>>>>>>>>>>>>>> Client
>>>>>>>>>>>>>>>> lk-version
>>>>>>>>>>>>>>>> numbers are not
>>>>>>>>>>>>>>>> same, reopening
>>>>>>>>>>>>>>>> the fds
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.962288] I
>>>>>>>>>>>>>>>> [fuse-bridge.c:5042:fuse_graph_setup]
>>>>>>>>>>>>>>>> 0-fuse:
>>>>>>>>>>>>>>>> switched to graph 0
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.962351] I
>>>>>>>>>>>>>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> Server lk
>>>>>>>>>>>>>>>> version = 1
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.962362] I
>>>>>>>>>>>>>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> Server lk
>>>>>>>>>>>>>>>> version = 1
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:42:52.962424] I
>>>>>>>>>>>>>>>> [fuse-bridge.c:3971:fuse_init]
>>>>>>>>>>>>>>>> 0-glusterfs-fuse:
>>>>>>>>>>>>>>>> FUSE inited
>>>>>>>>>>>>>>>> with protocol
>>>>>>>>>>>>>>>> versions:
>>>>>>>>>>>>>>>> glusterfs 7.22
>>>>>>>>>>>>>>>> kernel 7.14
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:13.352234] I
>>>>>>>>>>>>>>>> [glusterfsd-mgmt.c:56:mgmt_cbk_spec]
>>>>>>>>>>>>>>>> 0-mgmt: Volume
>>>>>>>>>>>>>>>> file changed
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.518667] I
>>>>>>>>>>>>>>>> [dht-shared.c:337:dht_init_regex]
>>>>>>>>>>>>>>>> 2-testvol-dht:
>>>>>>>>>>>>>>>> using regex
>>>>>>>>>>>>>>>> rsync-hash-regex =
>>>>>>>>>>>>>>>> ^\.(.+)\.[^.]+$
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.520034] W
>>>>>>>>>>>>>>>> [graph.c:344:_log_if_unknown_option]
>>>>>>>>>>>>>>>> 2-testvol-quota: option
>>>>>>>>>>>>>>>> 'timeout' is
>>>>>>>>>>>>>>>> not recognized
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.520091] I
>>>>>>>>>>>>>>>> [client.c:2280:notify]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> parent
>>>>>>>>>>>>>>>> translators are
>>>>>>>>>>>>>>>> ready,
>>>>>>>>>>>>>>>> attempting
>>>>>>>>>>>>>>>> connect on
>>>>>>>>>>>>>>>> transport
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.524546] I
>>>>>>>>>>>>>>>> [client.c:2280:notify]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> parent
>>>>>>>>>>>>>>>> translators are
>>>>>>>>>>>>>>>> ready,
>>>>>>>>>>>>>>>> attempting
>>>>>>>>>>>>>>>> connect on
>>>>>>>>>>>>>>>> transport
>>>>>>>>>>>>>>>> Final graph:
>>>>>>>>>>>>>>>> +------------------------------------------------------------------------------+
>>>>>>>>>>>>>>>> 1: volume
>>>>>>>>>>>>>>>> testvol-client-0
>>>>>>>>>>>>>>>> 2: type
>>>>>>>>>>>>>>>> protocol/client
>>>>>>>>>>>>>>>> 3: option
>>>>>>>>>>>>>>>> ping-timeout 42
>>>>>>>>>>>>>>>> 4: option
>>>>>>>>>>>>>>>> remote-host
>>>>>>>>>>>>>>>> gluster24
>>>>>>>>>>>>>>>> 5: option
>>>>>>>>>>>>>>>> remote-subvolume /export25/brick
>>>>>>>>>>>>>>>> 6: option
>>>>>>>>>>>>>>>> transport-type
>>>>>>>>>>>>>>>> socket
>>>>>>>>>>>>>>>> 7: option
>>>>>>>>>>>>>>>> send-gids true
>>>>>>>>>>>>>>>> 8: end-volume
>>>>>>>>>>>>>>>> 9:
>>>>>>>>>>>>>>>> 10: volume
>>>>>>>>>>>>>>>> testvol-client-1
>>>>>>>>>>>>>>>> 11: type
>>>>>>>>>>>>>>>> protocol/client
>>>>>>>>>>>>>>>> 12: option
>>>>>>>>>>>>>>>> ping-timeout 42
>>>>>>>>>>>>>>>> 13: option
>>>>>>>>>>>>>>>> remote-host
>>>>>>>>>>>>>>>> gluster25
>>>>>>>>>>>>>>>> 14: option
>>>>>>>>>>>>>>>> remote-subvolume /export25/brick
>>>>>>>>>>>>>>>> 15: option
>>>>>>>>>>>>>>>> transport-type
>>>>>>>>>>>>>>>> socket
>>>>>>>>>>>>>>>> 16: option
>>>>>>>>>>>>>>>> send-gids true
>>>>>>>>>>>>>>>> 17: end-volume
>>>>>>>>>>>>>>>> 18:
>>>>>>>>>>>>>>>> 19: volume
>>>>>>>>>>>>>>>> testvol-dht
>>>>>>>>>>>>>>>> 20: type
>>>>>>>>>>>>>>>> cluster/distribute
>>>>>>>>>>>>>>>> 21: subvolumes
>>>>>>>>>>>>>>>> testvol-client-0 testvol-client-1
>>>>>>>>>>>>>>>> 22: end-volume
>>>>>>>>>>>>>>>> 23:
>>>>>>>>>>>>>>>> 24: volume
>>>>>>>>>>>>>>>> testvol-quota
>>>>>>>>>>>>>>>> 25: type
>>>>>>>>>>>>>>>> features/quota
>>>>>>>>>>>>>>>> 26: option
>>>>>>>>>>>>>>>> timeout 0
>>>>>>>>>>>>>>>> 27: option
>>>>>>>>>>>>>>>> deem-statfs off
>>>>>>>>>>>>>>>> 28: subvolumes
>>>>>>>>>>>>>>>> testvol-dht
>>>>>>>>>>>>>>>> 29: end-volume
>>>>>>>>>>>>>>>> 30:
>>>>>>>>>>>>>>>> 31: volume
>>>>>>>>>>>>>>>> testvol-write-behind
>>>>>>>>>>>>>>>> 32: type
>>>>>>>>>>>>>>>> performance/write-behind
>>>>>>>>>>>>>>>> 33: subvolumes
>>>>>>>>>>>>>>>> testvol-quota
>>>>>>>>>>>>>>>> 34: end-volume
>>>>>>>>>>>>>>>> 35:
>>>>>>>>>>>>>>>> 36: volume
>>>>>>>>>>>>>>>> testvol-read-ahead
>>>>>>>>>>>>>>>> 37: type
>>>>>>>>>>>>>>>> performance/read-ahead
>>>>>>>>>>>>>>>> 38: subvolumes
>>>>>>>>>>>>>>>> testvol-write-behind
>>>>>>>>>>>>>>>> 39: end-volume
>>>>>>>>>>>>>>>> 40:
>>>>>>>>>>>>>>>> 41: volume
>>>>>>>>>>>>>>>> testvol-io-cache
>>>>>>>>>>>>>>>> 42: type
>>>>>>>>>>>>>>>> performance/io-cache
>>>>>>>>>>>>>>>> 43: subvolumes
>>>>>>>>>>>>>>>> testvol-read-ahead
>>>>>>>>>>>>>>>> 44: end-volume
>>>>>>>>>>>>>>>> 45:
>>>>>>>>>>>>>>>> 46: volume
>>>>>>>>>>>>>>>> testvol-quick-read
>>>>>>>>>>>>>>>> 47: type
>>>>>>>>>>>>>>>> performance/quick-read
>>>>>>>>>>>>>>>> 48: subvolumes
>>>>>>>>>>>>>>>> testvol-io-cache
>>>>>>>>>>>>>>>> 49: end-volume
>>>>>>>>>>>>>>>> 50:
>>>>>>>>>>>>>>>> 51: volume
>>>>>>>>>>>>>>>> testvol-md-cache
>>>>>>>>>>>>>>>> 52: type
>>>>>>>>>>>>>>>> performance/md-cache
>>>>>>>>>>>>>>>> 53: subvolumes
>>>>>>>>>>>>>>>> testvol-quick-read
>>>>>>>>>>>>>>>> 54: end-volume
>>>>>>>>>>>>>>>> 55:
>>>>>>>>>>>>>>>> 56: volume testvol
>>>>>>>>>>>>>>>> 57: type
>>>>>>>>>>>>>>>> debug/io-stats
>>>>>>>>>>>>>>>> 58: option
>>>>>>>>>>>>>>>> latency-measurement
>>>>>>>>>>>>>>>> off
>>>>>>>>>>>>>>>> 59: option
>>>>>>>>>>>>>>>> count-fop-hits off
>>>>>>>>>>>>>>>> 60: subvolumes
>>>>>>>>>>>>>>>> testvol-md-cache
>>>>>>>>>>>>>>>> 61: end-volume
>>>>>>>>>>>>>>>> 62:
>>>>>>>>>>>>>>>> 63: volume
>>>>>>>>>>>>>>>> meta-autoload
>>>>>>>>>>>>>>>> 64: type meta
>>>>>>>>>>>>>>>> 65: subvolumes
>>>>>>>>>>>>>>>> testvol
>>>>>>>>>>>>>>>> 66: end-volume
>>>>>>>>>>>>>>>> 67:
>>>>>>>>>>>>>>>> +------------------------------------------------------------------------------+
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.530005] I
>>>>>>>>>>>>>>>> [rpc-clnt.c:1759:rpc_clnt_reconfig]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> changing port
>>>>>>>>>>>>>>>> to 49155 (from 0)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.530047] I
>>>>>>>>>>>>>>>> [rpc-clnt.c:1759:rpc_clnt_reconfig]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> changing port
>>>>>>>>>>>>>>>> to 49155 (from 0)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.539062] I
>>>>>>>>>>>>>>>> [client-handshake.c:1415:select_server_supported_programs]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> Using Program
>>>>>>>>>>>>>>>> GlusterFS 3.3,
>>>>>>>>>>>>>>>> Num (1298437),
>>>>>>>>>>>>>>>> Version (330)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.539299] I
>>>>>>>>>>>>>>>> [client-handshake.c:1415:select_server_supported_programs]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> Using Program
>>>>>>>>>>>>>>>> GlusterFS 3.3,
>>>>>>>>>>>>>>>> Num (1298437),
>>>>>>>>>>>>>>>> Version (330)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.539462] I
>>>>>>>>>>>>>>>> [client-handshake.c:1200:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> Connected to
>>>>>>>>>>>>>>>> testvol-client-1,
>>>>>>>>>>>>>>>> attached to
>>>>>>>>>>>>>>>> remote volume
>>>>>>>>>>>>>>>> '/export25/brick'.
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.539485] I
>>>>>>>>>>>>>>>> [client-handshake.c:1212:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> Server and
>>>>>>>>>>>>>>>> Client
>>>>>>>>>>>>>>>> lk-version
>>>>>>>>>>>>>>>> numbers are not
>>>>>>>>>>>>>>>> same, reopening
>>>>>>>>>>>>>>>> the fds
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.539729] I
>>>>>>>>>>>>>>>> [client-handshake.c:1200:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> Connected to
>>>>>>>>>>>>>>>> testvol-client-0,
>>>>>>>>>>>>>>>> attached to
>>>>>>>>>>>>>>>> remote volume
>>>>>>>>>>>>>>>> '/export25/brick'.
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.539751] I
>>>>>>>>>>>>>>>> [client-handshake.c:1212:client_setvolume_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> Server and
>>>>>>>>>>>>>>>> Client
>>>>>>>>>>>>>>>> lk-version
>>>>>>>>>>>>>>>> numbers are not
>>>>>>>>>>>>>>>> same, reopening
>>>>>>>>>>>>>>>> the fds
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.542878] I
>>>>>>>>>>>>>>>> [fuse-bridge.c:5042:fuse_graph_setup]
>>>>>>>>>>>>>>>> 0-fuse:
>>>>>>>>>>>>>>>> switched to graph 2
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.542959] I
>>>>>>>>>>>>>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> Server lk
>>>>>>>>>>>>>>>> version = 1
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:47:15.542987] I
>>>>>>>>>>>>>>>> [client-handshake.c:188:client_set_lk_version_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> Server lk
>>>>>>>>>>>>>>>> version = 1
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:04.586291] I
>>>>>>>>>>>>>>>> [client.c:2289:notify]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> current graph
>>>>>>>>>>>>>>>> is no longer
>>>>>>>>>>>>>>>> active,
>>>>>>>>>>>>>>>> destroying
>>>>>>>>>>>>>>>> rpc_client
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:04.586360] I
>>>>>>>>>>>>>>>> [client.c:2289:notify]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> current graph
>>>>>>>>>>>>>>>> is no longer
>>>>>>>>>>>>>>>> active,
>>>>>>>>>>>>>>>> destroying
>>>>>>>>>>>>>>>> rpc_client
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:04.586378] I
>>>>>>>>>>>>>>>> [client.c:2215:client_rpc_notify]
>>>>>>>>>>>>>>>> 0-testvol-client-0:
>>>>>>>>>>>>>>>> disconnected
>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>> testvol-client-0.
>>>>>>>>>>>>>>>> Client process
>>>>>>>>>>>>>>>> will keep
>>>>>>>>>>>>>>>> trying to
>>>>>>>>>>>>>>>> connect to
>>>>>>>>>>>>>>>> glusterd until
>>>>>>>>>>>>>>>> brick's port is
>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:04.586430] I
>>>>>>>>>>>>>>>> [client.c:2215:client_rpc_notify]
>>>>>>>>>>>>>>>> 0-testvol-client-1:
>>>>>>>>>>>>>>>> disconnected
>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>> testvol-client-1.
>>>>>>>>>>>>>>>> Client process
>>>>>>>>>>>>>>>> will keep
>>>>>>>>>>>>>>>> trying to
>>>>>>>>>>>>>>>> connect to
>>>>>>>>>>>>>>>> glusterd until
>>>>>>>>>>>>>>>> brick's port is
>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:04.589552] W
>>>>>>>>>>>>>>>> [client-rpc-fops.c:306:client3_3_mkdir_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-0:
>>>>>>>>>>>>>>>> remote
>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>> failed:
>>>>>>>>>>>>>>>> Transport
>>>>>>>>>>>>>>>> endpoint is not
>>>>>>>>>>>>>>>> connected.
>>>>>>>>>>>>>>>> Path: /test/a
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:04.589608] W
>>>>>>>>>>>>>>>> [fuse-bridge.c:481:fuse_entry_cbk]
>>>>>>>>>>>>>>>> 0-glusterfs-fuse:
>>>>>>>>>>>>>>>> 78: MKDIR()
>>>>>>>>>>>>>>>> /test/a => -1
>>>>>>>>>>>>>>>> (Transport
>>>>>>>>>>>>>>>> endpoint is not
>>>>>>>>>>>>>>>> connected)
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:11.073349] W
>>>>>>>>>>>>>>>> [client-rpc-fops.c:2212:client3_3_create_cbk]
>>>>>>>>>>>>>>>> 2-testvol-client-1:
>>>>>>>>>>>>>>>> remote
>>>>>>>>>>>>>>>> operation
>>>>>>>>>>>>>>>> failed:
>>>>>>>>>>>>>>>> Transport
>>>>>>>>>>>>>>>> endpoint is not
>>>>>>>>>>>>>>>> connected.
>>>>>>>>>>>>>>>> Path: /test/f
>>>>>>>>>>>>>>>> [2015-03-20
>>>>>>>>>>>>>>>> 03:48:11.073419] W
>>>>>>>>>>>>>>>> [fuse-bridge.c:1937:fuse_create_cbk]
>>>>>>>>>>>>>>>> 0-glusterfs-fuse:
>>>>>>>>>>>>>>>> 82: /test/f =>
>>>>>>>>>>>>>>>> -1 (Transport
>>>>>>>>>>>>>>>> endpoint is not
>>>>>>>>>>>>>>>> connected)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2015-03-20
>>>>>>>>>>>>>>>> 11:27 GMT+09:00
>>>>>>>>>>>>>>>> Vijaikumar M
>>>>>>>>>>>>>>>> <vmallika at redhat.com
>>>>>>>>>>>>>>>> <mailto:vmallika at redhat.com>>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Kondo,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you
>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>> provide all
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> glusterfs
>>>>>>>>>>>>>>>> log files?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Vijay
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Friday
>>>>>>>>>>>>>>>> 20 March
>>>>>>>>>>>>>>>> 2015 07:33
>>>>>>>>>>>>>>>> AM, K.Kondo
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Hello, experts
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I had a
>>>>>>>>>>>>>>>>> trouble
>>>>>>>>>>>>>>>>> about quota.
>>>>>>>>>>>>>>>>> I set
>>>>>>>>>>>>>>>>> quota to
>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>> distributed volume
>>>>>>>>>>>>>>>>> "vol12" as
>>>>>>>>>>>>>>>>> bellow.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> gluster>
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> quota
>>>>>>>>>>>>>>>>> vol12 enable
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> quota :
>>>>>>>>>>>>>>>>> success
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> gluster>
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> quota
>>>>>>>>>>>>>>>>> vol12
>>>>>>>>>>>>>>>>> limit-usage /test
>>>>>>>>>>>>>>>>> 10GB
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> quota :
>>>>>>>>>>>>>>>>> success
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But I
>>>>>>>>>>>>>>>>> couldn't
>>>>>>>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>> file and
>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>> with below
>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>> message.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In a
>>>>>>>>>>>>>>>>> client host,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> $cd test
>>>>>>>>>>>>>>>>> (mounted
>>>>>>>>>>>>>>>>> using fuse)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> $mkdir a
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mkdir:
>>>>>>>>>>>>>>>>> cannot
>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>> `a':
>>>>>>>>>>>>>>>>> Transport
>>>>>>>>>>>>>>>>> endpoint
>>>>>>>>>>>>>>>>> is not
>>>>>>>>>>>>>>>>> connected
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Additionally,
>>>>>>>>>>>>>>>>> I couldn't
>>>>>>>>>>>>>>>>> check
>>>>>>>>>>>>>>>>> quota
>>>>>>>>>>>>>>>>> status
>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>> gluster
>>>>>>>>>>>>>>>>> command.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> gluster>
>>>>>>>>>>>>>>>>> volume
>>>>>>>>>>>>>>>>> quota
>>>>>>>>>>>>>>>>> vol12 list
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Path
>>>>>>>>>>>>>>>>> Hard-limit
>>>>>>>>>>>>>>>>> Soft-limit
>>>>>>>>>>>>>>>>> Used
>>>>>>>>>>>>>>>>> Available
>>>>>>>>>>>>>>>>> Soft-limit
>>>>>>>>>>>>>>>>> exceeded?
>>>>>>>>>>>>>>>>> Hard-limit
>>>>>>>>>>>>>>>>> exceeded?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ---------------------------------------------------------------------------------------------------------------------------
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Here,
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> command
>>>>>>>>>>>>>>>>> stops, so
>>>>>>>>>>>>>>>>> I have to
>>>>>>>>>>>>>>>>> do Ctrl-C.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gluster
>>>>>>>>>>>>>>>>> version is
>>>>>>>>>>>>>>>>> 3.6.1 and
>>>>>>>>>>>>>>>>> 3.6.0.29
>>>>>>>>>>>>>>>>> for server
>>>>>>>>>>>>>>>>> and client
>>>>>>>>>>>>>>>>> respectively.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any idea
>>>>>>>>>>>>>>>>> for this?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> K. Kondo
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> Gluster-users mailing list
>>>>>>>>>>>>>>>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>>>>>>>>>>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150421/1ffda0d3/attachment-0001.html>
More information about the Gluster-users
mailing list