[Gluster-users] Failed to get quota limits
mabi
mabi at protonmail.ch
Tue Feb 13 08:26:41 UTC 2018
Hi Hari,
Sorry for not providing you more details from the start. Here below you will find all the relevant log entries and info. Regarding the quota.conf file I have found one for my volume but it is a binary file. Is it supposed to be binary or text?
Regards,
M.
*** gluster volume info myvolume ***
Volume Name: myvolume
Type: Replicate
Volume ID: e7a40a1b-45c9-4d3c-bb19-0c59b4eceec5
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (2 + 1) = 3
Transport-type: tcp
Bricks:
Brick1: gfs1a.domain.local:/data/myvolume/brick
Brick2: gfs1b.domain.local:/data/myvolume/brick
Brick3: gfs1c.domain.local:/srv/glusterfs/myvolume/brick (arbiter)
Options Reconfigured:
server.event-threads: 4
client.event-threads: 4
performance.readdir-ahead: on
nfs.disable: on
features.quota: on
features.inode-quota: on
features.quota-deem-statfs: on
transport.address-family: inet
*** /var/log/glusterfs/glusterd.log ***
[2018-02-13 08:16:09.929568] E [MSGID: 101042] [compat.c:569:gf_umount_lazy] 0-management: Lazy unmount of /var/run/gluster/myvolume_quota_list/
[2018-02-13 08:16:28.596527] I [MSGID: 106499] [glusterd-handler.c:4363:__glusterd_handle_status_volume] 0-management: Received status volume req for volume myvolume
[2018-02-13 08:16:28.601097] I [MSGID: 106419] [glusterd-utils.c:6110:glusterd_add_inode_size_to_dict] 0-management: the brick on data/myvolume (zfs) uses dynamic inode sizes
*** /var/log/glusterfs/cmd_history.log ***
[2018-02-13 08:16:28.605478] : volume status myvolume detail : SUCCESS
*** /var/log/glusterfs/quota-mount-myvolume.log ***
[2018-02-13 08:16:09.934117] I [MSGID: 100030] [glusterfsd.c:2503:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.10.7 (args: /usr/sbin/glusterfs --volfile-server localhost --volfile-id myvolume -l /var/log/glusterfs/quota-mount-myvolume.log -p /var/run/gluster/myvolume_quota_list.pid --client-pid -5 /var/run/gluster/myvolume_quota_list/)
[2018-02-13 08:16:09.940432] I [MSGID: 101190] [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-02-13 08:16:09.940491] E [socket.c:2327:socket_connect_finish] 0-glusterfs: connection to ::1:24007 failed (Connection refused); disconnecting socket
[2018-02-13 08:16:09.940519] I [glusterfsd-mgmt.c:2134:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected from remote-host: localhost
[2018-02-13 08:16:13.943827] I [afr.c:94:fix_quorum_options] 0-myvolume-replicate-0: reindeer: incoming qtype = none
[2018-02-13 08:16:13.943857] I [afr.c:116:fix_quorum_options] 0-myvolume-replicate-0: reindeer: quorum_count = 2147483647
[2018-02-13 08:16:13.945194] I [MSGID: 101190] [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2018-02-13 08:16:13.945237] I [MSGID: 101190] [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 4
[2018-02-13 08:16:13.945247] I [MSGID: 101190] [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3
[2018-02-13 08:16:13.945941] W [MSGID: 101174] [graph.c:361:_log_if_unknown_option] 0-myvolume-readdir-ahead: option 'parallel-readdir' is not recognized
[2018-02-13 08:16:13.946342] I [MSGID: 114020] [client.c:2352:notify] 0-myvolume-client-0: parent translators are ready, attempting connect on transport
[2018-02-13 08:16:13.946789] I [MSGID: 114020] [client.c:2352:notify] 0-myvolume-client-1: parent translators are ready, attempting connect on transport
[2018-02-13 08:16:13.947151] I [rpc-clnt.c:2000:rpc_clnt_reconfig] 0-myvolume-client-0: changing port to 49153 (from 0)
[2018-02-13 08:16:13.947846] I [MSGID: 114057] [client-handshake.c:1451:select_server_supported_programs] 0-myvolume-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2018-02-13 08:16:13.948073] I [MSGID: 114020] [client.c:2352:notify] 0-myvolume-client-2: parent translators are ready, attempting connect on transport
[2018-02-13 08:16:13.948579] I [rpc-clnt.c:2000:rpc_clnt_reconfig] 0-myvolume-client-1: changing port to 49154 (from 0)
[2018-02-13 08:16:13.948699] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 0-myvolume-client-0: Connected to myvolume-client-0, attached to remote volume '/data/myvolume/brick'.
[2018-02-13 08:16:13.948747] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 0-myvolume-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2018-02-13 08:16:13.948842] I [MSGID: 108005] [afr-common.c:4813:afr_notify] 0-myvolume-replicate-0: Subvolume 'myvolume-client-0' came back up; going online.
Final graph:
+------------------------------------------------------------------------------+
1: volume myvolume-client-0
2: type protocol/client
3: option opversion 31004
4: option clnt-lk-version 1
5: option volfile-checksum 0
6: option volfile-key myvolume
7: option client-version 3.10.7
8: option process-uuid gfs1a-14348-2018/02/13-08:16:09:933625-myvolume-client-0-0-0
9: option fops-version 1298437
10: option ping-timeout 42
11: option remote-host gfs1a.domain.local
12: option remote-subvolume /data/myvolume/brick
13: option transport-type socket
14: option transport.address-family inet
15: option username bea3e634-e174-4bb3-a1d6-25b09d03b536
16: option password 3a6f98bd-795e-4ec4-adfe-42c61ccfa0a6
17: option event-threads 4
18: option transport.tcp-user-timeout 0
19: option transport.socket.keepalive-time 20
20: option transport.socket.keepalive-interval 2
21: option transport.socket.keepalive-count 9
[2018-02-13 08:16:13.949147] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-myvolume-client-0: Server lk version = 1
22: option send-gids true
23: end-volume
24:
25: volume myvolume-client-1
26: type protocol/client
27: option ping-timeout 42
28: option remote-host gfs1b.domain.local
29: option remote-subvolume /data/myvolume/brick
30: option transport-type socket
31: option transport.address-family inet
32: option username bea3e634-e174-4bb3-a1d6-25b09d03b536
33: option password 3a6f98bd-795e-4ec4-adfe-42c61ccfa0a6
34: option event-threads 4
35: option transport.tcp-user-timeout 0
36: option transport.socket.keepalive-time 20
37: option transport.socket.keepalive-interval 2
38: option transport.socket.keepalive-count 9
39: option send-gids true
40: end-volume
41:
42: volume myvolume-client-2
43: type protocol/client
44: option ping-timeout 42
45: option remote-host gfs1c.domain.local
46: option remote-subvolume /srv/glusterfs/myvolume/brick
47: option transport-type socket
48: option transport.address-family inet
49: option username bea3e634-e174-4bb3-a1d6-25b09d03b536
50: option password 3a6f98bd-795e-4ec4-adfe-42c61ccfa0a6
51: option event-threads 4
52: option transport.tcp-user-timeout 0
53: option transport.socket.keepalive-time 20
54: option transport.socket.keepalive-interval 2
55: option transport.socket.keepalive-count 9
56: option send-gids true
57: end-volume
58:
59: volume myvolume-replicate-0
60: type cluster/replicate
61: option afr-pending-xattr myvolume-client-0,myvolume-client-1,myvolume-client-2
62: option arbiter-count 1
63: option use-compound-fops off
64: subvolumes myvolume-client-0 myvolume-client-1 myvolume-client-2
65: end-volume
[2018-02-13 08:16:13.949442] I [rpc-clnt.c:2000:rpc_clnt_reconfig] 0-myvolume-client-2: changing port to 49153 (from 0)
66:
67: volume myvolume-dht
68: type cluster/distribute
69: option lock-migration off
70: subvolumes myvolume-replicate-0
71: end-volume
72:
73: volume myvolume-write-behind
74: type performance/write-behind
75: subvolumes myvolume-dht
76: end-volume
77:
78: volume myvolume-read-ahead
79: type performance/read-ahead
80: subvolumes myvolume-write-behind
81: end-volume
82:
83: volume myvolume-readdir-ahead
84: type performance/readdir-ahead
85: option parallel-readdir off
86: option rda-request-size 131072
87: option rda-cache-limit 10MB
88: subvolumes myvolume-read-ahead
89: end-volume
90:
91: volume myvolume-io-cache
92: type performance/io-cache
93: subvolumes myvolume-readdir-ahead
94: end-volume
95:
96: volume myvolume-quick-read
97: type performance/quick-read
98: subvolumes myvolume-io-cache
99: end-volume
100:
101: volume myvolume-open-behind
102: type performance/open-behind
103: subvolumes myvolume-quick-read
104: end-volume
105:
106: volume myvolume-md-cache
107: type performance/md-cache
108: subvolumes myvolume-open-behind
109: end-volume
110:
111: volume myvolume
112: type debug/io-stats
113: option log-level INFO
114: option latency-measurement off
115: option count-fop-hits off
116: subvolumes myvolume-md-cache
117: end-volume
118:
119: volume meta-autoload
120: type meta
121: subvolumes myvolume
122: end-volume
123:
+------------------------------------------------------------------------------+
[2018-02-13 08:16:13.949813] I [MSGID: 114057] [client-handshake.c:1451:select_server_supported_programs] 0-myvolume-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2018-02-13 08:16:13.950686] I [MSGID: 114057] [client-handshake.c:1451:select_server_supported_programs] 0-myvolume-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2018-02-13 08:16:13.950698] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 0-myvolume-client-1: Connected to myvolume-client-1, attached to remote volume '/data/myvolume/brick'.
[2018-02-13 08:16:13.950759] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 0-myvolume-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2018-02-13 08:16:13.951009] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-myvolume-client-1: Server lk version = 1
[2018-02-13 08:16:13.951320] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 0-myvolume-client-2: Connected to myvolume-client-2, attached to remote volume '/srv/glusterfs/myvolume/brick'.
[2018-02-13 08:16:13.951348] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 0-myvolume-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2018-02-13 08:16:13.952626] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-myvolume-client-2: Server lk version = 1
[2018-02-13 08:16:13.952704] I [fuse-bridge.c:4138:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.23
[2018-02-13 08:16:13.952742] I [fuse-bridge.c:4823:fuse_graph_sync] 0-fuse: switched to graph 0
[2018-02-13 08:16:13.954337] I [MSGID: 108031] [afr-common.c:2357:afr_local_discovery_cbk] 0-myvolume-replicate-0: selecting local read_child myvolume-client-0
[2018-02-13 08:16:14.020681] I [fuse-bridge.c:5081:fuse_thread_proc] 0-fuse: unmounting /var/run/gluster/myvolume_quota_list/
[2018-02-13 08:16:14.020981] W [glusterfsd.c:1360:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8064) [0x7f92aaeef064] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x55d8ddb85f75] -->/usr/sbin/glusterfs(cleanup_and_exit+0x57) [0x55d8ddb85d97] ) 0-: received signum (15), shutting down
[2018-02-13 08:16:14.021017] I [fuse-bridge.c:5804:fini] 0-fuse: Unmounting '/var/run/gluster/myvolume_quota_list/'.
*** /var/log/glusterfs/quotad.log ***
[2018-02-13 08:16:14.023519] W [MSGID: 108027] [afr-common.c:2718:afr_discover_done] 0-myvolume-replicate-0: no read subvols for (null)
[2018-02-13 08:16:14.064410] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
[2018-02-13 08:16:14.074683] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
[2018-02-13 08:16:14.082040] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
[2018-02-13 08:16:14.086950] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
[2018-02-13 08:16:14.087939] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
[2018-02-13 08:16:14.089775] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
[2018-02-13 08:16:14.092937] W [dict.c:599:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x1f3d) [0x7f63153e7f3d] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.10.7/xlator/features/quotad.so(+0x2d72) [0x7f63153e8d72] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f631b9bd870] ) 0-dict: dict is NULL [Invalid argument]
The message "W [MSGID: 108027] [afr-common.c:2718:afr_discover_done] 0-myvolume-replicate-0: no read subvols for (null)" repeated 34 times between [2018-02-13 08:16:14.023519] and [2018-02-13 08:17:26.560943]
*** /var/log/glusterfs/cli.log ***
[2018-02-13 08:16:14.064404] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for 3df709ee-641d-46a2-bd61-889583e3033c
[2018-02-13 08:16:14.074693] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for a27818fe-0248-40fe-bb23-d43d61010478
[2018-02-13 08:16:14.082067] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for daf97388-bcec-4cc0-a8ef-5b93f05b30f6
[2018-02-13 08:16:14.086929] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for 3c768b36-2625-4509-87ef-fe5214cb9b01
[2018-02-13 08:16:14.087905] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for f8cf47d4-4f54-43c5-ab0d-75b45b4677a3
[2018-02-13 08:16:14.089788] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for b4c81a39-2152-45c5-95d3-b796d88226fe
[2018-02-13 08:16:14.092919] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for 16ac4cde-a5d4-451f-adcc-422a542fea24
[2018-02-13 08:16:14.092980] I [input.c:31:cli_batch] 0-: Exiting with: 0
*** /var/log/glusterfs/bricks/data-myvolume-brick.log ***
[2018-02-13 08:16:13.948065] I [addr.c:182:gf_auth] 0-/data/myvolume/brick: allowed = "*", received addr = "127.0.0.1"
[2018-02-13 08:16:13.948105] I [login.c:76:gf_auth] 0-auth/login: allowed user names: bea3e634-e174-4bb3-a1d6-25b09d03b536
[2018-02-13 08:16:13.948125] I [MSGID: 115029] [server-handshake.c:695:server_setvolume] 0-myvolume-server: accepted client from gfs1a-14348-2018/02/13-08:16:09:933625-myvolume-client-0-0-0 (version: 3.10.7)
[2018-02-13 08:16:14.022257] I [MSGID: 115036] [server.c:559:server_rpc_notify] 0-myvolume-server: disconnecting connection from gfs1a-14348-2018/02/13-08:16:09:933625-myvolume-client-0-0-0
[2018-02-13 08:16:14.022465] I [MSGID: 101055] [client_t.c:436:gf_client_unref] 0-myvolume-server: Shutting down connection gfs1a-14348-2018/02/13-08:16:09:933625-myvolume-client-0-0-0
-------- Original Message --------
On February 13, 2018 12:47 AM, Hari Gowtham <hgowtham at redhat.com> wrote:
> Hi,
>
> Can you provide more information like, the volume configuration, quota.conf file and the log files.
>
> On Sat, Feb 10, 2018 at 1:05 AM, mabi <mabi at protonmail.ch> wrote:
>> Hello,
>>
>> I am running GlusterFS 3.10.7 and just noticed by doing a "gluster volume quota <volname> list" that my quotas on that volume are broken. The command returns no output and no errors but by looking in /var/log/glusterfs.cli I found the following errors:
>>
>> [2018-02-09 19:31:24.242324] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for 3df709ee-641d-46a2-bd61-889583e3033c
>> [2018-02-09 19:31:24.249790] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for a27818fe-0248-40fe-bb23-d43d61010478
>> [2018-02-09 19:31:24.252378] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for daf97388-bcec-4cc0-a8ef-5b93f05b30f6
>> [2018-02-09 19:31:24.256775] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for 3c768b36-2625-4509-87ef-fe5214cb9b01
>> [2018-02-09 19:31:24.257434] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for f8cf47d4-4f54-43c5-ab0d-75b45b4677a3
>> [2018-02-09 19:31:24.259126] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for b4c81a39-2152-45c5-95d3-b796d88226fe
>> [2018-02-09 19:31:24.261664] E [cli-cmd-volume.c:1674:cli_cmd_quota_handle_list_all] 0-cli: Failed to get quota limits for 16ac4cde-a5d4-451f-adcc-422a542fea24
>> [2018-02-09 19:31:24.261719] I [input.c:31:cli_batch] 0-: Exiting with: 0
>>
>> How can I fix my quota on that volume again? I had around 30 quotas set on different directories of that volume.
>>
>> Thanks in advance.
>>
>> Regards,
>> M.
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>
> --
> Regards,
> Hari Gowtham.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180213/d9dc2d89/attachment.html>
More information about the Gluster-users
mailing list