[2019-01-07 19:16:37.015300] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned 0
[2019-01-07 19:16:37.015305] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:37.015309] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned 0
[2019-01-07 19:16:37.015321] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned 0
[2019-01-07 19:16:37.015330] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned 0
[2019-01-07 19:16:37.015338] T [MSGID: 0] [dht-inode-read.c:312:dht_attr_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:37.015347] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned 0
[2019-01-07 19:16:37.015424] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/lib64/libglusterfs.so.0(default_stat_cbk+0x15b) [0x7f6767cc172b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xc3ae) [0x7f67588c53ae] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888597 
[2019-01-07 19:16:37.015453] T [fuse-bridge.c:847:fuse_attr_cbk] 0-glusterfs-fuse: 1369: STAT() / => 1
[2019-01-07 19:16:37.015469] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 120/120 
[2019-01-07 19:16:44.793308] T [fuse-bridge.c:946:fuse_getattr_resume] 0-glusterfs-fuse: 1370: GETATTR 1 (/)
[2019-01-07 19:16:44.793432] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [io-stats.c:2109:io_stats_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0" repeated 4 times between [2019-01-07 19:15:35.383925] and [2019-01-07 19:16:37.015452]
[2019-01-07 19:16:44.793431] T [MSGID: 0] [fuse-bridge.c:949:fuse_getattr_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.793478] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.793489] T [MSGID: 0] [io-stats.c:2803:io_stats_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.793501] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: STAT scheduled as fast fop
[2019-01-07 19:16:44.793524] T [MSGID: 0] [defaults.c:2034:default_stat_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.793555] T [MSGID: 0] [md-cache.c:519:mdc_inode_iatt_get] 0-md-cache: iatt cache not valid for (00000000-0000-0000-0000-000000000001)
[2019-01-07 19:16:44.793567] T [MSGID: 0] [md-cache.c:1269:mdc_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.793577] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.793590] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.793602] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.793610] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.793618] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.793628] T [MSGID: 0] [write-behind.c:2060:wb_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.793639] T [MSGID: 0] [shard.c:1578:shard_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.793653] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.793689] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-0: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793702] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.793720] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793750] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.793763] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.793778] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-1: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793788] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.793798] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793817] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.793828] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.793842] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-2: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793852] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.793862] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793881] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x953 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.793891] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.793906] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-3: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793915] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.793925] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793947] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.793958] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.793976] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-4: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793985] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.793983] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.793995] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794028] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned 0
[2019-01-07 19:16:44.794041] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x887 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.794056] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.794058] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.794075] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-5: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.794087] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.794098] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794105] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x953 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.794123] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.794123] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned 0
[2019-01-07 19:16:44.794138] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.794157] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-6: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.794155] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned 0
[2019-01-07 19:16:44.794175] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.794193] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794192] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.794224] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned 0
[2019-01-07 19:16:44.794233] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa70 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.794247] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x887 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.794273] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned 0
[2019-01-07 19:16:44.794272] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.794327] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned 0
[2019-01-07 19:16:44.794335] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-7: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.794353] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.794354] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned 0
[2019-01-07 19:16:44.794328] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.794374] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794403] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.794429] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned 0
[2019-01-07 19:16:44.794440] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.794430] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned 0
[2019-01-07 19:16:44.794447] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned 0
[2019-01-07 19:16:44.794469] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned 0
[2019-01-07 19:16:44.794492] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa70 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.794505] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned 0
[2019-01-07 19:16:44.794519] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned 0
[2019-01-07 19:16:44.794640] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.794654] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned 0
[2019-01-07 19:16:44.794664] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned 0
[2019-01-07 19:16:44.794672] T [MSGID: 0] [dht-inode-read.c:312:dht_attr_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:44.794733] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/lib64/libglusterfs.so.0(default_stat_cbk+0x15b) [0x7f6767cc172b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xc3ae) [0x7f67588c53ae] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888604 
[2019-01-07 19:16:44.794760] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0" repeated 4 times between [2019-01-07 19:15:51.137455] and [2019-01-07 19:16:44.794758]
[2019-01-07 19:16:44.794760] T [MSGID: 0] [io-stats.c:2109:io_stats_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.794786] T [fuse-bridge.c:847:fuse_attr_cbk] 0-glusterfs-fuse: 1370: STAT() / => 1
[2019-01-07 19:16:44.794800] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 120/120 
[2019-01-07 19:16:44.794979] T [fuse-bridge.c:676:fuse_lookup_resume] 0-glusterfs-fuse: 1371: LOOKUP /test.Big(aa3ef10e-95e0-40d3-9464-133d72fa8a95)
[2019-01-07 19:16:44.794998] T [MSGID: 0] [fuse-bridge.c:689:fuse_lookup_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.795012] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.795022] T [MSGID: 0] [io-stats.c:2790:io_stats_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.795033] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: LOOKUP scheduled as fast fop
[2019-01-07 19:16:44.795069] T [MSGID: 0] [defaults.c:1974:default_lookup_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.795117] T [MSGID: 0] [md-cache.c:519:mdc_inode_iatt_get] 0-md-cache: iatt cache not valid for (aa3ef10e-95e0-40d3-9464-133d72fa8a95)
[2019-01-07 19:16:44.795166] T [MSGID: 0] [md-cache.c:1200:mdc_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.795188] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.795206] T [MSGID: 0] [quick-read.c:514:qr_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.795216] T [MSGID: 0] [io-cache.c:298:ioc_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.795227] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.795237] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.795246] T [MSGID: 0] [write-behind.c:2442:wb_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.795260] T [MSGID: 0] [shard.c:1386:shard_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.795279] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for test.Big
[2019-01-07 19:16:44.795315] D [MSGID: 0] [dht-common.c:3393:dht_lookup] 0-gv1-dht: calling revalidate lookup for /test.Big at gv1-replicate-0
[2019-01-07 19:16:44.795325] T [MSGID: 0] [dht-common.c:3397:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.795358] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.795383] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 1044, payload: 956, rpc hdr: 88
[2019-01-07 19:16:44.795414] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.795431] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.795455] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 1044, payload: 956, rpc hdr: 88
[2019-01-07 19:16:44.795481] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.795498] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.795520] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 1044, payload: 956, rpc hdr: 88
[2019-01-07 19:16:44.795541] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.795766] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.795874] T [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned 0
[2019-01-07 19:16:44.796037] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.796071] T [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned 0
[2019-01-07 19:16:44.796617] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49159: ping timer event already removed
[2019-01-07 19:16:44.796646] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.796652] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.796671] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x409 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.796684] T [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned 0
[2019-01-07 19:16:44.796723] T [MSGID: 0] [afr-common.c:2507:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.796743] D [MSGID: 0] [dht-common.c:1593:dht_revalidate_cbk] 0-gv1-dht: revalidate lookup of /test.Big returned with op_ret 0
[2019-01-07 19:16:44.796764] T [MSGID: 0] [dht-common.c:1909:dht_revalidate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:44.796780] T [MSGID: 0] [shard.c:1329:shard_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned 0
[2019-01-07 19:16:44.796795] T [MSGID: 0] [write-behind.c:2396:wb_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-write-behind returned 0
[2019-01-07 19:16:44.796809] T [MSGID: 0] [io-cache.c:207:ioc_inode_update] 0-gv1-io-cache: locked inode(0x7f6744961190)
[2019-01-07 19:16:44.796824] T [MSGID: 0] [io-cache.c:216:ioc_inode_update] 0-gv1-io-cache: unlocked inode(0x7f6744961190)
[2019-01-07 19:16:44.796836] T [MSGID: 0] [io-cache.c:224:ioc_inode_update] 0-gv1-io-cache: locked table(0x7f6754085360)
[2019-01-07 19:16:44.796848] T [MSGID: 0] [io-cache.c:229:ioc_inode_update] 0-gv1-io-cache: unlocked table(0x7f6754085360)
[2019-01-07 19:16:44.796862] T [MSGID: 0] [io-cache.c:268:ioc_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-cache returned 0
[2019-01-07 19:16:44.796882] T [MSGID: 0] [quick-read.c:473:qr_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-quick-read returned 0
[2019-01-07 19:16:44.796920] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/usr/lib64/glusterfs/4.1.6/xlator/performance/quick-read.so(+0x5185) [0x7f6758ce9185] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xbf12) [0x7f67588c4f12] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888604 
[2019-01-07 19:16:44.796948] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x409 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.796954] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/usr/lib64/glusterfs/4.1.6/xlator/performance/quick-read.so(+0x5185) [0x7f6758ce9185] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xbf2b) [0x7f67588c4f2b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(aa3ef10e-95e0-40d3-9464-133d72fa8a95) time:1546888604 
[2019-01-07 19:16:44.796970] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-2: Ping latency is 0ms
[2019-01-07 19:16:44.796992] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.open-fd-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797009] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.open-fd-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.796992] and [2019-01-07 19:16:44.797007]
[2019-01-07 19:16:44.797009] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.file-size doesn't satisfy caching requirements
[2019-01-07 19:16:44.797039] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.file-size doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797009] and [2019-01-07 19:16:44.797038]
[2019-01-07 19:16:44.797039] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.block-size doesn't satisfy caching requirements
[2019-01-07 19:16:44.797064] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.block-size doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797039] and [2019-01-07 19:16:44.797063]
[2019-01-07 19:16:44.797064] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.parent-entrylk doesn't satisfy caching requirements
[2019-01-07 19:16:44.797088] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.parent-entrylk doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797064] and [2019-01-07 19:16:44.797087]
[2019-01-07 19:16:44.797088] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.entrylk-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797112] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.entrylk-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797088] and [2019-01-07 19:16:44.797111]
[2019-01-07 19:16:44.797112] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.inodelk-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797139] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.inodelk-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797112] and [2019-01-07 19:16:44.797138]
[2019-01-07 19:16:44.797139] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key link-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797164] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key link-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797139] and [2019-01-07 19:16:44.797161]
[2019-01-07 19:16:44.797164] T [MSGID: 0] [md-cache.c:685:mdc_inode_xatt_set] 0-md-cache: xatt cache set for (aa3ef10e-95e0-40d3-9464-133d72fa8a95) time:1546888604
[2019-01-07 19:16:44.797182] T [MSGID: 0] [md-cache.c:1130:mdc_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-md-cache returned 0
[2019-01-07 19:16:44.797190] T [MSGID: 0] [defaults.c:1328:default_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0
[2019-01-07 19:16:44.797198] T [MSGID: 0] [io-stats.c:2278:io_stats_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.797205] T [fuse-bridge.c:518:fuse_entry_cbk] 0-glusterfs-fuse: 1371: LOOKUP() /test.Big => 10692692569927355029
[2019-01-07 19:16:44.797217] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 144/144 
[2019-01-07 19:16:44.797429] T [fuse-bridge.c:2327:fuse_open_resume] 0-glusterfs-fuse: 1372: OPEN /test.Big
[2019-01-07 19:16:44.797445] T [MSGID: 0] [fuse-bridge.c:2330:fuse_open_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.797455] T [MSGID: 0] [defaults.c:2537:default_open] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.797463] T [MSGID: 0] [io-stats.c:2963:io_stats_open] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.797473] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: OPEN scheduled as fast fop
[2019-01-07 19:16:44.797493] T [MSGID: 0] [defaults.c:1828:default_open_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.797512] T [MSGID: 0] [md-cache.c:1924:mdc_open] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.797529] T [MSGID: 0] [open-behind.c:290:ob_open_behind] 0-stack-trace: stack-address: 0x7f6748006778, gv1-open-behind returned 0
[2019-01-07 19:16:44.797540] T [MSGID: 0] [md-cache.c:1901:mdc_open_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-md-cache returned 0
[2019-01-07 19:16:44.797549] T [MSGID: 0] [defaults.c:1199:default_open_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0
[2019-01-07 19:16:44.797560] T [MSGID: 0] [io-stats.c:2098:io_stats_open_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.797567] T [fuse-bridge.c:1091:fuse_fd_cbk] 0-glusterfs-fuse: 1372: OPEN() /test.Big => 0x7f6748005418
[2019-01-07 19:16:44.797579] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 32/32 
[2019-01-07 19:16:44.797601] T [fuse-bridge.c:3120:fuse_fallocate_resume] 0-glusterfs-fuse: 1373: FALLOCATE (0x7f6748005418, flags=0, size=214748364800, offset=0)
[2019-01-07 19:16:44.797613] T [MSGID: 0] [fuse-bridge.c:3128:fuse_fallocate_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.797622] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.797629] T [MSGID: 0] [io-stats.c:3512:io_stats_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.797638] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: FALLOCATE scheduled as slow fop
[2019-01-07 19:16:44.797651] T [MSGID: 0] [defaults.c:2000:default_fallocate_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.797662] T [MSGID: 0] [md-cache.c:2730:mdc_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.797671] T [MSGID: 0] [open-behind.c:207:ob_fd_wake] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.797680] T [MSGID: 0] [quick-read.c:742:qr_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.797690] T [MSGID: 0] [io-cache.c:842:ioc_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.797698] T [MSGID: 0] [defaults.c:2537:default_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.797706] T [MSGID: 0] [read-ahead.c:190:ra_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.797713] T [MSGID: 0] [defaults.c:2537:default_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.797720] T [MSGID: 0] [shard.c:3459:shard_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.797734] T [MSGID: 0] [dht-inode-read.c:139:dht_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.797754] T [MSGID: 0] [afr-open.c:138:afr_open_continue] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.797767] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 32, rpc hdr: 88
[2019-01-07 19:16:44.797790] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.797801] T [MSGID: 0] [afr-open.c:138:afr_open_continue] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.797811] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 32, rpc hdr: 88
[2019-01-07 19:16:44.797834] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.797843] T [MSGID: 0] [afr-open.c:138:afr_open_continue] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.797853] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 32, rpc hdr: 88
[2019-01-07 19:16:44.797867] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.798094] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.798114] T [MSGID: 0] [client-rpc-fops_v2.c:276:client4_0_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-client-1 returned 0
[2019-01-07 19:16:44.798217] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.798211] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.798240] T [MSGID: 0] [client-rpc-fops_v2.c:276:client4_0_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-client-2 returned 0
[2019-01-07 19:16:44.798253] T [MSGID: 0] [client-rpc-fops_v2.c:276:client4_0_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-client-0 returned 0
[2019-01-07 19:16:44.798267] T [MSGID: 0] [afr-open.c:103:afr_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.798282] T [MSGID: 0] [dht-inode-read.c:66:dht_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-dht returned 0
[2019-01-07 19:16:44.798291] T [MSGID: 0] [shard.c:3450:shard_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-shard returned 0
[2019-01-07 19:16:44.798312] T [MSGID: 0] [read-ahead.c:101:ra_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-read-ahead returned 0
[2019-01-07 19:16:44.798321] T [MSGID: 0] [io-cache.c:579:ioc_open_cbk] 0-gv1-io-cache: locked table(0x7f6754085360)
[2019-01-07 19:16:44.798328] T [MSGID: 0] [io-cache.c:584:ioc_open_cbk] 0-gv1-io-cache: unlocked table(0x7f6754085360)
[2019-01-07 19:16:44.798335] T [MSGID: 0] [io-cache.c:586:ioc_open_cbk] 0-gv1-io-cache: locked inode(0x7f6744961190)
[2019-01-07 19:16:44.798341] T [MSGID: 0] [io-cache.c:594:ioc_open_cbk] 0-gv1-io-cache: unlocked inode(0x7f6744961190)
[2019-01-07 19:16:44.798348] T [MSGID: 0] [io-cache.c:609:ioc_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-io-cache returned 0
[2019-01-07 19:16:44.798354] T [MSGID: 0] [defaults.c:1199:default_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-quick-read returned 0
[2019-01-07 19:16:44.798365] T [MSGID: 0] [defaults.c:2000:default_fallocate_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.798373] T [MSGID: 0] [quick-read.c:706:qr_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.798381] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.798388] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.798407] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.798419] T [MSGID: 0] [write-behind.c:2620:wb_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.798446] D [MSGID: 0] [shard.c:1468:shard_lookup_base_file] 0-gv1-shard: Skipping lookup on base file: aa3ef10e-95e0-40d3-9464-133d72fa8a95Serving prebuf off the inode ctx cache
[2019-01-07 19:16:44.798465] T [MSGID: 0] [shard.c:4458:shard_common_inode_write_post_lookup_handler] 0-gv1-shard: FALLOCATE: gfid=aa3ef10e-95e0-40d3-9464-133d72fa8a95 first_block=0 last_block=204 num_blocks=205 offset=0 total_size=214748364800 flags=0
[2019-01-07 19:16:44.798487] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 1 already present. gfid=e9d8d28c-cf70-40fd-adb9-59463d2abf93. Saving inode for future.
[2019-01-07 19:16:44.798510] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 3 already present. gfid=6e9b581f-fe29-4039-9789-c08fcb9d26af. Saving inode for future.
[2019-01-07 19:16:44.798528] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 4 already present. gfid=2cb4185d-9f39-44cb-9f21-3c24ac123888. Saving inode for future.
[2019-01-07 19:16:44.798553] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 6 already present. gfid=26ef307b-0857-4311-9c0f-ef2dcc044048. Saving inode for future.
[2019-01-07 19:16:44.798570] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 7 already present. gfid=03355c43-26d5-418e-ab8f-f0b254a45acb. Saving inode for future.
[2019-01-07 19:16:44.798586] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 8 already present. gfid=efe89301-5eb4-4cdd-9225-90a36db433da. Saving inode for future.
[2019-01-07 19:16:44.798606] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 10 already present. gfid=0bc3064b-214d-468e-ac2c-570ba4b1dc5b. Saving inode for future.
[2019-01-07 19:16:44.798624] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 11 already present. gfid=b19ed2a4-0a12-4c5d-b328-8ccba5e97759. Saving inode for future.
[2019-01-07 19:16:44.798641] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 12 already present. gfid=8aa37d0d-8e18-4d3e-9a4c-5570b5041396. Saving inode for future.
[2019-01-07 19:16:44.798659] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 13 already present. gfid=20013878-944f-4523-bb3f-6496039a438e. Saving inode for future.
[2019-01-07 19:16:44.798677] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 14 already present. gfid=2db14d0d-a1ea-46f8-8c9f-b91408408b42. Saving inode for future.
[2019-01-07 19:16:44.798694] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 15 already present. gfid=751edbbe-a4bc-438d-b8cd-c933eeb5b232. Saving inode for future.
[2019-01-07 19:16:44.798711] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 16 already present. gfid=ba8a25df-9aac-47d1-b599-29a02777722e. Saving inode for future.
[2019-01-07 19:16:44.798728] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 17 already present. gfid=d13aa81e-d1ff-4fa7-b85d-05fc998eaeae. Saving inode for future.
[2019-01-07 19:16:44.798743] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 18 already present. gfid=47c4938e-6fc2-42c4-853d-623776d09468. Saving inode for future.
[2019-01-07 19:16:44.798760] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 19 already present. gfid=5acb2262-f37c-4791-b0d6-9d614a15fdff. Saving inode for future.
[2019-01-07 19:16:44.798778] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 20 already present. gfid=37730223-8d8b-4fbc-b7b3-ac7d9620f926. Saving inode for future.
[2019-01-07 19:16:44.798798] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 22 already present. gfid=944da85a-f60c-4754-89a8-fc4e20ba7ef2. Saving inode for future.
[2019-01-07 19:16:44.798815] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 23 already present. gfid=aa57b991-697b-466f-bf09-794320feea85. Saving inode for future.
[2019-01-07 19:16:44.798833] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 24 already present. gfid=5a93724c-9427-49c9-a7c5-d0e9cd3302b4. Saving inode for future.
[2019-01-07 19:16:44.798849] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 25 already present. gfid=7eabbcd6-d263-4ce2-ac55-f0e42a43df6d. Saving inode for future.
[2019-01-07 19:16:44.798866] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 26 already present. gfid=d4c6c4f6-aef4-4bdf-ab07-ab35e45f5a4e. Saving inode for future.
[2019-01-07 19:16:44.798883] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 27 already present. gfid=37fae235-363a-4689-9edb-e72615350bc8. Saving inode for future.
[2019-01-07 19:16:44.798900] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 28 already present. gfid=0e0dbb81-7b5d-402e-9373-426177ccaed5. Saving inode for future.
[2019-01-07 19:16:44.798922] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 30 already present. gfid=4404ba1e-178c-45dc-aea8-6585a76ca34c. Saving inode for future.
[2019-01-07 19:16:44.798943] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 32 already present. gfid=78ae325f-0bd3-46e6-8656-2b163e1d3eea. Saving inode for future.
[2019-01-07 19:16:44.798961] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 33 already present. gfid=5754dae2-aff1-48d0-a203-6a7720bc93aa. Saving inode for future.
[2019-01-07 19:16:44.798978] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 34 already present. gfid=0d95e301-7cad-4276-9449-9fd2c6c89db1. Saving inode for future.
[2019-01-07 19:16:44.798998] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 36 already present. gfid=8cf0e3cb-c39c-4325-b716-cebb487a9539. Saving inode for future.
[2019-01-07 19:16:44.799015] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 37 already present. gfid=fb1738c2-702f-45ee-8bfc-1e879fe5d300. Saving inode for future.
[2019-01-07 19:16:44.799029] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 38 already present. gfid=4a433e4f-368e-459a-91f2-7156bde6cba3. Saving inode for future.
[2019-01-07 19:16:44.799046] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 39 already present. gfid=dd538fdf-3bab-4038-a961-3a25bdd1d08d. Saving inode for future.
[2019-01-07 19:16:44.799064] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 41 already present. gfid=07ea2519-b445-4539-a3cb-affcf65a4d5a. Saving inode for future.
[2019-01-07 19:16:44.799080] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 42 already present. gfid=276ce4ad-b77c-4f2e-a344-24416d911854. Saving inode for future.
[2019-01-07 19:16:44.799096] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 43 already present. gfid=c8c67105-ef41-43b3-8c28-60371a3d8614. Saving inode for future.
[2019-01-07 19:16:44.799108] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 44 already present. gfid=8c6ad80e-b6ab-4f22-abaf-9a6f49caa199. Saving inode for future.
[2019-01-07 19:16:44.799125] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 45 already present. gfid=1a496b18-e3de-40fa-97c8-59fe3fd7555e. Saving inode for future.
[2019-01-07 19:16:44.799141] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 46 already present. gfid=bf2abbf4-5fb5-4d1c-9214-70257728b0f7. Saving inode for future.
[2019-01-07 19:16:44.799157] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 47 already present. gfid=41842ebb-97c2-4a2d-82e1-a5c0e3624bf5. Saving inode for future.
[2019-01-07 19:16:44.799175] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 48 already present. gfid=0d81d667-b230-41e1-8c28-d2a056797cf6. Saving inode for future.
[2019-01-07 19:16:44.799192] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 49 already present. gfid=8b155c3c-9cab-4cd6-8ca7-d132a4f75bb2. Saving inode for future.
[2019-01-07 19:16:44.799207] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 50 already present. gfid=914ce9bc-fc7d-49bc-b80e-ffd8993a1df3. Saving inode for future.
[2019-01-07 19:16:44.799222] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 51 already present. gfid=bfb436c6-6921-425d-a5b6-3a4936cb0f45. Saving inode for future.
[2019-01-07 19:16:44.799236] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 52 already present. gfid=f1feb40f-e64d-4e2c-b132-a67493f87372. Saving inode for future.
[2019-01-07 19:16:44.799253] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 53 already present. gfid=a9382161-252b-41d0-87f2-53c4057736eb. Saving inode for future.
[2019-01-07 19:16:44.799273] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 54 already present. gfid=df4968a0-7c97-4ac2-a17a-423e1fd96146. Saving inode for future.
[2019-01-07 19:16:44.799291] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 55 already present. gfid=8179a7e5-4cde-40bb-9388-6eeddeea0daa. Saving inode for future.
[2019-01-07 19:16:44.799308] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 56 already present. gfid=bc3e4454-bd99-4446-8f76-de1b995ecd77. Saving inode for future.
[2019-01-07 19:16:44.799325] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 57 already present. gfid=eae201b5-7158-4f88-a104-62f18fd31578. Saving inode for future.
[2019-01-07 19:16:44.799342] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 58 already present. gfid=6e8a6e46-6b29-4bfa-b199-e5d19292d3a8. Saving inode for future.
[2019-01-07 19:16:44.799360] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 59 already present. gfid=808b69cc-a179-4a53-b8f5-3495517200ea. Saving inode for future.
[2019-01-07 19:16:44.799377] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 60 already present. gfid=33a43c1e-2c8b-4c8f-88e8-5bfd5ef4ab64. Saving inode for future.
[2019-01-07 19:16:44.799398] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 61 already present. gfid=d6328725-50ec-44f2-b03b-baaace38f76f. Saving inode for future.
[2019-01-07 19:16:44.799419] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 62 already present. gfid=1d5d28ed-7974-48fd-ae7f-59c221d6ce6e. Saving inode for future.
[2019-01-07 19:16:44.799435] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 63 already present. gfid=ebdd91f1-5633-4e3b-a34e-776668f00521. Saving inode for future.
[2019-01-07 19:16:44.799453] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 64 already present. gfid=8e8eecfe-a8a6-4dfe-9bf6-bf4a2a11e72e. Saving inode for future.
[2019-01-07 19:16:44.799471] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 65 already present. gfid=d0a05d2a-1c71-434d-a652-636c319d1e1f. Saving inode for future.
[2019-01-07 19:16:44.799488] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 66 already present. gfid=93af9de0-19d7-49fb-b8d1-f295c9250402. Saving inode for future.
[2019-01-07 19:16:44.799506] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 67 already present. gfid=e51ddb91-b594-463f-b303-985bd13a7760. Saving inode for future.
[2019-01-07 19:16:44.799527] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 69 already present. gfid=10179039-d866-4165-b2b1-f7d32d1eae89. Saving inode for future.
[2019-01-07 19:16:44.799544] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 70 already present. gfid=88cacf3b-863e-40f6-b72e-eb264e3a4bdb. Saving inode for future.
[2019-01-07 19:16:44.799561] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 71 already present. gfid=790c07eb-2b67-4564-8c11-a5938e6c2644. Saving inode for future.
[2019-01-07 19:16:44.799578] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 72 already present. gfid=ca73ea1d-a79d-4e2e-a4a4-854994398f2e. Saving inode for future.
[2019-01-07 19:16:44.799595] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 73 already present. gfid=c5a0ebf1-9fa4-4e2c-838e-977a47c9c77a. Saving inode for future.
[2019-01-07 19:16:44.799613] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 74 already present. gfid=d9d066f6-bf54-497e-9b86-eefbac56a28e. Saving inode for future.
[2019-01-07 19:16:44.799630] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 75 already present. gfid=2048b5cd-a330-400b-b431-9874c07f9509. Saving inode for future.
[2019-01-07 19:16:44.799648] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 76 already present. gfid=6d528cb9-6eaf-4ed3-96f8-19bb4f880a83. Saving inode for future.
[2019-01-07 19:16:44.799669] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 77 already present. gfid=d72f4da5-d366-44df-be4b-3e81de506ef1. Saving inode for future.
[2019-01-07 19:16:44.799686] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 78 already present. gfid=079276e0-5d72-43f7-864f-bc19747b5e8d. Saving inode for future.
[2019-01-07 19:16:44.799704] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 79 already present. gfid=09755526-ae2e-4454-aec1-3e9462b81d45. Saving inode for future.
[2019-01-07 19:16:44.799722] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 80 already present. gfid=c869911f-afb0-45b8-a8fe-4a0033c58cb9. Saving inode for future.
[2019-01-07 19:16:44.799739] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 81 already present. gfid=da2998d3-8acb-4fa1-bb75-8f34624797f0. Saving inode for future.
[2019-01-07 19:16:44.799756] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 82 already present. gfid=e627cac2-3f6e-4c99-bbca-1ec6c601eb1f. Saving inode for future.
[2019-01-07 19:16:44.799774] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 83 already present. gfid=331d0595-7640-4e0c-bf1e-0b62a705ee80. Saving inode for future.
[2019-01-07 19:16:44.799791] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 84 already present. gfid=1cff125a-7cdc-4c6d-892c-31fe7226c832. Saving inode for future.
[2019-01-07 19:16:44.799809] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 85 already present. gfid=bbff4702-ea16-4070-94f7-b50cdce82ef5. Saving inode for future.
[2019-01-07 19:16:44.799827] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 86 already present. gfid=2d63f899-45ce-4b6b-b9ff-b3d688bdcccd. Saving inode for future.
[2019-01-07 19:16:44.799845] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 87 already present. gfid=99a77c9a-0742-4736-9dc7-18c43cbda9c7. Saving inode for future.
[2019-01-07 19:16:44.799896] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 88 already present. gfid=c399eb36-0df7-493f-b2f8-3cd1f062c0e4. Saving inode for future.
[2019-01-07 19:16:44.799914] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 89 already present. gfid=ee03721d-6d28-4916-b809-145d8c019542. Saving inode for future.
[2019-01-07 19:16:44.799932] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 90 already present. gfid=0c29b886-29a3-45da-aad0-01087bd58b3a. Saving inode for future.
[2019-01-07 19:16:44.799949] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 91 already present. gfid=b232f060-47ec-4c02-a547-db3cb79fbbaa. Saving inode for future.
[2019-01-07 19:16:44.799966] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 92 already present. gfid=82825f38-4654-4c4a-8a09-3ef802a7e33f. Saving inode for future.
[2019-01-07 19:16:44.799985] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 93 already present. gfid=c707bc8b-a650-4ccb-a607-6e7a7753ae8c. Saving inode for future.
[2019-01-07 19:16:44.800003] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 94 already present. gfid=57d73c33-0041-436f-8dea-ee6a6998eae7. Saving inode for future.
[2019-01-07 19:16:44.800020] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 95 already present. gfid=5c30f4a3-c6a5-41cc-babc-1950058784cf. Saving inode for future.
[2019-01-07 19:16:44.800038] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 96 already present. gfid=792743e2-69dc-4537-8b9f-d244ce540a68. Saving inode for future.
[2019-01-07 19:16:44.800056] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 97 already present. gfid=f495af4e-9eb3-4eff-80c7-bfe8b7db5f1d. Saving inode for future.
[2019-01-07 19:16:44.800077] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 98 already present. gfid=cec65637-2b91-442f-b67b-e2fe3357d8e8. Saving inode for future.
[2019-01-07 19:16:44.800095] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 99 already present. gfid=efff1212-0db3-477d-a01f-1211615bd31b. Saving inode for future.
[2019-01-07 19:16:44.800112] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 100 already present. gfid=df2c86a7-467d-4242-9e33-b470264ce841. Saving inode for future.
[2019-01-07 19:16:44.800129] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 101 already present. gfid=806aadfa-df54-447f-9ec9-1c3b4c620cd9. Saving inode for future.
[2019-01-07 19:16:44.800147] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 102 already present. gfid=ae7c18f2-7578-4549-9f28-e4ce09432ce8. Saving inode for future.
[2019-01-07 19:16:44.800164] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 103 already present. gfid=58a971b7-c283-4de4-bcf3-7f15d3dab726. Saving inode for future.
[2019-01-07 19:16:44.800181] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 104 already present. gfid=4910e7f4-2bf3-4ff6-9db9-1edf211ec779. Saving inode for future.
[2019-01-07 19:16:44.800196] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 105 already present. gfid=4dba8fd4-992f-408b-b7d1-796e5ce5d7f3. Saving inode for future.
[2019-01-07 19:16:44.800213] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 106 already present. gfid=f5a0351e-467d-4745-8e31-fd994be3a814. Saving inode for future.
[2019-01-07 19:16:44.800231] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 107 already present. gfid=f560e811-49b5-468f-8a03-9d45525bc8de. Saving inode for future.
[2019-01-07 19:16:44.800248] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 108 already present. gfid=f0c7f965-6419-4c46-819c-d600d4c4fd74. Saving inode for future.
[2019-01-07 19:16:44.800259] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 109 already present. gfid=b912e7bd-a80d-4f05-93ca-0ddf7602e60f. Saving inode for future.
[2019-01-07 19:16:44.800276] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 110 already present. gfid=4e05ec71-d790-43c4-91ec-8719cf0b9e02. Saving inode for future.
[2019-01-07 19:16:44.800293] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 111 already present. gfid=271371d1-b39f-469a-ad42-d9813d07d3eb. Saving inode for future.
[2019-01-07 19:16:44.800310] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 112 already present. gfid=55a5c209-6795-4a21-a760-8729a89c8968. Saving inode for future.
[2019-01-07 19:16:44.800328] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 113 already present. gfid=65e435ab-f2f0-4490-b1f1-465d2c52201d. Saving inode for future.
[2019-01-07 19:16:44.800344] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 114 already present. gfid=e6f45f8f-58cb-4d8d-a011-7bbad73528fc. Saving inode for future.
[2019-01-07 19:16:44.800357] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 115 already present. gfid=6c4cf62d-3645-45d1-ab83-b429a4bd4d41. Saving inode for future.
[2019-01-07 19:16:44.800375] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 116 already present. gfid=ae08c54d-c21c-4741-9f5a-e34929a2c11c. Saving inode for future.
[2019-01-07 19:16:44.800389] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 117 already present. gfid=91f51ae2-7a67-49fb-ac26-7b4b7747c931. Saving inode for future.
[2019-01-07 19:16:44.800414] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 118 already present. gfid=6a605369-c010-4211-b173-3967d71a70f6. Saving inode for future.
[2019-01-07 19:16:44.800431] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 119 already present. gfid=31cb7b78-6a38-40b7-b554-0fa447c5a0a7. Saving inode for future.
[2019-01-07 19:16:44.800450] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 120 already present. gfid=b8576114-cc10-4511-899d-2f866c676014. Saving inode for future.
[2019-01-07 19:16:44.800468] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 121 already present. gfid=9147f132-1736-40a2-bc39-bbd7f4e710d3. Saving inode for future.
[2019-01-07 19:16:44.800486] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 122 already present. gfid=cd1d5a7f-191f-4840-ab6b-9ce482b30205. Saving inode for future.
[2019-01-07 19:16:44.800504] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 123 already present. gfid=df997038-aa04-4095-8668-ccb65f3ab312. Saving inode for future.
[2019-01-07 19:16:44.800517] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 125 already present. gfid=ae7330b4-fee0-4b78-866d-fe2ddcf46039. Saving inode for future.
[2019-01-07 19:16:44.800533] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 126 already present. gfid=fce7b2f9-fb76-453e-aa45-229a4ad3b6ce. Saving inode for future.
[2019-01-07 19:16:44.800550] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 127 already present. gfid=c6864cd5-d548-450d-b728-83cface585fd. Saving inode for future.
[2019-01-07 19:16:44.800571] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 128 already present. gfid=cc24308e-995e-41a0-8285-efa172e5bb1c. Saving inode for future.
[2019-01-07 19:16:44.800589] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 129 already present. gfid=7cb305f1-f87b-42e8-9f3b-8d66679353ca. Saving inode for future.
[2019-01-07 19:16:44.800609] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 131 already present. gfid=45b1e9b8-ed0d-4d85-aa7f-b94b8732c707. Saving inode for future.
[2019-01-07 19:16:44.800626] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 132 already present. gfid=89d274ac-700b-44ac-91d7-ac6ad6053ced. Saving inode for future.
[2019-01-07 19:16:44.800643] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 133 already present. gfid=c1f12116-0b64-441a-840c-c7bb67935888. Saving inode for future.
[2019-01-07 19:16:44.800660] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 134 already present. gfid=3a3422d7-0384-4e97-b708-4f75067996f2. Saving inode for future.
[2019-01-07 19:16:44.800676] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 135 already present. gfid=b095c598-9d5a-4c98-a110-055e9c4e3202. Saving inode for future.
[2019-01-07 19:16:44.800693] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 136 already present. gfid=9baeb644-7218-4bb1-9cb6-4eca5444ca88. Saving inode for future.
[2019-01-07 19:16:44.800709] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 137 already present. gfid=fb1fd885-5c95-4a48-b6ce-5282d887e1da. Saving inode for future.
[2019-01-07 19:16:44.800727] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 138 already present. gfid=d1f63ea3-ed69-442a-8761-9f4625e9e4f8. Saving inode for future.
[2019-01-07 19:16:44.800747] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 140 already present. gfid=411b3820-b88f-41dc-a783-f0d85fb71ab7. Saving inode for future.
[2019-01-07 19:16:44.800764] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 141 already present. gfid=45cf512b-bf67-48a5-9b7d-39bb772eb70c. Saving inode for future.
[2019-01-07 19:16:44.800780] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 142 already present. gfid=9c250fb4-114c-41b5-a431-fc42885e6747. Saving inode for future.
[2019-01-07 19:16:44.800797] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 143 already present. gfid=5c5f994a-9030-4070-98cd-c2cea80257e8. Saving inode for future.
[2019-01-07 19:16:44.800817] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 144 already present. gfid=2b2e0752-44c3-4bdf-8f52-86316e99d5c7. Saving inode for future.
[2019-01-07 19:16:44.800834] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 145 already present. gfid=a5ea9929-2362-4055-84ea-e004dcde8650. Saving inode for future.
[2019-01-07 19:16:44.800848] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 146 already present. gfid=b2d75b34-cf5d-4f8b-8531-8ada36809ccb. Saving inode for future.
[2019-01-07 19:16:44.800882] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 147 already present. gfid=59aca59f-b177-4ee8-90fd-c128cbba6f27. Saving inode for future.
[2019-01-07 19:16:44.800897] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 148 already present. gfid=6250a5db-4cbb-4d4b-8478-6a02c3ee1212. Saving inode for future.
[2019-01-07 19:16:44.800912] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 149 already present. gfid=640ec298-1c21-417c-8712-a848b50fb9c2. Saving inode for future.
[2019-01-07 19:16:44.800927] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 150 already present. gfid=59150000-263a-48da-b856-c79227f270cc. Saving inode for future.
[2019-01-07 19:16:44.800944] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 151 already present. gfid=9276f1b2-cb95-42b0-b439-f024585ca0b0. Saving inode for future.
[2019-01-07 19:16:44.800963] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 153 already present. gfid=db4182e2-af65-4b7a-8085-be629917b3d7. Saving inode for future.
[2019-01-07 19:16:44.800980] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 154 already present. gfid=a9990b31-fb3d-41bb-a88a-3a6ab2fe464c. Saving inode for future.
[2019-01-07 19:16:44.800997] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 155 already present. gfid=b49f4978-eaee-4884-9331-87047d5354d5. Saving inode for future.
[2019-01-07 19:16:44.801014] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 156 already present. gfid=92d9f487-46ca-420c-86a3-0129f4a6df28. Saving inode for future.
[2019-01-07 19:16:44.801030] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 157 already present. gfid=c3310ecc-5df6-4f26-9d99-c3da11c83f1e. Saving inode for future.
[2019-01-07 19:16:44.801045] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 158 already present. gfid=a11f1142-1dcd-4214-9111-e9537348decd. Saving inode for future.
[2019-01-07 19:16:44.801062] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 159 already present. gfid=22bcf1a8-8ee0-4b5e-ba95-6d08c0231bc6. Saving inode for future.
[2019-01-07 19:16:44.801078] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 160 already present. gfid=ade16de3-c448-45da-81dc-5365df4620e1. Saving inode for future.
[2019-01-07 19:16:44.801096] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 161 already present. gfid=2d409bc9-073e-47ad-9046-46183be0adc0. Saving inode for future.
[2019-01-07 19:16:44.801114] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 162 already present. gfid=c2995ff6-8630-40aa-ad9a-09b5c1cb79ff. Saving inode for future.
[2019-01-07 19:16:44.801130] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 163 already present. gfid=ee9d37f3-60c6-4909-85cf-15243d3500e4. Saving inode for future.
[2019-01-07 19:16:44.801147] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 164 already present. gfid=a77f60fb-ee43-4588-9905-717ed2404970. Saving inode for future.
[2019-01-07 19:16:44.801162] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 165 already present. gfid=b80ed048-8f40-45ef-ad1a-1f013721fde9. Saving inode for future.
[2019-01-07 19:16:44.801183] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 166 already present. gfid=8f080cc8-249a-4c54-8783-0bd4b976ef5e. Saving inode for future.
[2019-01-07 19:16:44.801200] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 167 already present. gfid=c87a1cb0-a831-4d4b-b0d8-9a8bc9fa56c7. Saving inode for future.
[2019-01-07 19:16:44.801218] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 169 already present. gfid=75d12dc9-0118-494a-8a15-057c88132497. Saving inode for future.
[2019-01-07 19:16:44.801232] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 171 already present. gfid=f4a2dabc-d4ef-4499-8db3-1c8e8785f3bb. Saving inode for future.
[2019-01-07 19:16:44.801248] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 172 already present. gfid=eff6ec95-c2ea-49b0-9a05-b81fcc2e7fde. Saving inode for future.
[2019-01-07 19:16:44.801271] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 174 already present. gfid=8e395322-2fe0-413a-b34e-6427bb80ae95. Saving inode for future.
[2019-01-07 19:16:44.801287] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 175 already present. gfid=a312e7ff-e652-4271-83b4-f3142df75692. Saving inode for future.
[2019-01-07 19:16:44.801299] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 176 already present. gfid=3757b73e-0bce-4dc8-a860-1bf9818f0b56. Saving inode for future.
[2019-01-07 19:16:44.801315] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 177 already present. gfid=5656e6c2-ae7a-474f-9394-d799d5d9e161. Saving inode for future.
[2019-01-07 19:16:44.801331] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 178 already present. gfid=307d5323-40d1-4b6b-901a-3d16f5558564. Saving inode for future.
[2019-01-07 19:16:44.801347] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 179 already present. gfid=7eafd8cc-1761-4cbf-b6b0-3c63182bc30c. Saving inode for future.
[2019-01-07 19:16:44.801363] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 180 already present. gfid=d1c84c0c-423a-4b95-9740-664d79980b45. Saving inode for future.
[2019-01-07 19:16:44.801383] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 182 already present. gfid=2dbee6fa-42ad-4abe-ac91-f8501f33e651. Saving inode for future.
[2019-01-07 19:16:44.801430] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 186 already present. gfid=f5752290-9a59-4636-865e-126843caf606. Saving inode for future.
[2019-01-07 19:16:44.801453] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 187 already present. gfid=f0aede4c-d1fb-4c2f-ae34-ede8f8726e25. Saving inode for future.
[2019-01-07 19:16:44.801470] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 188 already present. gfid=8dcebbcc-afa9-4231-bdd9-4c4f4f58ca0d. Saving inode for future.
[2019-01-07 19:16:44.801485] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 189 already present. gfid=36485bdd-0a8f-4051-b61b-e7765348074b. Saving inode for future.
[2019-01-07 19:16:44.801501] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 190 already present. gfid=b3050a5b-b6c0-4968-9ddf-a5d5b88f93dc. Saving inode for future.
[2019-01-07 19:16:44.801516] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 191 already present. gfid=9fe1be20-91b8-4077-8a92-bd8d2b690329. Saving inode for future.
[2019-01-07 19:16:44.801531] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 192 already present. gfid=020f1206-2c0a-40c7-b99e-e4b23641c35f. Saving inode for future.
[2019-01-07 19:16:44.801547] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 193 already present. gfid=c525999e-dc72-4290-83ad-e45e44f26754. Saving inode for future.
[2019-01-07 19:16:44.801565] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 194 already present. gfid=626a7f5c-a970-4788-9156-91c91f3efbcf. Saving inode for future.
[2019-01-07 19:16:44.801589] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 195 already present. gfid=4f9b7607-5ead-4ec1-81c1-68c4b492c77b. Saving inode for future.
[2019-01-07 19:16:44.801608] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 196 already present. gfid=056fe284-9bed-4732-998c-847ffb8aa9cd. Saving inode for future.
[2019-01-07 19:16:44.801623] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 197 already present. gfid=ad5b1f62-aea4-439a-8875-289803bd5d8f. Saving inode for future.
[2019-01-07 19:16:44.801640] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 198 already present. gfid=42639461-bbc6-41fd-a821-f81de5e52267. Saving inode for future.
[2019-01-07 19:16:44.801655] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 199 already present. gfid=70b671d1-e10e-4be2-b347-84cb9652ffa8. Saving inode for future.
[2019-01-07 19:16:44.801666] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 200 already present. gfid=dac1924f-c077-4f8b-9e5a-c49622e182dc. Saving inode for future.
[2019-01-07 19:16:44.801682] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 201 already present. gfid=6a1d24cf-c59d-4eaa-a20c-5a651191ace2. Saving inode for future.
[2019-01-07 19:16:44.801698] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 202 already present. gfid=de9a4b9a-032d-4d6c-ac95-dd9d65c90856. Saving inode for future.
[2019-01-07 19:16:44.801708] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 203 already present. gfid=1e532b8e-68bb-4034-8a44-b59b9de91b88. Saving inode for future.
[2019-01-07 19:16:44.801724] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 204 already present. gfid=422c196e-63a2-4802-8f49-de69dd82d8aa. Saving inode for future.
[2019-01-07 19:16:44.801974] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.801999] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.802016] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223621823
[2019-01-07 19:16:44.802027] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.802045] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.802058] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.802069] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.802113] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.802142] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802174] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.802190] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.802217] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802243] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.802262] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.802288] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802328] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.802344] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.802385] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.802423] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802448] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.802463] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.802479] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802500] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.802510] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.802526] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802532] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.802543] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.802555] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.802572] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802680] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.802695] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.802706] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802727] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.802741] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.802750] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802762] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49158: ping timer event already removed
[2019-01-07 19:16:44.802785] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.802807] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.802823] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.802867] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.802886] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802905] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6b9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.802917] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.802923] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.802932] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802948] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.802959] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x954 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.802964] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802968] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.802992] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803007] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.803009] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803039] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x349 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803040] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.803059] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.803070] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803093] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.803104] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.803118] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803130] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803141] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.803178] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x34e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.803189] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-5: Ping latency is 0ms
[2019-01-07 19:16:44.803205] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49157: ping timer event already removed
[2019-01-07 19:16:44.803226] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.803243] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x954 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.803249] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34a Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803262] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.803264] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.803280] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803317] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.803343] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803369] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x771 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.803380] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.803405] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803401] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6b9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.803436] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.803433] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.803449] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803467] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.803490] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803509] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x401 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.803531] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34a Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803541] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-8: Ping latency is 0ms
[2019-01-07 19:16:44.803638] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x349 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803652] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.803662] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803675] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803669] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49156: ping timer event already removed
[2019-01-07 19:16:44.803700] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.803705] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.803736] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x402 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.803751] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.803753] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.803770] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.803787] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803802] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.803845] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803858] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x771 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.803878] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x888 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.803880] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.803907] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803924] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.803959] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x402 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.803969] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803980] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-11: Ping latency is 0ms
[2019-01-07 19:16:44.804001] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ed Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.804014] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.804031] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804050] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x27d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804064] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x888 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.804081] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.804095] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804117] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x401 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.804129] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.804138] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804151] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804164] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.804216] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ed Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.804228] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.804239] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804263] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49153: ping timer event already removed
[2019-01-07 19:16:44.804291] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.804315] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x27e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804330] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.804372] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.804399] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804430] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.804444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x27d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804445] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.804467] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.804479] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804495] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804480] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804507] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.804531] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.804549] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.804557] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x27e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804570] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-14: Ping latency is 0ms
[2019-01-07 19:16:44.804574] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804596] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.804690] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.804713] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.804724] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804746] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.804759] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.804772] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804773] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49160: ping timer event already removed
[2019-01-07 19:16:44.804794] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.804813] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x362 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.804828] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.804872] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.804891] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804916] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa71 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.804926] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.804942] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804959] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.804971] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.804985] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805001] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x466 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805079] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.805100] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.805115] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805130] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805142] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.805152] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49158: ping timer event already removed
[2019-01-07 19:16:44.805173] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.805176] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x362 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.805186] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-17: Ping latency is 0ms
[2019-01-07 19:16:44.805202] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa71 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.805207] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x467 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805213] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.805228] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.805228] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805259] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.805271] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.805272] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.805291] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805290] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805321] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.805332] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.805347] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805365] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.805374] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.805388] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x402 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805469] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x467 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805480] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-20: Ping latency is 0ms
[2019-01-07 19:16:44.805546] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.805559] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.805573] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805583] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49159: ping timer event already removed
[2019-01-07 19:16:44.805593] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x466 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805612] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.805615] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.805632] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805637] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x403 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805653] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805666] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.805681] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.805694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.805721] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.805722] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.805741] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805756] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1087134360
[2019-01-07 19:16:44.805767] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.805779] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.805786] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.805794] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.805816] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x403 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805819] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.805828] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-23: Ping latency is 0ms
[2019-01-07 19:16:44.805848] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805869] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.805878] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.805893] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805908] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.805917] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.805930] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805937] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x402 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805944] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.805953] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.805954] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.805974] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805988] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805991] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.806009] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806027] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.806019] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.806043] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.806044] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.806064] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806053] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806095] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.806109] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.806133] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806153] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.806162] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.806188] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.806195] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.806205] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806216] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.806225] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ba Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.806226] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806236] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.806251] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806252] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.806264] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.806267] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x955 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.806284] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.806287] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806300] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806316] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.806317] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.806331] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.806331] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.806350] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806365] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.806375] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.806381] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806401] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.806414] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.806416] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806435] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806426] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.806459] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.806464] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806496] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.806499] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.806509] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.806511] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.806524] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806530] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806547] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x403 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.806547] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.806557] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.806569] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.806581] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806586] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.806603] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806600] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806619] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.806631] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x889 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.806642] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.806649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6ba Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.806659] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806661] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.806671] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806675] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ee Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.806685] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.806689] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.806699] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806700] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.806715] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806717] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x27f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.806732] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.806737] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x955 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.806747] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.806757] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.806759] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806773] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806786] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806798] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.806810] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.806812] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.806828] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806861] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.806852] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.806888] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.806888] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.806919] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806926] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806952] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.806953] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x363 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.806967] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.806979] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806980] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.807001] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ee Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.807010] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.807011] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.807030] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807031] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807050] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x889 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.807060] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.807063] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa72 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.807068] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807081] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.807088] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x403 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.807097] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807126] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.807137] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807145] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.807149] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807162] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.807162] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.807181] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807198] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.807198] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x468 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.807218] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.807218] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.807233] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807247] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.807252] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.807262] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807263] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.807278] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807288] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.807296] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x27f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.807298] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.807315] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.807320] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807334] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807342] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.807347] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807353] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.807358] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.807368] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807384] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.807391] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa72 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.807419] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.807425] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.807445] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.807445] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807456] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1085903184
[2019-01-07 19:16:44.807474] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.807487] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x363 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.807512] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.807515] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.807521] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.807531] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.807533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807546] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807557] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.807559] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.807588] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807590] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.807602] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.807607] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.807611] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807624] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.807631] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.807639] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807642] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.807651] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807655] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.807664] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.807668] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x468 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.807679] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807678] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.807695] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807696] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.807713] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.807714] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807731] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.807740] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.807758] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807757] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.807779] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.807780] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.807790] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807802] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.807818] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807833] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.807842] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.807849] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.807875] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807890] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.807900] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807909] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.807911] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807922] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.807934] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.807944] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.807949] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.807952] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807965] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807983] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.807988] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.807992] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.808001] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.808013] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808016] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808028] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x956 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.808037] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.808042] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.808050] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808053] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.808062] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808068] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.808077] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.808080] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.808096] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.808104] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.808105] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808136] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808136] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808153] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.808155] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.808171] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.808180] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.808188] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808200] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.808206] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.808238] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.808239] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808256] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808266] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.808273] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.808277] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.808288] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808289] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.808309] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.808319] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.808320] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.808331] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808342] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808344] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808354] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.808366] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.808376] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.808377] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x956 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.808399] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.808402] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808421] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808429] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ef Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.808442] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.808444] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.808453] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.808462] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808463] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808478] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x280 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.808486] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.808487] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.808504] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.808515] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808520] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.808533] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.808545] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.808536] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808554] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808579] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.808585] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.808589] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.808600] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808614] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808625] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.808649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.808659] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.808600] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.808674] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808699] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808708] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.808722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.808725] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.808742] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.808742] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808761] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808772] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808778] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x364 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.808793] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.808791] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.808830] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.808846] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808863] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ef Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.808880] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.808883] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa73 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.808896] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.808898] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808913] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808921] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.808940] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.808941] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.808954] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808966] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.808975] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x280 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.808982] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808993] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.809000] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x469 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.809003] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809020] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.809022] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809036] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.809048] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.809066] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809065] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.809084] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.809091] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.809095] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809114] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.809129] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa73 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.809131] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.809156] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.809157] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809167] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.809188] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809187] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x364 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.809204] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.809205] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.809221] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809232] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809240] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.809242] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.809257] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.809268] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223600709
[2019-01-07 19:16:44.809274] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.809276] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.809293] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.809297] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.809303] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809307] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.809320] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.809324] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.809334] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.809343] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809345] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.809363] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.809366] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809383] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.809399] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.809401] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809422] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.809431] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x469 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.809438] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809448] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.809457] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.809458] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809474] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.809476] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809492] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809490] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.809510] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.809520] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.809561] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.809576] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809593] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.809602] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.809615] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809629] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.809637] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.809654] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809669] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.809669] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.809701] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.809702] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.809718] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809727] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.809738] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.809742] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809757] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.809766] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.809767] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809784] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.809786] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809805] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809804] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.809819] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.809823] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.809833] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.809834] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809847] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809865] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.809877] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.809884] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.809923] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.809932] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809944] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.809952] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.809961] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809962] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.809972] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809978] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.810004] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.810016] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.810026] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.810032] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810047] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810055] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.810067] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810067] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.810087] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.810092] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810113] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.810116] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.810128] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.810129] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.810144] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810158] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.810165] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.810173] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810176] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.810185] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810197] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.810196] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810215] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.810215] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.810240] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810247] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.810257] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.810258] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.810274] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810274] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.810294] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.810304] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.810308] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810319] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810327] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x281 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.810337] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.810339] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.810357] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.810362] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.810385] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810382] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.810433] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.810434] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810447] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.810451] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810483] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.810495] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.810497] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.810504] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810522] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.810530] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.810542] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810541] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.810558] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810561] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x365 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.810575] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.810577] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.810593] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.810603] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.810603] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810625] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810648] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa74 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.810650] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.810659] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.810668] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.810676] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810679] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810697] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.810702] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.810714] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.810715] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.810729] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810731] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810747] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810757] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.810759] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.810775] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.810791] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.810802] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.810811] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810807] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.810830] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x281 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.810836] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810852] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.810862] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810867] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.810879] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.810880] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810895] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810893] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.810912] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.810922] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.810926] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.810939] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810938] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.810955] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.810954] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810981] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa74 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.810990] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.810991] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.811007] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811007] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.811024] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1240589523
[2019-01-07 19:16:44.811028] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x365 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.811031] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.811047] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.811049] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.811059] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811064] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.811075] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.811076] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811087] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.811101] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.811114] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.811124] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811125] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.811141] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811147] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.811157] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.811159] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.811172] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811173] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.811183] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811187] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.811196] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.811200] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.811210] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811210] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.811228] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811239] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.811248] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.811250] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.811268] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.811278] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811283] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.811296] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811300] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811329] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.811337] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.811347] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.811358] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.811364] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811371] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.811379] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.811380] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811389] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.811411] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811414] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811431] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.811439] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.811441] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.811460] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811461] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.811502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.811505] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.811519] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811518] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.811535] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811538] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.811548] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.811562] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811576] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.811584] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.811599] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811603] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.811613] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.811622] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.811622] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.811643] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811656] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.811686] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.811697] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.811690] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811705] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.811723] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811733] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.811746] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.811750] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811771] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.811780] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.811789] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.811801] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.811807] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811810] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811823] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.811829] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.811832] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.811844] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.811855] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811859] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.811873] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811873] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.811889] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.811896] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.811898] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811908] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.811924] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811929] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811946] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.811952] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.811962] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.811974] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.811979] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811990] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.811996] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x282 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.811999] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812011] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.812019] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.812029] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.812037] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.812038] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812052] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812058] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.812068] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.812074] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x970 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.812075] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812107] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812112] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.812125] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.812133] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812155] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.812158] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.812174] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.812184] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812175] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.812204] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.812214] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.812217] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812229] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812237] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x366 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.812248] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.812250] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.812260] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.812269] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812278] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.812293] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812292] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.812310] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.812317] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa75 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.812320] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812338] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.812355] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812356] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.812375] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.812374] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.812386] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.812397] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812413] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812423] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812423] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.812443] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.812455] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x970 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.812456] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.812475] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.812486] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812497] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.812506] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x282 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.812513] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812524] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.812533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812537] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.812549] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.812550] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812565] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812563] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.812585] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.812595] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.812598] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.812609] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.812611] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812620] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812633] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.812638] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa75 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.812648] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.812656] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812665] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.812674] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x366 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.812685] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.812686] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.812702] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812703] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1319971604
[2019-01-07 19:16:44.812718] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.812720] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812733] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.812733] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.812748] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.812756] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.812761] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.812782] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.812792] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812795] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.812810] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.812812] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812828] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.812838] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812838] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.812855] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.812857] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.812871] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.812879] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812871] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812896] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.812902] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.812906] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.812916] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812918] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.812936] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812941] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812955] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.812963] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.812976] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.813003] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.813017] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813033] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.813041] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.813054] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813068] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.813076] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.813089] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813102] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.813110] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.813132] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.813145] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813155] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.813159] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.813175] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.813177] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.813194] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813198] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813217] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.813219] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.813237] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.813238] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.813253] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813267] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813267] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813287] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.813297] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.813284] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.813327] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.813323] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.813346] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813347] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813363] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.813384] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.813402] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.813421] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813422] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.813440] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813443] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.813463] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.813466] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.813478] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813478] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.813490] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813494] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.813504] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.813509] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.813526] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.813536] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.813536] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813556] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813567] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813567] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.813593] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.813599] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.813603] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.813612] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.813622] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813622] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813640] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.813649] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.813650] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.813663] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813663] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.813678] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813680] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x283 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.813696] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.813695] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813718] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.813734] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.813753] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813752] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.813770] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.813777] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x971 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.813780] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813805] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.813796] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.813816] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.813826] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813844] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.813847] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813862] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.813875] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.813876] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813899] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.813903] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813916] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813924] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.813933] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x367 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.813946] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.813964] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.813990] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.813993] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.814005] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814017] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814031] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa76 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.814041] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.814050] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.814056] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814062] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.814072] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814082] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.814090] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.814092] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.814102] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.814111] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814112] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814126] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.814130] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.814147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.814156] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814159] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.814174] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.814185] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.814185] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.814208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814214] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814221] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814233] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.814238] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.814252] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.814258] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x971 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.814269] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814269] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.814285] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814288] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.814298] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.814303] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x283 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.814312] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814314] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.814325] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814328] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.814336] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814346] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.814361] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.814370] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.814382] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.814383] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.814417] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814417] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1413476598
[2019-01-07 19:16:44.814432] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.814440] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa76 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.814443] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.814456] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.814458] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.814469] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814470] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.814495] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x367 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.814502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.814515] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.814523] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814544] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.814556] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.814558] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814571] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814588] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.814584] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.814598] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.814620] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814634] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.814639] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.814659] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.814660] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.814683] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814699] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.814705] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.814714] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814716] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.814727] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814733] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.814743] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.814745] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.814759] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814760] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.814775] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814777] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.814792] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.814797] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.814808] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814808] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.814823] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814825] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.814843] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.814844] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814876] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.814886] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.814902] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.814906] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814921] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.814930] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.814940] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.814931] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814954] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814969] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.814969] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.814979] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.814988] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.814995] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815005] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815011] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.815020] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.815026] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.815037] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.815049] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815050] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.815074] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815091] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.815100] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.815113] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815128] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.815128] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.815136] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.815155] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.815160] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815178] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815184] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.815194] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.815195] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815210] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.815240] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.815241] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.815255] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.815266] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815266] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815292] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.815292] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.815311] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.815311] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.815329] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815329] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815350] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.815357] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.815360] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.815369] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.815377] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815379] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815397] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x284 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.815391] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815419] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.815419] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.815450] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.815455] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.815466] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815466] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.815492] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x972 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.815504] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.815505] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815523] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815533] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.815525] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815577] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.815586] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815591] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.815605] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.815634] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815650] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x368 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.815654] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.815674] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.815677] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.815699] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.815695] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815716] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815735] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.815741] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa77 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.815759] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.815757] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.815778] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815781] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815795] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.815807] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815815] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.815826] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.815833] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815858] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.815867] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.815868] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.815910] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.815926] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815929] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.815960] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815975] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.815982] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.816004] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.816005] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.816016] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816043] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816052] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.816060] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.816063] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.816074] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816075] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.816095] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.816097] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816112] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.816119] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.816121] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816135] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816153] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.816170] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.816173] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.816191] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1252560672
[2019-01-07 19:16:44.816199] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.816201] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.816217] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.816217] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.816228] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.816242] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.816243] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816269] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.816270] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x972 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.816285] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816285] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.816300] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816306] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.816323] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x284 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.816321] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.816343] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.816356] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816357] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816374] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816379] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.816385] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.816414] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.816430] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.816462] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.816470] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816462] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816487] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa77 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.816497] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.816498] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.816511] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.816513] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816534] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.816537] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.816548] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.816554] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816569] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816577] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.816587] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.816587] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.816621] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.816626] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816637] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816643] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.816656] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x368 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.816656] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.816675] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.816680] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816691] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816698] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.816709] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.816710] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816723] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.816734] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.816749] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.816751] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816767] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.816776] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.816777] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816792] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.816815] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.816808] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816833] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.816840] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.816843] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816856] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.816857] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816874] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816877] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.816890] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.816900] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.816932] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.816945] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.816960] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816957] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.816978] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.816979] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816987] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.817009] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.817013] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817031] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.817040] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.817051] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.817053] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817065] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817074] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817079] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.817112] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.817113] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.817145] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.817156] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817157] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.817183] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817203] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.817208] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.817226] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.817224] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.817246] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817248] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817262] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.817272] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.817285] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817280] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.817299] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x285 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.817300] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.817315] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817315] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.817330] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817342] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.817345] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.817382] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817381] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.817410] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x973 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.817438] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.817438] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.817464] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817491] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.817501] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.817515] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.817515] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.817534] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817553] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x369 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.817559] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.817571] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.817571] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.817586] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817598] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817598] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.817619] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.817624] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.817660] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.817664] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa78 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.817668] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817684] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.817692] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.817702] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817703] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.817718] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.817738] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.817742] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.817753] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817753] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.817768] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817773] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.817784] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.817784] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817801] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.817811] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.817826] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.817831] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817842] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.817852] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817856] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.817867] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.817870] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.817882] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817883] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.817900] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.817898] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817911] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.817931] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817932] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.817943] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.817946] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.817963] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817983] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.817988] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.817993] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.818003] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818003] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.818022] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818021] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394291141
[2019-01-07 19:16:44.818038] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.818049] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.818063] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.818071] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.818072] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.818080] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.818099] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.818116] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818127] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.818142] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x973 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.818156] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.818146] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818165] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818183] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.818183] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x285 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.818199] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.818200] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.818209] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818221] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818222] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818232] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.818236] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.818252] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.818261] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.818268] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818299] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.818305] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.818310] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818315] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.818334] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa78 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.818348] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.818351] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.818389] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818445] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.818461] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.818483] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818500] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818511] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.818525] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.818536] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.818549] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.818550] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818572] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818580] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.818594] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.818596] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.818614] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.818624] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818638] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.818643] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x369 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.818662] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.818665] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818671] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818692] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818698] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.818724] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.818725] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.818768] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818772] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.818793] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.818808] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.818810] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.818830] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818832] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818862] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.818865] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.818918] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.818958] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.818982] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819009] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.819024] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.819047] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819073] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.819088] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.819112] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819107] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.819145] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.819146] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819174] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.819179] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819200] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.819223] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.819251] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819256] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.819282] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.819286] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x890 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.819312] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.819313] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819357] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.819342] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819385] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.819405] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.819411] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819451] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.819468] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.819475] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819486] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.819501] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x286 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.819524] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.819566] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.819590] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819502] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819618] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x974 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.819629] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.819630] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819647] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819653] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.819669] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.819679] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.819697] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819697] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.819711] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.819726] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.819728] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.819745] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819751] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.819764] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819778] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819783] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.819796] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.819809] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819811] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa79 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.819848] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.819873] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.819886] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819888] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.819901] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819913] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.819925] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.819930] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.819936] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.819960] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819978] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819989] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.820005] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.820010] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.820029] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.820044] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820048] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.820091] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820080] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820119] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.820126] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.820145] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.820169] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.820197] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.820176] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820214] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820245] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.820261] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.820265] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.820283] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.820286] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820300] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820318] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.820327] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.820344] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.820357] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820366] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.820386] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820390] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.820429] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.820430] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1080394250
[2019-01-07 19:16:44.820456] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.820475] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.820477] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.820490] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.820508] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.820509] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.820529] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820570] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.820575] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.820591] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820600] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.820613] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.820615] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820630] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.820641] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.820648] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820660] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.820668] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.820671] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820689] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.820693] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820706] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820707] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.820722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.820732] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.820735] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.820747] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.820760] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820760] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.820782] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x890 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.820785] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820800] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.820809] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.820810] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820821] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.820837] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820836] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x286 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.820852] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.820853] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.820863] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820874] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.820876] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820890] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820888] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.820920] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.820921] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.820940] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.820940] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.820955] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820970] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.820973] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x974 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.820991] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820992] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.821006] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821011] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.821023] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.821025] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa79 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.821037] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821036] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.821052] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821054] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.821070] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.821073] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.821084] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.821085] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821101] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821108] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.821119] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.821119] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821135] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.821145] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.821162] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.821165] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821196] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.821205] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.821206] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821224] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.821233] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.821241] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821244] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.821253] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821257] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.821267] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.821271] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.821281] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.821286] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821296] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821303] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.821313] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.821315] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.821331] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.821340] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821340] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.821371] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821377] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821388] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.821415] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x891 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.821427] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.821431] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.821443] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.821445] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821454] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821466] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821469] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.821480] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.821481] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.821496] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821496] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.821510] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821513] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x287 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.821528] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.821550] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.821561] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.821562] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.821579] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821579] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821603] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.821605] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x975 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.821622] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.821622] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.821643] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821661] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821664] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.821675] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.821679] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.821687] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821688] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.821706] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821712] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821723] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.821730] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.821745] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.821751] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.821762] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.821771] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821772] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.821791] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.821796] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821807] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.821817] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821822] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.821837] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.821838] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.821858] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.821869] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821882] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821892] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.821868] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821915] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.821926] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.821928] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.821934] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821946] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.821954] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.821964] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.821966] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821975] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821989] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x470 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.821993] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.822011] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.822020] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822016] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.822043] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822054] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.822064] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.822083] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.822094] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.822095] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822124] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822141] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.822143] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x891 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.822162] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.822163] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.822176] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822184] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822196] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.822203] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.822206] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.822221] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822223] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.822241] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.822244] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822258] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.822267] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.822268] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822285] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822297] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.822311] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.822325] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.822328] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.822343] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.822344] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1277578615
[2019-01-07 19:16:44.822355] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822359] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.822372] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.822375] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x975 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.822386] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.822410] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.822438] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.822398] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.822454] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822463] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822475] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.822488] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.822490] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.822508] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.822514] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822524] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822535] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.822544] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.822546] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.822565] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.822570] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822581] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822588] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.822599] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.822600] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x287 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.822617] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.822626] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822627] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.822646] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822651] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822666] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.822675] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.822686] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.822694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.822703] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822704] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.822719] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.822733] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.822738] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.822751] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.822761] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822766] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822778] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.822788] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.822792] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.822797] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822816] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822827] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.822851] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.822861] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.822863] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.822869] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822894] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x470 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.822903] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.822905] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.822920] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822923] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822939] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822949] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.822950] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.822965] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.822977] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.822984] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822991] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.823000] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.823001] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823011] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.823023] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823028] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823042] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.823049] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.823052] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.823063] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823064] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.823107] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.823118] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.823120] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.823135] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823141] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823155] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.823160] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.823171] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.823171] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.823185] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823191] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823204] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.823219] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.823228] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823226] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.823249] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823251] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.823267] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.823279] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823294] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.823305] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.823305] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.823323] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823325] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.823343] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.823353] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.823361] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823372] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.823379] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.823389] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.823391] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823418] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823431] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823436] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x892 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.823453] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.823453] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.823483] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823491] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.823501] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.823502] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.823512] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.823517] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823528] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823537] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.823542] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x288 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.823555] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.823556] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.823574] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823590] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.823594] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.823605] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.823606] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823616] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823627] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823637] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.823638] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x976 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.823662] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.823661] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.823679] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.823689] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823700] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823710] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.823721] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.823723] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x892 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.823738] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823739] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.823750] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823755] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.823769] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.823780] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.823764] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.823788] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823807] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.823819] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.823819] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.823836] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823843] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823854] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823865] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.823871] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.823887] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.823894] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.823911] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.823914] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823927] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823937] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.823946] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x976 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.823950] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.823962] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.823968] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823986] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823992] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x471 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.824008] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.824018] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x288 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.824033] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.824036] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.824050] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824051] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824074] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.824070] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824086] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.824106] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.824111] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824140] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.824142] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.824152] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.824161] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.824170] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824171] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824188] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.824195] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.824205] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.824214] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824225] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.824231] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.824240] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.824241] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.824259] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824260] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1342729215
[2019-01-07 19:16:44.824276] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.824278] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824289] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.824290] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.824305] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.824320] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.824324] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.824333] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.824343] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824354] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.824363] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.824371] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824380] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.824401] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.824403] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824428] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.824444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.824448] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824476] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.824489] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.824493] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824514] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.824530] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x471 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.824540] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824552] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.824562] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.824563] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824582] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.824583] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824622] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.824617] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.824641] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824661] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.824662] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.824685] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.824687] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.824704] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824701] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.824723] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824731] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.824754] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824750] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.824770] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.824767] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.824790] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.824791] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824823] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.824840] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824865] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.824872] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.824889] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.824890] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.824910] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824913] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824927] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.824937] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.824945] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.824951] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824965] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.824975] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.824986] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.824988] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825012] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.825020] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.825027] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825037] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.825045] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.825047] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825060] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.825062] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825077] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825076] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.825095] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.825105] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.825108] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.825125] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.825156] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.825158] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825171] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.825181] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.825197] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.825197] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.825212] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825214] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825233] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.825245] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x893 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.825249] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.825256] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.825273] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825270] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825289] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.825290] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825299] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.825316] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.825322] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825366] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x289 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.825369] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.825390] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.825397] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.825410] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825437] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.825448] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.825447] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.825462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825481] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825487] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x893 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.825505] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.825516] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x977 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.825519] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825538] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.825553] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.825563] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825587] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.825587] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.825610] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825611] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.825645] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825644] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.825670] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.825668] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.825697] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825698] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.825731] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.825745] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.825744] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.825768] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825769] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825785] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825799] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.825795] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.825827] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.825849] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.825857] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825873] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.825884] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.825889] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825906] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.825916] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.825923] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825934] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.825942] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x472 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.825944] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825952] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.825970] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825987] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.825989] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.826004] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826031] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.826031] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x289 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.826045] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.826056] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.826062] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826080] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826087] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.826102] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826103] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.826127] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826126] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.826144] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.826181] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x977 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.826197] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.826200] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.826219] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.826220] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826242] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1288142377
[2019-01-07 19:16:44.826253] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.826255] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.826269] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.826269] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.826287] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.826287] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826297] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.826313] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.826323] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.826324] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.826339] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826342] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826361] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.826368] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.826381] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.826383] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.826403] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826407] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.826421] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826429] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.826437] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.826444] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826468] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x417 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.826472] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.826477] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.826490] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.826505] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826509] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.826525] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826525] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.826543] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.826559] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.826561] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.826582] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826587] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826606] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.826608] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.826625] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.826625] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.826639] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826642] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826660] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x472 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.826663] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.826678] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.826679] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.826700] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826715] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826716] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.826734] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.826739] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826759] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.826763] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.826776] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.826779] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.826794] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826792] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826812] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.826811] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826829] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.826829] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.826846] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826845] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.826860] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826876] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.826890] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.826901] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.826914] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.826927] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.826927] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826952] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826961] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x417 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.826970] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.826972] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.826980] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.826985] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826995] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827010] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.827019] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.827022] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.827032] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827043] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.827049] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.827054] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827059] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.827070] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827084] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.827086] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.827112] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827119] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.827131] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.827135] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x894 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.827140] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827158] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.827166] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.827174] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827177] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.827187] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827191] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.827201] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.827208] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.827219] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.827218] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827230] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827242] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827245] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.827257] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.827257] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.827286] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.827298] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.827302] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827317] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.827327] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827333] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x978 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.827343] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.827346] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.827366] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827367] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.827388] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.827389] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827417] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.827434] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827433] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.827447] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.827451] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.827462] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.827462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827483] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827492] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.827495] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.827508] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827522] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x894 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.827531] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.827533] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.827550] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827550] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.827571] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.827579] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827599] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.827610] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827613] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.827630] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.827629] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.827652] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.827665] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827666] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827693] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.827693] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x473 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.827710] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.827712] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.827719] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827739] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.827751] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.827751] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.827768] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827771] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827779] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827790] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.827798] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.827809] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.827815] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x978 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.827837] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827840] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.827850] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827857] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.827869] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.827873] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.827885] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.827891] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827894] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827906] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827908] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.827925] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.827950] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.827949] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.827968] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.827973] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.827979] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827984] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1374046734
[2019-01-07 19:16:44.827993] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.827998] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.828010] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.828010] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.828033] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.828035] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828042] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.828057] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.828068] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.828068] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.828083] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828086] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828101] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.828106] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.828117] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.828117] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.828132] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828134] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828150] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828157] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.828161] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.828166] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.828191] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828193] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.828204] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.828206] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x418 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.828215] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828230] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.828239] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.828249] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.828255] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.828270] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828273] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828294] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x473 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.828297] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.828311] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.828311] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.828325] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828328] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828342] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828349] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.828353] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.828359] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.828378] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828389] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.828398] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.828419] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.828419] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.828436] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828449] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828451] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.828470] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.828475] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828486] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.828495] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828495] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.828512] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.828535] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828551] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.828559] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.828573] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828566] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x418 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.828586] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.828626] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.828626] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.828653] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.828654] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828668] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828686] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.828691] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.828695] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.828712] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.828718] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828740] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.828755] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.828758] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828770] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828791] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.828796] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.828800] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.828817] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.828834] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.828835] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828852] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828862] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828876] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x895 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.828888] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.828888] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.828905] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828921] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fa Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.828929] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.828936] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.828946] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828961] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.828969] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.828982] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.828983] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829007] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.829024] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829024] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.829041] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.829046] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x979 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.829069] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.829073] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829086] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829112] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.829115] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.829121] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.829136] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.829144] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829162] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829167] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.829182] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.829187] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.829198] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.829208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829208] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.829228] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829228] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.829244] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.829252] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.829256] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829268] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.829270] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829285] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829286] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.829301] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.829312] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.829321] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x895 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.829328] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829340] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.829349] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x474 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.829361] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.829363] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829386] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.829404] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.829410] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829424] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.829441] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.829439] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829454] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.829469] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829476] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.829484] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.829502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.829504] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.829518] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829521] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829534] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.829546] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829567] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.829564] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.829586] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.829603] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1111239957
[2019-01-07 19:16:44.829611] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.829612] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.829629] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.829631] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.829638] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.829649] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.829652] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829675] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.829684] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.829691] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829702] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.829715] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x788 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.829717] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829725] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.829741] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829748] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829774] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.829768] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.829784] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.829806] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829821] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x419 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.829818] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fa Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.829830] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.829842] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.829858] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.829858] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829881] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829898] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.829899] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x979 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.829916] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.829919] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.829931] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829934] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829947] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.829956] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.829963] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.829970] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829985] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.829994] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.830006] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.830008] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830028] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830033] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.830058] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830055] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.830075] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.830085] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.830097] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.830102] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830116] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.830125] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.830135] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.830135] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830156] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830171] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.830180] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.830173] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.830209] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.830218] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.830240] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830236] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830257] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.830267] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.830268] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.830281] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830292] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.830297] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.830318] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.830321] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830334] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830349] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.830352] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.830358] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.830375] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.830402] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.830389] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830421] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.830447] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x896 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.830466] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.830464] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.830484] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830487] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830503] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.830518] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.830521] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830534] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830542] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.830551] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.830563] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.830584] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.830591] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.830611] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.830618] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830631] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830645] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.830657] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.830664] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x474 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.830672] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830687] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.830696] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.830707] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.830710] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830721] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830730] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830736] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x370 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.830761] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.830763] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.830796] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.830813] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830819] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.830835] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.830837] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.830856] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.830856] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830874] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830883] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830891] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.830907] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.830908] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.830922] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830934] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.830940] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x475 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.830956] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.830958] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830987] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.831012] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831032] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.831036] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.831058] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.831060] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.831074] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831075] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831093] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.831106] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x419 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.831107] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.831125] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.831135] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831146] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831153] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.831178] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x788 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.831192] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.831194] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.831207] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.831210] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831217] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1135612244
[2019-01-07 19:16:44.831230] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.831232] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831242] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.831250] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.831250] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.831258] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.831293] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.831299] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.831313] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.831321] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831339] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831347] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x789 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.831361] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.831370] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.831376] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831389] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.831405] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.831422] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.831425] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831437] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831453] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.831462] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.831461] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.831485] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.831489] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.831516] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831512] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831532] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.831534] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831543] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.831560] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.831568] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831592] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.831597] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.831602] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.831611] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.831622] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831637] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831645] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.831657] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.831659] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.831671] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.831680] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831682] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.831698] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.831700] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831717] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.831725] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.831747] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831758] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.831773] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831775] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.831789] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.831834] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.831790] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.831850] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831869] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831876] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.831892] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.831902] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x896 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.831918] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.831922] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.831944] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831941] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831961] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.831971] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.831972] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.831985] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832000] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.832008] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.832007] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.832029] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832030] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832045] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.832056] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.832060] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832083] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.832106] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.832118] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832143] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x897 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.832145] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.832159] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.832170] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.832176] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832193] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832200] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.832214] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.832223] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.832228] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832240] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.832248] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.832275] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.832277] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832300] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.832306] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.832316] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832322] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.832336] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832340] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.832363] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.832366] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832380] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832397] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.832404] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.832424] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.832441] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.832459] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x371 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.832460] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.832477] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.832478] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832506] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.832513] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.832522] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832530] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.832545] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa80 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.832546] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832563] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.832567] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832580] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832598] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.832593] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.832608] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.832635] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832647] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.832651] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x476 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.832669] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.832666] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.832689] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832696] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.832717] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832721] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.832739] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.832737] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.832751] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.832760] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832768] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832792] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x370 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.832794] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.832812] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.832811] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.832829] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832832] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832845] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.832857] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832879] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.832878] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.832908] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.832920] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1353889563
[2019-01-07 19:16:44.832931] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.832933] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.832942] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.832950] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.832951] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.832958] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.832977] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832997] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.833013] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833011] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.833032] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.833030] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.833044] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.833060] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833054] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833075] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa25 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.833084] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.833091] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.833098] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833111] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.833120] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.833142] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.833145] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833169] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.833167] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833185] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833187] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.833201] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.833213] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.833231] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833230] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.833246] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.833257] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.833259] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.833272] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833289] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.833285] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833308] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.833325] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x789 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.833342] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.833344] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.833370] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833369] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833404] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.833419] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.833419] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x475 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.833434] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833443] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.833451] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.833470] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.833471] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833487] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833499] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833505] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.833519] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.833541] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.833542] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.833582] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.833585] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833598] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833616] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.833628] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.833648] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833644] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.833664] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.833673] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.833673] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.833690] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833722] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833728] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.833746] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.833746] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833772] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.833778] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.833810] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833819] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.833838] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x898 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.833836] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.833850] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.833870] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833863] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833887] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.833896] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.833900] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.833911] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833918] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.833926] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.833938] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.833939] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833964] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.833984] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833974] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.834013] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.834013] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.834047] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.834048] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834064] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834083] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.834073] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834093] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.834109] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.834116] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834140] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x372 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.834150] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.834169] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.834176] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.834196] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.834201] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834213] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834226] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa81 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.834236] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.834237] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.834252] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834254] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.834270] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834273] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.834288] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.834295] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.834303] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834306] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.834316] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834319] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x477 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.834329] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.834328] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834347] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.834359] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.834377] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834376] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.834401] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.834408] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.834421] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.834426] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834437] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834460] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.834464] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x897 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.834469] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.834495] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834491] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.834513] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.834514] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834555] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.834570] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.834572] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.834579] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394995644
[2019-01-07 19:16:44.834593] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.834593] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.834613] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.834614] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834624] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.834639] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.834650] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.834664] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.834676] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.834681] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834703] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834710] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.834726] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.834735] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.834745] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834761] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.834772] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa26 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.834780] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834788] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.834809] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.834814] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834828] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.834839] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.834843] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834850] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.834864] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834882] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.834882] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.834899] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834919] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.834925] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.834929] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.834944] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.834958] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834977] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834983] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.834999] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.834998] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835015] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835017] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.835035] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.835045] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.835075] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.835083] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.835090] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835105] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.835115] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.835120] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835125] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.835148] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835151] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa80 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.835163] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.835167] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.835172] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.835196] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835192] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835211] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.835221] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.835224] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.835241] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.835247] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.835269] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835267] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835287] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.835297] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.835306] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x371 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.835312] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835325] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.835335] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.835340] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835345] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.835361] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835368] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835388] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.835400] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.835429] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.835442] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.835458] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.835459] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.835474] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835474] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835498] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x899 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.835502] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.835508] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.835518] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.835527] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835550] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835556] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fe Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.835573] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.835583] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa25 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.835588] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835602] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.835611] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.835622] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.835624] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835651] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.835654] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.835671] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835670] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.835694] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835700] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.835717] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.835716] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835738] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835742] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.835756] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.835766] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.835783] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835784] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x476 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.835797] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x373 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.835807] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.835835] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.835806] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.835850] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835868] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835880] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa82 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.835892] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.835894] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835907] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835915] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.835923] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.835939] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.835941] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.835954] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835956] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835970] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x478 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.835980] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.836017] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.836047] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836059] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.836079] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.836083] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.836106] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.836107] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836131] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836141] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.836156] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.836158] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.836174] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.836181] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836195] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836204] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836217] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.836221] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.836269] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.836268] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.836294] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.836301] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.836310] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836317] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1373794485
[2019-01-07 19:16:44.836331] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.836341] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.836348] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.836363] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.836364] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.836378] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.836404] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836429] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.836441] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.836455] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836457] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.836480] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836487] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.836505] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.836506] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836531] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836531] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.836557] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa27 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.836573] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.836577] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.836594] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.836598] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836612] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836633] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.836641] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.836648] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.836666] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.836682] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836703] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.836731] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836737] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.836758] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.836763] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.836774] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.836793] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836819] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836802] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836848] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.836860] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.836877] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.836889] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.836901] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.836904] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836919] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836930] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x360 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.836938] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x898 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.836951] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.836952] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.836960] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836981] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.836992] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.836995] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.837003] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837017] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837023] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.837036] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.837035] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.837048] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.837056] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837068] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837084] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.837086] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.837101] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.837119] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837114] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.837136] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.837138] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837146] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.837168] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.837174] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.837186] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.837204] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837223] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837240] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.837287] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.837305] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.837318] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837336] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837352] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.837191] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837390] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.837391] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.837426] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.837426] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.837452] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837464] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837490] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.837489] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa81 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.837501] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.837512] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.837517] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837535] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837541] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.837558] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.837566] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.837582] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.837585] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.837613] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837609] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837641] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.837645] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x372 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.837651] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.837662] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.837668] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837687] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837693] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ff Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.837710] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.837708] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837725] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837727] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.837741] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x290 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.837751] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.837774] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.837781] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.837791] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.837803] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837806] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837829] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.837834] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.837840] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.837857] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.837866] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837884] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837890] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.837906] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.837916] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa26 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.837921] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837935] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.837944] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x374 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.837956] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.837957] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837982] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.837988] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.837997] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838004] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.838019] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa83 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.838019] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838037] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.838040] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838053] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838065] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.838071] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.838094] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.838108] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x477 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.838121] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838139] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x479 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.838149] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.838147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.838171] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838180] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.838195] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838202] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838216] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.838233] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.838232] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.838245] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.838258] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838266] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838288] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.838298] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.838320] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838321] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.838335] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.838338] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.838353] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838369] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.838383] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.838388] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.838412] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.838415] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1285124113
[2019-01-07 19:16:44.838433] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.838445] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.838438] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838463] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.838472] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.838468] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838494] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.838502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.838528] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838542] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.838562] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.838548] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.838576] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838593] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.838613] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838617] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.838633] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa28 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.838637] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.838643] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.838669] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838687] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.838687] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838696] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.838722] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.838728] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.838741] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.838749] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838765] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838773] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.838785] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.838784] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838805] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838807] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.838821] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.838831] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.838849] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838848] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.838863] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.838865] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.838874] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.838888] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838907] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.838924] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838922] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.838964] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.838965] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.838981] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.838982] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839000] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839016] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.839019] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.839025] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.839040] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.839046] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839063] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839069] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.839087] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.839086] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839113] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.839123] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.839144] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839161] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.839159] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.839171] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.839181] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.839187] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839201] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839207] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.839217] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.839225] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x899 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.839232] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839243] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.839249] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.839253] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839258] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.839273] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.839282] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.839283] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.839300] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839300] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839324] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.839325] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.839341] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.839342] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.839358] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839357] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839376] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x600 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.839383] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.839385] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.839424] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.839430] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839450] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839457] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x291 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.839472] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.839481] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fe Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.839498] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.839504] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.839528] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839525] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839547] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839553] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.839570] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.839571] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.839587] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839604] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.839612] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.839614] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.839628] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839631] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.839643] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x375 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.839646] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839652] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.839673] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839690] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.839692] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.839708] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839731] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa84 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.839733] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.839749] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.839752] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.839767] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839767] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839784] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.839793] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.839806] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839815] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa82 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.839821] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x47a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.839835] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.839838] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.839859] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839877] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.839895] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839891] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.839915] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.839918] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.839939] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.839942] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839958] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839972] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x373 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.839980] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.839990] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.839988] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.840006] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.840009] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840022] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.840034] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840052] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.840062] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.840078] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.840091] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840091] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.840114] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa27 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.840114] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.840132] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.840133] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840143] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840164] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x478 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.840166] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.840187] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.840190] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.840208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840234] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840239] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840260] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.840279] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.840292] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.840300] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840302] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.840319] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.840333] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840339] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.840358] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.840363] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.840367] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840382] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.840386] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840412] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.840414] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840440] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x360 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.840451] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.840450] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.840462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840475] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.840486] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.840491] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840497] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.840511] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840512] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840535] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.840543] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.840559] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.840570] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840589] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.840598] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.840598] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.840606] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840625] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.840626] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840652] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.840654] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840680] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.840686] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.840691] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.840705] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.840707] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840729] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840736] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x290 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.840753] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.840762] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840761] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.840781] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ff Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.840784] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.840791] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.840809] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840809] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840835] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840843] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.840845] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.840860] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.840876] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.840875] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840895] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.840900] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840906] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840922] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.840931] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa83 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.840943] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.840953] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840961] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.840970] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x374 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.840980] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.840978] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.840992] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841010] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841010] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841021] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.841045] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.841043] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.841065] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.841068] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.841074] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841089] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841096] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa28 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.841118] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.841127] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841125] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x479 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.841146] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.841150] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.841156] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.841173] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841175] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841199] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841202] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841229] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.841249] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.841267] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.841270] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.841290] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.841289] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841309] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841341] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.841356] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.841366] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841371] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.841390] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.841388] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841417] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.841418] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841447] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.841448] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.841466] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.841467] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.841483] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841484] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841502] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841504] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.841521] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.841521] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.841534] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841556] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.841565] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.841564] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.841576] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841589] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.841599] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.841604] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841621] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.841625] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841631] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841651] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.841658] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.841676] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.841688] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841693] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.841706] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x291 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.841709] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.841716] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.841726] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841726] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841760] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x600 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.841760] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.841776] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.841779] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.841785] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841800] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841801] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841811] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.841840] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.841845] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.841859] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.841862] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.841880] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841882] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841898] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa84 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.841903] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841908] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.841920] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841922] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.841939] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.841950] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.841960] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841962] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x375 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.841977] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.841978] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.841994] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.841995] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842004] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842022] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842028] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.842048] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.842049] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.842057] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842081] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x47a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.842091] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.842090] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.842107] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842109] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.842126] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842126] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842148] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.842147] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842173] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.842189] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.842201] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.843510] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.843587] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.843712] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.843729] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843754] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.843770] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.843782] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843798] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa29 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.843808] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.843820] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.843830] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843844] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.843853] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.843863] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843877] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.843885] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.843897] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.843907] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843921] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.843930] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.843939] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843953] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.843965] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.843977] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.843986] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844002] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.844010] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.844020] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844033] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.844044] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.844058] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.844068] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844093] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.844102] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.844113] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844128] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x601 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.844137] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.844131] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa29 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.844157] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.844158] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-1 returned 0
[2019-01-07 19:16:44.844169] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844184] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.844191] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x980 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.844194] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-0 returned 0
[2019-01-07 19:16:44.844215] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.844216] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.844234] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844253] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.844259] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-0': avail_percent is: 22.00 and avail_space is: 9153228115968 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844267] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.844285] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.844293] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.844302] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844305] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-3 returned 0
[2019-01-07 19:16:44.844334] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.844336] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa85 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.844350] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-12 returned 0
[2019-01-07 19:16:44.844357] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.844374] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844379] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.844391] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7ea Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.844398] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-6 returned 0
[2019-01-07 19:16:44.844414] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.844429] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.844435] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.844439] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844460] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-4 returned 0
[2019-01-07 19:16:44.844472] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.844475] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-1 returned 0
[2019-01-07 19:16:44.844482] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.844495] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844499] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-1': avail_percent is: 23.00 and avail_space is: 9570127962112 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844510] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.844524] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.844532] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.844535] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223621823
[2019-01-07 19:16:44.844552] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.844556] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-7 returned 0
[2019-01-07 19:16:44.844560] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.844589] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844589] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-2 returned 0
[2019-01-07 19:16:44.844601] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 2 failed: Input/output error
[2019-01-07 19:16:44.844620] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-2': avail_percent is: 23.00 and avail_space is: 9299665338368 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844645] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.844664] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.844665] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-10 returned 0
[2019-01-07 19:16:44.844683] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1087134360
[2019-01-07 19:16:44.844692] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.844694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.844699] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.844715] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844717] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-9 returned 0
[2019-01-07 19:16:44.844734] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 5 failed: Input/output error
[2019-01-07 19:16:44.844734] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-3 returned 0
[2019-01-07 19:16:44.844757] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-3': avail_percent is: 23.00 and avail_space is: 9231467974656 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844764] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844778] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.844786] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1085903184
[2019-01-07 19:16:44.844786] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x980 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.844793] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.844807] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.844810] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-15 returned 0
[2019-01-07 19:16:44.844816] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844837] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 9 failed: Input/output error
[2019-01-07 19:16:44.844844] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x601 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.844862] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844859] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-13 returned 0
[2019-01-07 19:16:44.844873] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.844888] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223600709
[2019-01-07 19:16:44.844891] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-4 returned 0
[2019-01-07 19:16:44.844895] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.844912] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.844919] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-4': avail_percent is: 19.00 and avail_space is: 7846502244352 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844920] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844943] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 21 failed: Input/output error
[2019-01-07 19:16:44.844955] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.844968] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844969] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-16 returned 0
[2019-01-07 19:16:44.844978] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.844989] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1240589523
[2019-01-07 19:16:44.845000] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.844989] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-5 returned 0
[2019-01-07 19:16:44.845007] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.845022] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845027] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-5': avail_percent is: 19.00 and avail_space is: 7783764815872 and avail_inodes is: 99.00
[2019-01-07 19:16:44.845031] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 29 failed: Input/output error
[2019-01-07 19:16:44.845058] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa85 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.845065] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845085] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.845087] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-18 returned 0
[2019-01-07 19:16:44.845117] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7ea Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.845125] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1319971604
[2019-01-07 19:16:44.845127] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-19 returned 0
[2019-01-07 19:16:44.845159] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-6 returned 0
[2019-01-07 19:16:44.845160] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.845177] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.845183] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-6': avail_percent is: 22.00 and avail_space is: 9016154447872 and avail_inodes is: 99.00
[2019-01-07 19:16:44.845187] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845199] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 31 failed: Input/output error
[2019-01-07 19:16:44.845211] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.845222] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-21 returned 0
[2019-01-07 19:16:44.845232] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845240] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.845243] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.845255] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1413476598
[2019-01-07 19:16:44.845257] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-22 returned 0
[2019-01-07 19:16:44.845268] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-7 returned 0
[2019-01-07 19:16:44.845269] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.845279] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-7': avail_percent is: 22.00 and avail_space is: 9128420376576 and avail_inodes is: 99.00
[2019-01-07 19:16:44.845282] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.845293] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845311] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 35 failed: Input/output error
[2019-01-07 19:16:44.845338] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845347] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.845355] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1252560672
[2019-01-07 19:16:44.845361] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.845367] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.845375] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845384] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 40 failed: Input/output error
[2019-01-07 19:16:44.845416] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845429] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.845437] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394291141
[2019-01-07 19:16:44.845443] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.845450] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.845457] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845466] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 68 failed: Input/output error
[2019-01-07 19:16:44.845490] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845499] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.845507] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1080394250
[2019-01-07 19:16:44.845513] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.845520] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.845527] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845536] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 124 failed: Input/output error
[2019-01-07 19:16:44.845558] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845566] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.845574] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1277578615
[2019-01-07 19:16:44.845580] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.845589] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.845597] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845605] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 130 failed: Input/output error
[2019-01-07 19:16:44.845627] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845636] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.845643] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1342729215
[2019-01-07 19:16:44.845650] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.845656] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.845663] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845671] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 139 failed: Input/output error
[2019-01-07 19:16:44.845693] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845701] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.845708] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1288142377
[2019-01-07 19:16:44.845715] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.845721] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.845728] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845737] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 152 failed: Input/output error
[2019-01-07 19:16:44.845758] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845766] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.845773] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1374046734
[2019-01-07 19:16:44.845780] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.845786] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.845793] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845802] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 168 failed: Input/output error
[2019-01-07 19:16:44.845823] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845834] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.845847] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1111239957
[2019-01-07 19:16:44.845854] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.845860] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.845867] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845876] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 170 failed: Input/output error
[2019-01-07 19:16:44.845902] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845911] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.845918] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1135612244
[2019-01-07 19:16:44.845925] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.845931] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.845939] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845949] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 173 failed: Input/output error
[2019-01-07 19:16:44.845971] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845980] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.845987] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1353889563
[2019-01-07 19:16:44.845996] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.846002] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.846009] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846018] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 181 failed: Input/output error
[2019-01-07 19:16:44.846039] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846048] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.846055] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394995644
[2019-01-07 19:16:44.846061] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.846067] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.846074] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846085] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 183 failed: Input/output error
[2019-01-07 19:16:44.846107] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846115] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.846123] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1373794485
[2019-01-07 19:16:44.846129] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.846135] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.846142] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846151] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 184 failed: Input/output error
[2019-01-07 19:16:44.846172] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846180] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.846187] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1285124113
[2019-01-07 19:16:44.846194] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.846200] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.846207] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846215] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 185 failed: Input/output error
[2019-01-07 19:16:44.846223] D [MSGID: 0] [shard.c:720:shard_common_inode_write_failure_unwind] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846234] D [MSGID: 0] [defaults.c:1352:default_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-quick-read returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846244] D [MSGID: 0] [defaults.c:1352:default_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-open-behind returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846254] D [MSGID: 0] [md-cache.c:2715:mdc_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-md-cache returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846264] D [MSGID: 0] [defaults.c:1352:default_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846274] D [MSGID: 0] [io-stats.c:2528:io_stats_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846284] W [fuse-bridge.c:1441:fuse_err_cbk] 0-glusterfs-fuse: 1373: FALLOCATE() ERR => -1 (Input/output error)
[2019-01-07 19:16:44.846298] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 16/16 
[2019-01-07 19:16:44.846630] T [fuse-bridge.c:2652:fuse_flush] 0-glusterfs-fuse: 1374: FLUSH 0x7f6748005418
[2019-01-07 19:16:44.846652] T [MSGID: 0] [fuse-bridge.c:2631:fuse_flush_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.846669] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.846679] T [MSGID: 0] [io-stats.c:3049:io_stats_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.846689] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: FLUSH scheduled as normal fop
[2019-01-07 19:16:44.846710] T [MSGID: 0] [defaults.c:1722:default_flush_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.846735] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.846753] T [MSGID: 0] [defaults.c:1722:default_flush_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.846765] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.846777] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.846792] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.846805] T [MSGID: 0] [read-ahead.c:602:ra_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.846819] T [MSGID: 0] [write-behind.c:1969:wb_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.846832] T [MSGID: 0] [shard.c:4603:shard_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846847] T [MSGID: 0] [dht-inode-read.c:836:dht_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.846869] T [MSGID: 0] [afr-common.c:3645:afr_flush_wrapper] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.846890] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 124, payload: 36, rpc hdr: 88
[2019-01-07 19:16:44.846920] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.846936] T [MSGID: 0] [afr-common.c:3645:afr_flush_wrapper] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.846953] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 124, payload: 36, rpc hdr: 88
[2019-01-07 19:16:44.846977] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.846991] T [MSGID: 0] [afr-common.c:3645:afr_flush_wrapper] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.847007] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 124, payload: 36, rpc hdr: 88
[2019-01-07 19:16:44.847030] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847175] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.847193] T [MSGID: 0] [client-lk.c:385:delete_granted_locks_owner] 0-gv1-client-1: Number of locks cleared=0
[2019-01-07 19:16:44.847205] T [MSGID: 0] [client-rpc-fops_v2.c:707:client4_0_flush_cbk] 0-gv1-client-1: deleting locks of owner (ac824f7f6618b134) returned 0
[2019-01-07 19:16:44.847213] T [MSGID: 0] [client-rpc-fops_v2.c:721:client4_0_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned 0
[2019-01-07 19:16:44.847236] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.847256] T [MSGID: 0] [client-lk.c:385:delete_granted_locks_owner] 0-gv1-client-0: Number of locks cleared=0
[2019-01-07 19:16:44.847268] T [MSGID: 0] [client-rpc-fops_v2.c:707:client4_0_flush_cbk] 0-gv1-client-0: deleting locks of owner (ac824f7f6618b134) returned 0
[2019-01-07 19:16:44.847276] T [MSGID: 0] [client-rpc-fops_v2.c:721:client4_0_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned 0
[2019-01-07 19:16:44.847278] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847296] T [MSGID: 0] [client-lk.c:385:delete_granted_locks_owner] 0-gv1-client-2: Number of locks cleared=0
[2019-01-07 19:16:44.847308] T [MSGID: 0] [client-rpc-fops_v2.c:707:client4_0_flush_cbk] 0-gv1-client-2: deleting locks of owner (ac824f7f6618b134) returned 0
[2019-01-07 19:16:44.847315] T [MSGID: 0] [client-rpc-fops_v2.c:721:client4_0_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned 0
[2019-01-07 19:16:44.847323] T [MSGID: 0] [afr-common.c:3622:afr_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.847332] T [MSGID: 0] [dht-inode-read.c:770:dht_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:44.847339] T [MSGID: 0] [shard.c:4595:shard_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned 0
[2019-01-07 19:16:44.847347] T [MSGID: 0] [defaults.c:1109:default_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-write-behind returned 0
[2019-01-07 19:16:44.847355] T [MSGID: 0] [read-ahead.c:565:ra_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-read-ahead returned 0
[2019-01-07 19:16:44.847362] T [MSGID: 0] [defaults.c:1109:default_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-open-behind returned 0
[2019-01-07 19:16:44.847370] T [MSGID: 0] [defaults.c:1109:default_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0
[2019-01-07 19:16:44.847379] T [MSGID: 0] [io-stats.c:2356:io_stats_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.847386] T [fuse-bridge.c:1431:fuse_err_cbk] 0-glusterfs-fuse: 1374: FLUSH() ERR => 0
[2019-01-07 19:16:44.847403] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 16/16 
[2019-01-07 19:16:44.847424] T [fuse-bridge.c:2690:fuse_release] 0-glusterfs-fuse: 1375: RELEASE 0x7f6748005418
[2019-01-07 19:16:44.847446] T [MSGID: 0] [client-rpc-fops.c:3194:client_fdctx_destroy] 0-gv1-client-1: sending release on fd
[2019-01-07 19:16:44.847460] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 108, payload: 28, rpc hdr: 80
[2019-01-07 19:16:44.847482] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2b Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.847495] T [MSGID: 0] [client-rpc-fops.c:3194:client_fdctx_destroy] 0-gv1-client-2: sending release on fd
[2019-01-07 19:16:44.847505] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 108, payload: 28, rpc hdr: 80
[2019-01-07 19:16:44.847520] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x420 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847530] T [MSGID: 0] [client-rpc-fops.c:3194:client_fdctx_destroy] 0-gv1-client-0: sending release on fd
[2019-01-07 19:16:44.847539] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 108, payload: 28, rpc hdr: 80
[2019-01-07 19:16:44.847557] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x790 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.847573] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 16/16 
[2019-01-07 19:16:44.847707] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2b Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.847770] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x420 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847863] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x790 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:48.643531] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-0: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643566] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-1: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643581] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-3: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643592] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-4: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643600] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-6: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643616] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-7: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643627] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-9: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643640] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-10: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643651] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-12: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643660] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-13: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643669] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-15: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643677] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-16: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643688] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-18: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643699] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-19: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643711] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-21: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643722] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-22: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:52.769158] T [fuse-bridge.c:3273:fuse_statfs_resume] 0-glusterfs-fuse: 1376: STATFS
[2019-01-07 19:16:52.769190] T [MSGID: 0] [fuse-bridge.c:3276:fuse_statfs_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from fuse to meta-autoload
[2019-01-07 19:16:52.769209] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from meta-autoload to gv1
[2019-01-07 19:16:52.769224] T [MSGID: 0] [io-stats.c:3035:io_stats_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:52.769239] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: STATFS scheduled as fast fop
[2019-01-07 19:16:52.769264] T [MSGID: 0] [defaults.c:2275:default_statfs_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:52.769296] T [MSGID: 0] [md-cache.c:1080:mdc_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:52.769312] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:52.769325] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:52.769336] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:52.769348] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:52.769362] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:52.769372] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:52.769383] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:52.769407] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:52.769432] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:52.769450] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.769477] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x791 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770375] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49156: ping timer event already removed
[2019-01-07 19:16:52.770407] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.770414] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x791 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770440] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x792 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770445] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-0 returned 0
[2019-01-07 19:16:52.770455] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:52.770484] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.770506] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770674] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49156: ping timer event already removed
[2019-01-07 19:16:52.770687] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x792 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770699] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.770712] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-0: Ping latency is 0ms
[2019-01-07 19:16:52.770733] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2d Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770747] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:52.770769] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:52.770782] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.770798] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770805] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.770844] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-1 returned 0
[2019-01-07 19:16:52.770859] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-0 returned 0
[2019-01-07 19:16:52.770867] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2d Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770901] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-1: Ping latency is 0ms
[2019-01-07 19:16:52.770978] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49157: ping timer event already removed
[2019-01-07 19:16:52.770999] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771025] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d3 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.771039] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:52.771053] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771076] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771127] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.771150] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-3 returned 0
[2019-01-07 19:16:52.771185] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d3 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.771209] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-3: Ping latency is 0ms
[2019-01-07 19:16:52.771216] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49157: ping timer event already removed
[2019-01-07 19:16:52.771236] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771260] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771273] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:52.771288] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:52.771305] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771326] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771391] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771451] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-4: Ping latency is 0ms
[2019-01-07 19:16:52.771486] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771490] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49158: ping timer event already removed
[2019-01-07 19:16:52.771520] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-4 returned 0
[2019-01-07 19:16:52.771522] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771544] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-1 returned 0
[2019-01-07 19:16:52.771555] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cf Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771569] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:52.771581] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771600] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.771603] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771617] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-6 returned 0
[2019-01-07 19:16:52.771694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cf Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771704] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-6: Ping latency is 0ms
[2019-01-07 19:16:52.771744] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49158: ping timer event already removed
[2019-01-07 19:16:52.771764] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771787] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96a Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.771799] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:52.771814] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:52.771829] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771848] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.771932] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x96a Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.771949] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-7: Ping latency is 0ms
[2019-01-07 19:16:52.771983] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49159: ping timer event already removed
[2019-01-07 19:16:52.772002] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772003] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.772023] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-7 returned 0
[2019-01-07 19:16:52.772028] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x787 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.772039] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-2 returned 0
[2019-01-07 19:16:52.772052] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:52.772071] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.772097] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772159] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.772182] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-9 returned 0
[2019-01-07 19:16:52.772242] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49159: ping timer event already removed
[2019-01-07 19:16:52.772263] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772289] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa22 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772293] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x787 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.772329] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:52.772326] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-9: Ping latency is 0ms
[2019-01-07 19:16:52.772356] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:52.772375] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.772405] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772437] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-10 returned 0
[2019-01-07 19:16:52.772456] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-3 returned 0
[2019-01-07 19:16:52.772481] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa22 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772494] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-10: Ping latency is 0ms
[2019-01-07 19:16:52.772601] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49156: ping timer event already removed
[2019-01-07 19:16:52.772627] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772633] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772647] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-12 returned 0
[2019-01-07 19:16:52.772661] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772678] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:52.772698] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.772711] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772727] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-12: Ping latency is 0ms
[2019-01-07 19:16:52.772727] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x602 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.772915] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49156: ping timer event already removed
[2019-01-07 19:16:52.772947] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772949] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x602 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.772976] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-13 returned 0
[2019-01-07 19:16:52.772981] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x603 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.772997] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-4 returned 0
[2019-01-07 19:16:52.773008] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:52.773032] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:52.773052] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.773057] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x603 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.773072] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-13: Ping latency is 0ms
[2019-01-07 19:16:52.773094] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x981 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773284] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49157: ping timer event already removed
[2019-01-07 19:16:52.773312] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.773316] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x981 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773337] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-15 returned 0
[2019-01-07 19:16:52.773348] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x982 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773367] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:52.773398] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.773410] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x982 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773426] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-15: Ping latency is 0ms
[2019-01-07 19:16:52.773431] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773617] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49157: ping timer event already removed
[2019-01-07 19:16:52.773645] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.773652] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773674] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e7 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773674] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-16 returned 0
[2019-01-07 19:16:52.773701] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-5 returned 0
[2019-01-07 19:16:52.773702] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:52.773730] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:52.773750] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.773762] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e7 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773776] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-16: Ping latency is 0ms
[2019-01-07 19:16:52.773788] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa86 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.773971] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49158: ping timer event already removed
[2019-01-07 19:16:52.773999] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.774001] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa86 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.774023] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-18 returned 0
[2019-01-07 19:16:52.774036] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa87 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.774055] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:52.774073] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.774101] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa87 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.774101] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7eb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774112] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-18: Ping latency is 0ms
[2019-01-07 19:16:52.774312] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49158: ping timer event already removed
[2019-01-07 19:16:52.774338] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7eb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774339] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.774353] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-19 returned 0
[2019-01-07 19:16:52.774368] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-6 returned 0
[2019-01-07 19:16:52.774379] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7ec Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774405] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:52.774431] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:52.774450] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.774480] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7ec Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774493] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-19: Ping latency is 0ms
[2019-01-07 19:16:52.774489] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774681] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49159: ping timer event already removed
[2019-01-07 19:16:52.774706] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.774711] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774736] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa23 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774734] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-21 returned 0
[2019-01-07 19:16:52.774764] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:52.774793] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.774823] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.774819] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa23 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774844] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-21: Ping latency is 0ms
[2019-01-07 19:16:52.774976] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49159: ping timer event already removed
[2019-01-07 19:16:52.774998] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.775002] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.775016] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-22 returned 0
[2019-01-07 19:16:52.775023] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x788 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.775036] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-7 returned 0
[2019-01-07 19:16:52.775048] T [MSGID: 0] [dht-common.c:6205:dht_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-dht returned 0
[2019-01-07 19:16:52.775058] T [MSGID: 0] [md-cache.c:1041:mdc_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-md-cache returned 0
[2019-01-07 19:16:52.775066] T [MSGID: 0] [defaults.c:1608:default_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-io-threads returned 0
[2019-01-07 19:16:52.775075] T [MSGID: 0] [io-stats.c:2415:io_stats_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1 returned 0
[2019-01-07 19:16:52.775089] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 96/96 
[2019-01-07 19:16:52.775102] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x788 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.775116] T [fuse-bridge.c:946:fuse_getattr_resume] 0-glusterfs-fuse: 1377: GETATTR 1 (/)
[2019-01-07 19:16:52.775120] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-22: Ping latency is 0ms
[2019-01-07 19:16:52.775153] T [MSGID: 0] [fuse-bridge.c:949:fuse_getattr_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from fuse to meta-autoload
[2019-01-07 19:16:52.775174] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from meta-autoload to gv1
[2019-01-07 19:16:52.775189] T [MSGID: 0] [io-stats.c:2803:io_stats_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:52.775203] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: STAT scheduled as fast fop
[2019-01-07 19:16:52.775225] T [MSGID: 0] [defaults.c:2034:default_stat_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:52.775253] T [MSGID: 0] [md-cache.c:519:mdc_inode_iatt_get] 0-md-cache: iatt cache not valid for (00000000-0000-0000-0000-000000000001)
[2019-01-07 19:16:52.775265] T [MSGID: 0] [md-cache.c:1269:mdc_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:52.775274] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:52.775283] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:52.775292] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:52.775299] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:52.775312] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:52.775322] T [MSGID: 0] [write-behind.c:2060:wb_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:52.775332] T [MSGID: 0] [shard.c:1578:shard_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:52.775345] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:52.775359] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-0: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775372] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:52.775388] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775421] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.775433] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:52.775451] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-1: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775463] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:52.775473] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775500] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.775516] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:52.775538] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-2: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775552] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:52.775568] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775593] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.775609] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:52.775635] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-3: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775650] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:52.775665] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775690] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.775705] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:52.775727] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-4: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775741] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:52.775742] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.775759] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775774] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-1 returned 0
[2019-01-07 19:16:52.775788] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-0 returned 0
[2019-01-07 19:16:52.775795] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.775814] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:52.775837] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-5: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775850] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:52.775856] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.775866] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775883] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-4 returned 0
[2019-01-07 19:16:52.775941] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.775904] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x983 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.775979] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:52.775984] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-12 returned 0
[2019-01-07 19:16:52.775981] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-1 returned 0
[2019-01-07 19:16:52.776018] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-4 returned 0
[2019-01-07 19:16:52.776020] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-6: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.776036] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.776053] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:52.776057] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x983 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.776072] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.776092] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-7 returned 0
[2019-01-07 19:16:52.776094] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-15 returned 0
[2019-01-07 19:16:52.776108] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa88 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.776124] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-2 returned 0
[2019-01-07 19:16:52.776122] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-5 returned 0
[2019-01-07 19:16:52.776155] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:52.776181] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-7: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.776195] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:52.776209] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.776160] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.776233] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-10 returned 0
[2019-01-07 19:16:52.776242] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.776246] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-3 returned 0
[2019-01-07 19:16:52.776271] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa88 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.776290] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-18 returned 0
[2019-01-07 19:16:52.776307] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-6 returned 0
[2019-01-07 19:16:52.776348] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.776364] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-21 returned 0
[2019-01-07 19:16:52.776376] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-7 returned 0
[2019-01-07 19:16:52.776388] T [MSGID: 0] [dht-inode-read.c:312:dht_attr_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-dht returned 0
[2019-01-07 19:16:52.776419] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-shard returned 0
[2019-01-07 19:16:52.776435] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-write-behind returned 0
[2019-01-07 19:16:52.776494] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/lib64/libglusterfs.so.0(default_stat_cbk+0x15b) [0x7f6767cc172b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xc3ae) [0x7f67588c53ae] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888612 
[2019-01-07 19:16:52.776506] T [MSGID: 0] [md-cache.c:1231:mdc_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-md-cache returned 0
[2019-01-07 19:16:52.776519] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-io-threads returned 0
[2019-01-07 19:16:52.776534] T [MSGID: 0] [io-stats.c:2109:io_stats_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1 returned 0
[2019-01-07 19:16:52.776548] T [fuse-bridge.c:847:fuse_attr_cbk] 0-glusterfs-fuse: 1377: STAT() / => 1
[2019-01-07 19:16:52.776566] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 120/120