<div dir="ltr"><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Jose Sanchez</b> <span dir="ltr"><<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>></span><br>Date: 11 January 2018 at 22:05<br>Subject: Re: [Gluster-users] Creating cluster replica on 2 nodes 2 bricks each.<br>To: Nithya Balachandran <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>><br>Cc: gluster-users <<a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a>><br><br><br><div style="word-wrap:break-word"><div>Hi Nithya</div><div><br></div><div>Thanks for helping me with this, I understand now , but I have few questions. </div><div><br></div><div>When i had it setup in replica (just 2 nodes with 2 bricks) and tried to added , it failed. </div><span class="m_-8983901309846673932gmail-"><div><br></div><div><blockquote type="cite"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word"><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume add-brick scratch replica 2 gluster01ib:/gdata/brick2/scra<wbr>tch gluster02ib:/gdata/brick2/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">volume add-brick: failed: /gdata/brick2/scratch is already part of a volume</span></div></div></div></blockquote></div></div></blockquote><br></div><div>Did you try the add brick operation several times with the same bricks? If yes, that could be the cause as Gluster sets xattrs on the brick root directory.</div><div><br></div></span><div>and after that, I ran the status and info in it and on the status i get just the two brikcs</div><span class="m_-8983901309846673932gmail-"><div><br></div><div><blockquote type="cite"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word"><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick1/scra<wbr>tch 49152 49153 Y 3140 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick1/scra<wbr>tch 49153 49154 Y 2634</span></div></div></div></blockquote></div></div></blockquote><br></div></span><div>and on the info i get all 4 ( 2 x2) is this normal?? behavior?</div><div><br></div><div>So the brick count does not match for the same volume in the gluster volume status and gluster volume info commands? No, that is not normal.</div><span class="m_-8983901309846673932gmail-"><div><br></div><div><blockquote type="cite"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word"><div><div style="word-wrap:break-word"><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Bricks:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick1: gluster01ib:/gdata/brick1/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick2: gluster02ib:/gdata/brick1/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick3: gluster01ib:/gdata/brick2/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick4: gluster02ib:/gdata/brick2/scra<wbr>tch</span></div></div></div></div></div></blockquote></div></div></blockquote></div><div><br></div><div> </div></span><div>Now when i try to mount it , i still get only 14 tb and not 28? Am i doing something wrong? also when I start/stop services, cluster goes back to replicated mode from distributed-replicate</div><div><br></div><div><font face="times new roman, serif" size="4">If the fuse mount sees only 2 bricks , that would explain the 14TB.</font></div><div><br></div><div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">gluster01ib:/scratch 14T 34M 14T 1% /mnt/gluster_test</span></div></div><div><br></div><div>—— Gluster mount log file ——</div><div><br></div><div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:06:44.963043] I [MSGID: 114046] [client-handshake.c:1216:clien<wbr>t_setvolume_cbk] 0-scratch-client-1: Connected to scratch-client-1, attached to remote volume '/gdata/brick1/scratch'.</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:06:44.963065] I [MSGID: 114047] [client-handshake.c:1227:clien<wbr>t_setvolume_cbk] 0-scratch-client-1: Server and Client lk-version numbers are not same, reopening the fds</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:06:44.968291] I [MSGID: 114035] [client-handshake.c:202:client<wbr>_set_lk_version_cbk] 0-scratch-client-1: Server lk version = 1</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:06:44.968404] I [fuse-bridge.c:4147:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.22</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:06:44.968438] I [fuse-bridge.c:4832:fuse_graph<wbr>_sync] 0-fuse: switched to graph 0</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:06:44.969544] I [MSGID: 108031] [afr-common.c:2166:afr_local_d<wbr>iscovery_cbk] 0-scratch-replicate-0: selecting local read_child scratch-client-0</span></div></div><div><br></div><div>—— CLI Log File ——</div><div><br></div><div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 glusterfs]# tail cli.log</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:14.468122] I [socket.c:2403:socket_event_ha<wbr>ndler] 0-transport: disconnecting now</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:14.468737] I [cli-rpc-ops.c:817:gf_cli_get_<wbr>volume_cbk] 0-cli: Received resp to get vol: 0</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:14.469462] I [cli-rpc-ops.c:817:gf_cli_get_<wbr>volume_cbk] 0-cli: Received resp to get vol: 0</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:14.469530] I [input.c:31:cli_batch] 0-: Exiting with: 0</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:03:40.422568] I [cli.c:728:main] 0-cli: Started running gluster with version 3.8.15</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:03:40.430195] I [cli-cmd-volume.c:1828:cli_che<wbr>ck_gsync_present] 0-: geo-replication not installed</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:03:40.430492] I [MSGID: 101190] [event-epoll.c:628:event_dispa<wbr>tch_epoll_worker] 0-epoll: Started thread with index 1</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:03:40.430568] I [socket.c:2403:socket_event_ha<wbr>ndler] 0-transport: disconnecting now</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:03:40.485256] I [cli-rpc-ops.c:2244:gf_cli_set<wbr>_volume_cbk] 0-cli: Received resp to set</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 16:03:40.485497] I [input.c:31:cli_batch] 0-: Exiting with: 0</span></div></div><div><br></div><div>—— <span style="background-color:rgb(223,219,195)"><font color="#4d2f2d" face="Courier"><span style="font-size:19px">etc-glusterfs-glusterd.vol.<wbr>log — </span></font></span></div><div><br></div><div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 14:59:23.676814] I [MSGID: 106499] [glusterd-handler.c:4349:__glu<wbr>sterd_handle_status_volume] 0-management: Received status volume req for volume scratch</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:00:29.516071] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:09.872082] I [MSGID: 106482] [glusterd-brick-ops.c:447:__gl<wbr>usterd_handle_add_brick] 0-management: Received add brick req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:09.872128] I [MSGID: 106578] [glusterd-brick-ops.c:499:__gl<wbr>usterd_handle_add_brick] 0-management: replica-count is 2</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:09.876763] E [MSGID: 106451] [glusterd-utils.c:6207:gluster<wbr>d_is_path_in_use] 0-management: /gdata/brick2/scratch is already part of a volume [File exists]</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:09.876807] W [MSGID: 106122] [glusterd-mgmt.c:188:gd_mgmt_v<wbr>3_pre_validate_fn] 0-management: ADD-brick prevalidation failed.</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:09.876822] E [MSGID: 106122] [glusterd-mgmt.c:884:glusterd_<wbr>mgmt_v3_pre_validate] 0-management: Pre Validation failed for operation Add brick on local node</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:09.876834] E [MSGID: 106122] [glusterd-mgmt.c:2009:glusterd<wbr>_mgmt_v3_initiate_all_phases] 0-management: Pre Validation Failed</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.005881] I [run.c:191:runner_log] (-->/usr/lib64/glusterfs/3.8.1<wbr>5/xlator/mgmt/glusterd.so(+0x3<wbr>3045) [0x7f1066d15045] -->/usr/lib64/glusterfs/3.8.15<wbr>/xlator/mgmt/glusterd.so(+0xcb<wbr>d85) [0x7f1066dadd85] -->/lib64/libglusterfs.so.0(ru<wbr>nner_log+0x115) [0x7f10726491e5] ) 0-management: Ran script: /var/lib/glusterd/hooks/1/add-<wbr>brick/pre/S28Quota-enable-root<wbr>-xattr-heal.sh --volname=scratch --version=1 --volume-op=add-brick --gd-workdir=/var/lib/glusterd</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:15.982929] E [MSGID: 106451] [glusterd-utils.c:6207:gluster<wbr>d_is_path_in_use] 0-management: /gdata/brick2/scratch is already part of a volume [File exists]</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.005959] I [MSGID: 106578] [glusterd-brick-ops.c:1352:glu<wbr>sterd_op_perform_add_bricks] 0-management: replica-count is set 0</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div style="margin:0px;line-height:normal;font-family:Courier;color:rgb(77,47,45)"><span style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)">Atin, is this correct? It looks like it tries to add the bricks even though the prevalidation failed</span><span style="font-size:19px;background-color:rgb(223,219,195);font-variant-ligatures:no-common-ligatures"><br></span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.006018] I [MSGID: 106578] [glusterd-brick-ops.c:1362:glu<wbr>sterd_op_perform_add_bricks] 0-management: type is set 0, need to change it</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062001] I [MSGID: 106143] [glusterd-pmap.c:250:pmap_regi<wbr>stry_bind] 0-pmap: adding brick /gdata/brick2/scratch on port 49154</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062137] I [MSGID: 106143] [glusterd-pmap.c:250:pmap_regi<wbr>stry_bind] 0-pmap: adding brick /gdata/brick2/scratch.rdma on port 49155</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062673] E [MSGID: 106005] [glusterd-utils.c:4877:gluster<wbr>d_brick_start] 0-management: Unable to start brick gluster01ib:/gdata/brick2/scra<wbr>tch</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062715] E [MSGID: 106074] [glusterd-brick-ops.c:2493:glu<wbr>sterd_op_add_brick] 0-glusterd: Unable to add bricks</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062729] E [MSGID: 106123] [glusterd-mgmt.c:294:gd_mgmt_v<wbr>3_commit_fn] 0-management: Add-brick commit failed.</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062741] E [MSGID: 106123] [glusterd-mgmt.c:1427:glusterd<wbr>_mgmt_v3_commit] 0-management: Commit failed for operation Add brick on local node</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:16.062754] E [MSGID: 106123] [glusterd-mgmt.c:2018:glusterd<wbr>_mgmt_v3_initiate_all_phases] 0-management: Commit Op Failed</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:35.914090] I [MSGID: 106499] [glusterd-handler.c:4349:__glu<wbr>sterd_handle_status_volume] 0-management: Received status volume req for volume scratch</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:15.979236] I [MSGID: 106482] [glusterd-brick-ops.c:447:__gl<wbr>usterd_handle_add_brick] 0-management: Received add brick req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 15:01:15.979250] I [MSGID: 106578] [glusterd-brick-ops.c:499:__gl<wbr>usterd_handle_add_brick] 0-management: replica-count is 2</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">The message "I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req" repeated 3 times between [2018-01-10 15:00:29.516071] and [2018-01-10 15:01:39.652014]</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 16:16:42.776653] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-10 16:16:42.777614] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:45:09.023393] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:45:19.916301] I [MSGID: 106499] [glusterd-handler.c:4349:__glu<wbr>sterd_handle_status_volume] 0-management: Received status volume req for volume scratch</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:45:09.024217] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:10.172137] I [MSGID: 106499] [glusterd-handler.c:4349:__glu<wbr>sterd_handle_status_volume] 0-management: Received status volume req for volume scratch</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:14.468529] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div><div style="margin:0px;font-size:19px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[2018-01-11 15:54:14.469408] I [MSGID: 106488] [glusterd-handler.c:1537:__glu<wbr>sterd_handle_cli_get_volume] 0-management: Received get vol req</span></div></div><span class="m_-8983901309846673932gmail-"><div><br></div><div>Thanks</div><div><br></div><div>Jose</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><br><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div>------------------------------<wbr>---</div><div>Jose Sanchez</div><div>Center of Advanced Research Computing</div><div>Albuquerque, NM 87131</div><div><br></div></div>
</div>
<br></span><div><div class="m_-8983901309846673932gmail-h5"><div><blockquote type="cite"><div>On Jan 10, 2018, at 9:02 PM, Nithya Balachandran <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>> wrote:</div><br class="m_-8983901309846673932gmail-m_8996297448510270637Apple-interchange-newline"><div><div dir="ltr">Hi Jose,<div><br></div><div>Gluster is working as expected. The Distribute-replicated type just means that there are now 2 replica sets and files will be distributed across them. </div><pre class="m_-8983901309846673932gmail-m_8996297448510270637gmail-m_-8420281586111106388gmail-bz_comment_text m_-8983901309846673932gmail-m_8996297448510270637gmail-m_-8420281586111106388gmail-bz_wrap_comment_text" id="m_-8983901309846673932gmail-m_8996297448510270637gmail-m_-8420281586111106388gmail-comment_text_3" style="white-space:pre-wrap;word-wrap:break-word;width:50em"><font face="arial, helvetica, sans-serif">A volume of type Replicate (1xn where n is the number of bricks in the replica set) indicates there is no distribution (all files on the volume will be present on all the bricks in the volume).
</font></pre><div><font face="arial, helvetica, sans-serif"><span style="white-space:pre-wrap">A volume of type Distributed-Replicate indicates the volume is both distributed (as in files will only be created on one of the replicated sets) and replicated. So in the above example, a file will exist on either Brick1 and Brick2 or Brick3 and Brick4.</span> </font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><div>After the add brick, the volume will have a total capacity of 28TB and store 2 copies of every file. Let me know if that is not what you are looking for.</div></div><div><br></div><div><br></div><div>Regards,</div><div>Nithya</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 10 January 2018 at 20:40, Jose Sanchez <span dir="ltr"><<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div><br></div><div>Hi Nithya</div><div><br></div><div>This is what i have so far, I have peer both cluster nodes together as replica, from node 1A and 1B , now when i tried to add it , i get the error that it is already part of a volume. when i run the cluster volume info , i see that has switch to distributed-replica. </div><div><br></div><div>Thanks</div><div><br></div><div>Jose</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume status</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status of volume: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Gluster process TCP Port RDMA Port Online Pid</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick1/scra<wbr>tch 49152 49153 Y 3140 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick1/scra<wbr>tch 49153 49154 Y 2634 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on localhost N/A N/A Y 3132 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on gluster02ib N/A N/A Y 2626 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491webkit-block-placeholder"></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Task Status of Volume scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">There are no active volume tasks</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491webkit-block-placeholder"></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]#</span></div></div><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume info</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491webkit-block-placeholder"></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume Name: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Type: <b>Replicate</b></span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume ID: a6e20f7d-13ed-4293-ab8b-d783d1<wbr>748246</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status: Started</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Snapshot Count: 0</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Number of Bricks: 1 x 2 = 2</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Transport-type: tcp,rdma</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Bricks:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick1: gluster01ib:/gdata/brick1/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick2: gluster02ib:/gdata/brick1/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Options Reconfigured:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">performance.readdir-ahead: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">nfs.disable: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]#</span></div></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><br></div><div>------------------------------<wbr>-------</div><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume add-brick scratch replica 2 gluster01ib:/gdata/brick2/scra<wbr>tch gluster02ib:/gdata/brick2/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">volume add-brick: failed: /gdata/brick2/scratch is already part of a volume</span></div></div><br><div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume status</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status of volume: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Gluster process TCP Port RDMA Port Online Pid</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick1/scra<wbr>tch 49152 49153 Y 3140 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick1/scra<wbr>tch 49153 49154 Y 2634 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on gluster02ib N/A N/A Y 2626 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on localhost N/A N/A Y 3132 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491webkit-block-placeholder"></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Task Status of Volume scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">There are no active volume tasks</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491webkit-block-placeholder"></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume info</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491webkit-block-placeholder"></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume Name: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Type: <b>Distributed-Replicate</b></span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume ID: a6e20f7d-13ed-4293-ab8b-d783d1<wbr>748246</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status: Started</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Snapshot Count: 0</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Number of Bricks: 2 x 2 = 4</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Transport-type: tcp,rdma</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Bricks:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick1: gluster01ib:/gdata/brick1/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick2: gluster02ib:/gdata/brick1/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick3: gluster01ib:/gdata/brick2/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick4: gluster02ib:/gdata/brick2/scra<wbr>tch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Options Reconfigured:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">performance.readdir-ahead: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">nfs.disable: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# </span></div></div><span><div><br></div><div><br></div><div><br></div><div>------------------------------<wbr>--</div><div>Jose Sanchez</div><div>Center of Advanced Research Computing</div></span><div>Albuquerque, NM 87131-0001</div><div><a href="http://carc.unm.edu/" target="_blank">carc.unm.edu</a></div><div><br></div></div>
</div><div><div class="m_-8983901309846673932gmail-m_8996297448510270637h5">
<br><div><blockquote type="cite"><div>On Jan 9, 2018, at 9:04 PM, Nithya Balachandran <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>> wrote:</div><br class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491Apple-interchange-newline"><div><div dir="ltr">Hi,<div><br></div><div>Please let us know what commands you ran so far and the output of the <b>gluster volume info</b> command.</div><div><br></div><div>Thanks,</div><div>Nithya</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 9 January 2018 at 23:06, Jose Sanchez <span dir="ltr"><<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word">Hello<div><br></div><div>We are trying to setup Gluster for our project/scratch storage HPC machine using a replicated mode with 2 nodes, 2 bricks each (14tb each). </div><div><br></div><div>Our goal is to be able to have a replicated system between node 1 and 2 (A bricks) and add an additional 2 bricks (B bricks) from the 2 nodes. so we can have a total of 28tb replicated mode. </div><div><br></div><div>Node 1 [ (Brick A) (Brick B) ]</div><div>Node 2 [ (Brick A) (Brick B) ]</div><div>------------------------------<wbr>--------------</div><div><span class="m_-8983901309846673932gmail-m_8996297448510270637m_4198548956380517491m_-5268881903074545341Apple-tab-span" style="white-space:pre-wrap"> </span>14Tb + 14Tb = 28Tb</div><div><br></div><div>At this I was able to create the replica nodes between node 1 and 2 (brick A) but I’ve not been able to add to the replica together, Gluster switches to distributed replica when i add it with only 14Tb.</div><div><br></div><div>Any help will be appreciated.</div><div><br></div><div>Thanks</div><div><br></div><div>Jose</div><div><br></div><div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div>------------------------------<wbr>---</div><div>Jose Sanchez</div><div>Center of Advanced Research Computing</div><div>Albuquerque, NM 87131</div><div><a href="http://carc.unm.edu/" target="_blank">carc.unm.edu</a></div><div><br></div></div></div></div><br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></div><br></div>