<div dir="ltr">Hi Jose,<div><br></div><div>Gluster is working as expected. The Distribute-replicated type just means that there are now 2 replica sets and files will be distributed across them. </div><pre class="gmail-m_-8420281586111106388gmail-bz_comment_text gmail-m_-8420281586111106388gmail-bz_wrap_comment_text" id="gmail-m_-8420281586111106388gmail-comment_text_3" style="white-space:pre-wrap;word-wrap:break-word;width:50em"><font color="#000000" face="arial, helvetica, sans-serif">A volume of type Replicate (1xn where n is the number of bricks in the replica set) indicates there is no distribution  (all files on the volume will be present on all the bricks in the volume). 
 </font></pre><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="white-space:pre-wrap">A volume of type Distributed-Replicate indicates the volume is both distributed (as in files will only be created on one of the replicated sets) and replicated.  So in the above example, a file will exist on either Brick1 and Brick2 or Brick3 and Brick4.</span> </font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><div>After the add brick, the volume will have a total capacity of 28TB and store 2 copies of every file. Let me know if that is not what you are looking for.</div></div><div><br></div><div><br></div><div>Regards,</div><div>Nithya</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 10 January 2018 at 20:40, Jose Sanchez <span dir="ltr">&lt;<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div><br></div><div>Hi Nithya</div><div><br></div><div>This is what i have so far, I have peer both cluster nodes together as replica, from node 1A and 1B , now when i tried to add it , i get the error that it is already part of a volume. when i run the cluster volume info , i see that has switch to distributed-replica. </div><div><br></div><div>Thanks</div><div><br></div><div>Jose</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume status</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status of volume: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Gluster process                             TCP Port  RDMA Port  Online  Pid</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick1/<wbr>scratch     49152     49153      Y       3140 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick1/<wbr>scratch     49153     49154      Y       2634 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on localhost               N/A       N/A        Y       3132 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on gluster02ib             N/A       N/A        Y       2626 </span></div><p style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_4198548956380517491webkit-block-placeholder"></p><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Task Status of Volume scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">There are no active volume tasks</span></div><p style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_4198548956380517491webkit-block-placeholder"></p><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]#</span></div></div><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume info</span></div><p style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_4198548956380517491webkit-block-placeholder"></p><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume Name: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Type: <b>Replicate</b></span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume ID: a6e20f7d-13ed-4293-ab8b-<wbr>d783d1748246</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status: Started</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Snapshot Count: 0</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Number of Bricks: 1 x 2 = 2</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Transport-type: tcp,rdma</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Bricks:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick1: gluster01ib:/gdata/brick1/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick2: gluster02ib:/gdata/brick1/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Options Reconfigured:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">performance.readdir-ahead: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">nfs.disable: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]#</span></div></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><br></div><div>------------------------------<wbr>-------</div><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume add-brick scratch replica 2 gluster01ib:/gdata/brick2/<wbr>scratch gluster02ib:/gdata/brick2/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">volume add-brick: failed: /gdata/brick2/scratch is already part of a volume</span></div></div><br><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div><br></div><div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume status</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status of volume: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Gluster process                             TCP Port  RDMA Port  Online  Pid</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick1/<wbr>scratch     49152     49153      Y       3140 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick1/<wbr>scratch     49153     49154      Y       2634 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on gluster02ib             N/A       N/A        Y       2626 </span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on localhost               N/A       N/A        Y       3132 </span></div><p style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_4198548956380517491webkit-block-placeholder"></p><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Task Status of Volume scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">There are no active volume tasks</span></div><p style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_4198548956380517491webkit-block-placeholder"></p><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume info</span></div><p style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:20px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_4198548956380517491webkit-block-placeholder"></p><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume Name: scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Type: <b>Distributed-Replicate</b></span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume ID: a6e20f7d-13ed-4293-ab8b-<wbr>d783d1748246</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status: Started</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Snapshot Count: 0</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Number of Bricks: 2 x 2 = 4</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Transport-type: tcp,rdma</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Bricks:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick1: gluster01ib:/gdata/brick1/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick2: gluster02ib:/gdata/brick1/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick3: gluster01ib:/gdata/brick2/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick4: gluster02ib:/gdata/brick2/<wbr>scratch</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Options Reconfigured:</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">performance.readdir-ahead: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">nfs.disable: on</span></div><div style="margin:0px;font-size:17px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# </span></div></div><span class=""><div><br></div><div><br></div><div><br></div><div>------------------------------<wbr>--</div><div>Jose Sanchez</div><div>Center of Advanced Research Computing</div></span><div>Albuquerque, NM 87131-0001</div><div><a href="http://carc.unm.edu" target="_blank">carc.unm.edu</a></div><div><br></div></div>
</div><div><div class="h5">
<br><div><blockquote type="cite"><div>On Jan 9, 2018, at 9:04 PM, Nithya Balachandran &lt;<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>&gt; wrote:</div><br class="m_4198548956380517491Apple-interchange-newline"><div><div dir="ltr">Hi,<div><br></div><div>Please let us know what commands you ran so far and the output of the <b>gluster volume info</b> command.</div><div><br></div><div>Thanks,</div><div>Nithya</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 9 January 2018 at 23:06, Jose Sanchez <span dir="ltr">&lt;<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Hello<div><br></div><div>We are trying to setup Gluster for our project/scratch storage HPC machine using a replicated mode with 2 nodes, 2 bricks each (14tb each). </div><div><br></div><div>Our goal is to be able to have a replicated system between node 1 and 2 (A bricks) and add an additional 2 bricks (B bricks)  from the 2 nodes. so we can have a total of 28tb replicated mode. </div><div><br></div><div>Node 1 [ (Brick A) (Brick B) ]</div><div>Node 2 [ (Brick A) (Brick B) ]</div><div>------------------------------<wbr>--------------</div><div><span class="m_4198548956380517491m_-5268881903074545341Apple-tab-span" style="white-space:pre-wrap">                </span>14Tb + 14Tb = 28Tb</div><div><br></div><div>At this  I was able to create the replica nodes between node 1 and 2 (brick A) but I’ve not been able to add to the replica together, Gluster switches to distributed replica   when i add it with only 14Tb.</div><div><br></div><div>Any help will be appreciated.</div><div><br></div><div>Thanks</div><div><br></div><div>Jose</div><div><br></div><div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div>------------------------------<wbr>---</div><div>Jose Sanchez</div><div>Center of Advanced Research Computing</div><div>Albuquerque, NM 87131</div><div><a href="http://carc.unm.edu/" target="_blank">carc.unm.edu</a></div><div><br></div></div></div></div><br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></blockquote></div><br></div>