<div dir="ltr">Hi Jose,<div><br></div><div>Thanks for providing the volume info. You have 2 subvolumes. Data is replicated within the bricks of that subvolumes.</div><div>First one consisting of Node A's brick1 & Node B's brick1 and the second one consisting of Node A's brick2 and Node B's brick2.</div><div>You don't have the same data on all the 4 bricks. Data are distributed between these two subvolumes.</div><div>To remove the replica you can use the command</div><div>gluster volume remove-brick scratch replica 1 <span style="font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;color:rgb(77,47,45);font-family:Courier;font-size:14px;font-variant-ligatures:no-common-ligatures;background-color:rgb(223,219,195);float:none;display:inline">gluster02ib:/gdata/brick1/</span><wbr style="font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;color:rgb(77,47,45);font-family:Courier;font-size:14px;font-variant-ligatures:no-common-ligatures;background-color:rgb(223,219,195)"><span style="font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;color:rgb(77,47,45);font-family:Courier;font-size:14px;font-variant-ligatures:no-common-ligatures;background-color:rgb(223,219,195);float:none;display:inline">scratch <span style="color:rgb(77,47,45);font-family:Courier;font-size:14px;font-style:normal;font-variant-ligatures:no-common-ligatures;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(223,219,195);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">gluster02ib:/gdata/brick2/</span><wbr style="color:rgb(77,47,45);font-family:Courier;font-size:14px;font-style:normal;font-variant-ligatures:no-common-ligatures;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(223,219,195);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(77,47,45);font-family:Courier;font-size:14px;font-style:normal;font-variant-ligatures:no-common-ligatures;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(223,219,195);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">scratch </span></span>force<span style="color:rgb(77,47,45);font-family:Courier;font-size:14px;font-style:normal;font-variant-ligatures:no-common-ligatures;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(223,219,195);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"></span></div><div>So you will have one copy of data present from both the distributes.</div><div>Before doing this make sure "gluster volume heal scratch info" value is zero. So copies you retain will have the correct data.</div><div>After the remove-brick erase the data from the backend.</div><div>Then you can expand the volume by following the steps at [1].</div><div><br></div><div>[1] <a href="https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/#expanding-volumes">https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/#expanding-volumes</a></div><div><br></div><div>Regards,</div><div>Karthik</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 6, 2018 at 11:39 PM, Jose Sanchez <span dir="ltr"><<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>Hi Karthik</div><div><br></div><div>this is our configuration, is 2x2 =4 , they are all replicated , each brick has 14tb. we have 2 nodes A and B, each one with brick 1 and 2. </div><div><br></div><div>Node A (replicated A1 (14tb) and B1 (14tb) ) same with node B (Replicated A2 (14tb) and B2 (14tb)).</div><div><br></div><div>Do you think we need to degrade the node first before removing it. i believe the same copy of data is on all 4 bricks, we would like to keep one of them, and add the other bricks as extra space</div><div><br></div><div>Thanks for your help on this</div><div><br></div><div>Jose</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume info scratch</span></div><p style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:17px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_7358755837527423437webkit-block-placeholder"></p><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume Name: scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Type: Distributed-Replicate</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Volume ID: 23f1e4b1-b8e0-46c3-874a-<wbr>58b4728ea106</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status: Started</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Snapshot Count: 0</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Number of Bricks: 2 x 2 = 4</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Transport-type: tcp,rdma</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Bricks:</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick1: gluster01ib:/gdata/brick1/<wbr>scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick2: gluster02ib:/gdata/brick1/<wbr>scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick3: gluster01ib:/gdata/brick2/<wbr>scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick4: gluster02ib:/gdata/brick2/<wbr>scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Options Reconfigured:</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">performance.readdir-ahead: on</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">nfs.disable: on</span></div></div><div><br></div><div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">[root@gluster01 ~]# gluster volume status all</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Status of volume: scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Gluster process TCP Port RDMA Port Online Pid</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick1/<wbr>scratch 49152 49153 Y 1743 </span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick1/<wbr>scratch 49156 49157 Y 1732 </span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster01ib:/gdata/brick2/<wbr>scratch 49154 49155 Y 1738 </span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Brick gluster02ib:/gdata/brick2/<wbr>scratch 49158 49159 Y 1733 </span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on localhost N/A N/A Y 1728 </span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Self-heal Daemon on gluster02ib N/A N/A Y 1726 </span></div><p style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195);min-height:17px"><span style="font-variant-ligatures:no-common-ligatures"> </span><br class="m_7358755837527423437webkit-block-placeholder"></p><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">Task Status of Volume scratch</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">------------------------------<wbr>------------------------------<wbr>------------------</span></div><div style="margin:0px;font-size:14px;line-height:normal;font-family:Courier;color:rgb(77,47,45);background-color:rgb(223,219,195)"><span style="font-variant-ligatures:no-common-ligatures">There are no active volume tasks</span></div></div><br><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div>------------------------------<wbr>---</div><div>Jose Sanchez</div><div>Systems/Network Analyst 1</div><span class=""><div>Center of Advanced Research Computing</div><div><a href="https://maps.google.com/?q=1601+Central+Ave&entry=gmail&source=g">1601 Central Ave</a>.</div><div>MSC 01 1190</div><div>Albuquerque, NM 87131-0001</div><div><a href="http://carc.unm.edu" target="_blank">carc.unm.edu</a></div><div>575.636.4232</div></span></div>
</div><div><div class="h5">
<br><div><blockquote type="cite"><div>On Apr 6, 2018, at 3:49 AM, Karthik Subrahmanya <<a href="mailto:ksubrahm@redhat.com" target="_blank">ksubrahm@redhat.com</a>> wrote:</div><br class="m_7358755837527423437Apple-interchange-newline"><div><div dir="ltr">Hi Jose,<div><br></div><div>By switching into pure distribute volume you will lose availability if something goes bad.</div><div><br></div><div>I am guessing you have a nX2 volume.</div><div>If you want to preserve one copy of the data in all the distributes, you can do that by decreasing the replica count in the remove-brick operation.</div><div>If you have any inconsistency, heal them first using the "gluster volume heal <volname>" command and wait till the</div><div>"gluster volume heal <volname> info" output becomes zero, before removing the bricks, so that you will have the correct data.</div><div>If you do not want to preserve the data then you can directly remove the bricks.</div><div>Even after removing the bricks the data will be present in the backend of the removed bricks. You have to manually erase them (both data and .glusterfs folder).</div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">See [1] for more details on remove-brick.<span> </span></span><br></div><div><br></div><div>[1]. <a href="https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/#shrinking-volumes" target="_blank">https://docs.gluster.org/<wbr>en/latest/Administrator%<wbr>20Guide/Managing%20Volumes/#<wbr>shrinking-volumes</a></div><div><br></div><div>HTH,</div><div>Karthik</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 5, 2018 at 8:17 PM, Jose Sanchez <span dir="ltr"><<a href="mailto:josesanc@carc.unm.edu" target="_blank">josesanc@carc.unm.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div>We have a Gluster setup with 2 nodes (distributed replication) and we would like to switch it to the distributed mode. I know the data is duplicated between those nodes, what is the proper way of switching it to a distributed, we would like to double or gain the storage space on our gluster storage node. what happens with the data, do i need to erase one of the nodes? </div><div><br></div><div>Jose</div><div><br></div><div><br></div><div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div>------------------------------<wbr>---</div><div>Jose Sanchez</div><div>Systems/Network Analyst </div><div>Center of Advanced Research Computing</div><div><a href="https://maps.google.com/?q=1601+Central+Ave&entry=gmail&source=g">1601 Central Ave</a>.</div><div>MSC 01 1190</div><div>Albuquerque, NM 87131-0001</div><div><a href="http://carc.unm.edu/" target="_blank">carc.unm.edu</a></div><div>575.636.4232</div></div>
</div>
<br></div><br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></blockquote></div><br></div>