<div dir="ltr">All,<div><br></div><div>We recently added 4 new bricks to an establish distributed replicated volume.  The original volume was created via gdeploy as:</div><div><br></div><div>[volume1]<br>action=create<br>volname=tank<br>replica_count=2<br>force=yes<br>key=performance.parallel-readdir,network.inode-lru-limit,performance.md-cache-timeout,performance.cache-invalidation,performance.stat-prefetch,features.cache-invalidation-timeout,features.cache-invalidation,performance.cache-samba-metadata<br>value=on,500000,600,on,on,600,on,on<br>brick_dirs=/gluster_bricks/data1,/gluster_bricks/data2<br>ignore_errors=no<br></div><div><br></div><div>This created the volume as:</div><div><br></div><div># gluster vol status tank<br>Status of volume: tank<br>Gluster process                             TCP Port  RDMA Port  Online  Pid<br>------------------------------------------------------------------------------<br>Brick server1:/gluster_bricks/data1       49162     0          Y       20318<br>Brick server2:/gluster_bricks/data1       49166     0          Y       3432 <br>Brick server1:/gluster_bricks/data2       49163     0          Y       20323<br>Brick server2:/gluster_bricks/data2       49167     0          Y       3435<br></div><div>Self-heal Daemon on localhost               N/A       N/A        Y       25874<br>Self-heal Daemon on server2               N/A       N/A        Y       12536<br> <br>Task Status of Volume tank<br>------------------------------------------------------------------------------<br></div><div>There are no active volume tasks<br></div><div><br></div><div>I have read (<a href="https://docs.gluster.org/en/latest/Administrator%20Guide/Setting%20Up%20Volumes">https://docs.gluster.org/en/latest/Administrator%20Guide/Setting%20Up%20Volumes</a>) that the way one creates distributed replicated volumes is sensitive to replica-sets:</div><div><br></div><div><strong style="box-sizing:inherit;color:rgba(0,0,0,0.54);font-family:Roboto,&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif;font-size:17.6px">Note</strong><span style="color:rgba(0,0,0,0.54);font-family:Roboto,&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif;font-size:17.6px">: The number of bricks should be a multiple of the replica count for a distributed replicated volume. Also, the order in which bricks are specified has a great effect on data protection. Each replica_count consecutive bricks in the list you give will form a replica set, with all replica sets combined into a volume-wide distribute set. To make sure that replica-set members are not placed on the same node, list the first brick on every server, then the second brick on every server in the same order, and so on.</span><br></div><div><br></div><div>I just noticed that the way we added the new bricks we did not indicate a replica:<br></div><div><br></div><div>gluster volume add-brick tank server1:/gluster_bricks/data3 server1:/gluster_bricks/data4 force<br>gluster volume add-brick tank server1:/gluster_bricks/data5 server1:/gluster_bricks/data6 force<br>gluster volume add-brick tank server2:/gluster_bricks/data3 server2:/gluster_bricks/data4 force<br>gluster volume add-brick tank server2:/gluster_bricks/data5 server2:/gluster_bricks/data6 force<br></div><div><br></div><div>Which modified the volume as:</div><div><br></div><div># gluster vol status tank<br>Status of volume: tank<br>Gluster process                             TCP Port  RDMA Port  Online  Pid<br>------------------------------------------------------------------------------<br>Brick server1:/gluster_bricks/data1       49162     0          Y       20318<br>Brick server2:/gluster_bricks/data1       49166     0          Y       3432 <br>Brick server1:/gluster_bricks/data2       49163     0          Y       20323<br>Brick server2:/gluster_bricks/data2       49167     0          Y       3435 <br>Brick server1:/gluster_bricks/data3       49164     0          Y       4625 <br>Brick server1:/gluster_bricks/data4       49165     0          Y       4644 <br>Brick server1:/gluster_bricks/data5       49166     0          Y       5088 <br>Brick server1:/gluster_bricks/data6       49167     0          Y       5128 <br>Brick server2:/gluster_bricks/data3       49168     0          Y       22314<br>Brick server2:/gluster_bricks/data4       49169     0          Y       22345<br>Brick server2:/gluster_bricks/data5       49170     0          Y       22889<br>Brick server2:/gluster_bricks/data6       49171     0          Y       22932<br>Self-heal Daemon on localhost               N/A       N/A        Y       12366<br>Self-heal Daemon on server2               N/A       N/A        Y       21446<br> <br>Task Status of Volume tank<br>------------------------------------------------------------------------------<br>Task                 : Rebalance           <br>ID                   : ec958aee-edbd-4106-b896-97c688fde0e3<br>Status               : completed<br></div><div><br></div><div>As you can see the added 3,4,5,6 bricks appear differently:</div><div><br></div><div>Brick server1:/gluster_bricks/data3       49164     0          Y       4625 <br>Brick server1:/gluster_bricks/data4       49165     0          Y       4644 <br>Brick server1:/gluster_bricks/data5       49166     0          Y       5088 <br>Brick server1:/gluster_bricks/data6       49167     0          Y       5128 <br>Brick server2:/gluster_bricks/data3       49168     0          Y       22314<br>Brick server2:/gluster_bricks/data4       49169     0          Y       22345<br>Brick server2:/gluster_bricks/data5       49170     0          Y       22889<br>Brick server2:/gluster_bricks/data6       49171     0          Y       22932 </div><div><br></div><div>My question is what does this mean for the volume?  Everything appears to be running as expected, but:</div><div><br></div><div>- Is there a serious problem with the way the volume is now configured?</div><div>- Have we messed up the high availability of the 2 nodes?</div><div>- Is there a way to reconfigure the volume to get it to a more optimal state?</div><div><br></div><div>Any help is greatly appreciated...</div><div><br></div><div>Thanks in advance,</div><div><br></div><div>HB</div></div>