<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>An even replica count is prone to split-brains with the default
quorum settings, so replica 3 is recommended. <br>
</p>
<p> But a replica 4 setup should not cause any checksum change if
one node is unreachable, as long as the reads/writes are done via
the gluster mount (and not directly the bricks). I wasn't able to
re-create this when I copied a huge file on to a 1x4 volume
(glusterfs 6.3) with one of the bricks down. Is this something
that you can reproduce? Do you see anything suspicious in the
mount or brick logs?</p>
Regards,<br>
Ravi<br>
<div class="moz-cite-prefix">On 28/07/19 10:47 AM, Strahil wrote:<br>
</div>
<blockquote type="cite"
cite="mid:vy23vmjkarue0pe26q2luyiq.1564291055002@email.android.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<p dir="ltr">I never thought that replica 4 is allowed optiion. I
always thought that 3 copies is the maximum.</p>
<p dir="ltr">Best Regards,<br>
Strahil Nikolov</p>
<div class="quote">On Jul 27, 2019 16:30, Matthew Evans
<a class="moz-txt-link-rfc2396E" href="mailto:runmatt@live.com"><runmatt@live.com></a> wrote:<br type="attribution">
<blockquote class="quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
Hi Ravishankar - I figured out the issue. The 4th node was
showing "online" under 'gluster peer status' as well as
'gluster volume status' - but 'gluster volume status'
wasn't showing a TCP port for that 4th node. When I opened
49152 in firewalld and then re-copied the ISO, the hash
didn't change.</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
So, now I guess the question would be, why would having
one malfunctioning node override 3 functioning nodes and
cause a file to be altered? I wasn't performing the
initial copy onto the malfunctioning node.</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
matt@docker1:~$ sudo glusterfs --version<br>
glusterfs 6.3</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
matt@docker1:~$ sudo gluster volume info<br>
<div><br>
</div>
<div>Volume Name: swarm-vols<br>
</div>
<div>Type: Replicate<br>
</div>
<div>Volume ID: 0b51e6b3-786e-454e-8a16-89b47e94828a<br>
</div>
<div>Status: Started<br>
</div>
<div>Snapshot Count: 0<br>
</div>
<div>Number of Bricks: 1 x 4 = 4<br>
</div>
<div>Transport-type: tcp<br>
</div>
<div>Bricks:<br>
</div>
<div>Brick1: docker1:/gluster/data<br>
</div>
<div>Brick2: docker2:/gluster/data<br>
</div>
<div>Brick3: docker3:/gluster/data<br>
</div>
<div>Brick4: docker4:/gluster/data<br>
</div>
<div>Options Reconfigured:<br>
</div>
<div>performance.client-io-threads: off<br>
</div>
<div>nfs.disable: on<br>
</div>
<div>transport.address-family: inet<br>
</div>
auth.allow: 10.5.22.*<br>
</div>
<div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font style="font-size:11pt" face="Calibri,
sans-serif" color="#000000"><b>From:</b> Ravishankar N
<a class="moz-txt-link-rfc2396E" href="mailto:ravishankar@redhat.com"><ravishankar@redhat.com></a><br>
<b>Sent:</b> Saturday, July 27, 2019 2:04 AM<br>
<b>To:</b> Matthew Evans <a class="moz-txt-link-rfc2396E" href="mailto:runmatt@live.com"><runmatt@live.com></a>;
<a class="moz-txt-link-abbreviated" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:gluster-users@gluster.org"><gluster-users@gluster.org></a><br>
<b>Subject:</b> Re: [Gluster-users] GlusterFS Changing
Hash of Large Files?</font>
<div> </div>
</div>
<div style="background-color:#ffffff">
<p style="margin-top:0px;margin-bottom:0px"><br>
</p>
<div>On 26/07/19 6:50 PM, Matthew Evans wrote:<br>
</div>
<blockquote>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
I've got a new glusterfs 4 node replica cluster
running under CentOS 7. All hosts are backed by SSD
drives and are connected to a 1Gbps Ethernet
network. <span style="font-family:'calibri' ,
'helvetica' , sans-serif;background-color:rgb( 255
, 255 , 255 );display:inline">3 nodes are running
on CentOS 7 under ESXi on the same physical host,
1 is running on CentOS 7 under Hyper-V. I use this
for my docker swarm persistent storage and all
seems to work well.</span></div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
Yesterday however, I copied a 4GB .ISO file to my
volume for a friend to download. I noticed the
SHA256 hash of the ISO was altered. I downloaded a
fresh copy to my desktop, verified the hash, scp'd
it to the local glusterfs host storage and again,
re-verified the hash. The moment I copied it to my
glusterfs volume, the file hash changed. When my
friend downloaded the ISO, his hash matched changed
hash.</div>
</blockquote>
Can you provide the below details?<br>
- glusterfs version<br>
<p style="margin-top:0px;margin-bottom:0px">-`gluster
volume info`</p>
<br>
<blockquote>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
<br>
</div>
<div style="font-family:'calibri' , 'helvetica' ,
sans-serif;font-size:12pt;color:rgb( 0 , 0 , 0 )">
I am new to glusterfs, having deployed this as my
first cluster ever about a week ago. Can someone
help me work through why this file</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>