<div dir="ltr"><div><br></div><div>Hi Cedric,</div><div><br></div><div>The 120 seconds is given to allow a window for things to settle. i.e. imagine the following situation</div><div><br></div><div>1) open file (fd1 as file descriptor)</div><div>2) modify the file via fd1</div><div>3) close the file descriptor (fd1)</div><div>4) Again open the file (fd2)</div><div>5) modify</div><div><br></div><div>In the above set of operations, by the time bitrot daemon tries to calculate the signature after 1st fd (fd1) is closed, active IO could be happening again on the new file descriptor (fd2). And The signature calculated might not be correct while active IO is happening. </div><div>So in gluster bitrot daemon waits for 120 seconds to sign the file after all the file descriptors associated with that file are closed.</div><div><br></div><div>So with 120 seconds time what happens is, once all the file descriptors associated with a file are closed (by the application), then a notification is sent to bitrot daemon that a object (file to be precise with details about that file) is modified. When all the file descriptors of a file are closed a operation called "release" is received by the brick. So the brick process sends a notification to bitrot daemon about a object (i.e. file) when release operation is received on that file (means all the file descriptors are closed). And the bitrot daemon waits for 120 seconds after receiving the notice. And before the file is signed (i.e. within the 120 seconds of wait time), if someone again opens it and modifies it, the brick process will let the bit rot daemon know about it so that bitrot daemon wont attempt to sign the file (as it is actively being modified).</div><div><br></div><div>The above value is configurable. And can be changed to some other value. You can use the below command to change it to a different value</div><div><br></div><div>"gluster volume set <volume name> features.expiry-time <value>"</div><div><br></div><div>But as you said, currently the comparison of the signature by the scrubber is local. i.e. while scrubbing, it calculates the checksum of the file, compares with the stored checksum (as a extended attribute) to determine whether the object is corrupted or not. </div><div>So yes, if the object is corrupted before the signing happens, then as of now the scrubber does not have the mechanism to know that.</div><div><br></div><div>Regards,</div><div>Raghavendra</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 18, 2018 at 2:20 PM, Cedric Lemarchand <span dir="ltr"><<a href="mailto:yipikai7@gmail.com" target="_blank">yipikai7@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Hi Sweta,<div><br></div><div>Thanks, this drive me some more questions:</div><div><br></div><div>1. What is the reason of delaying signature creation ?</div><div><br></div><div>2. As a same file (replicated or dispersed) having different signature thought bricks is by definition an error, it would be good to triggered it during a scrub, or with a different tool. Is something like this planned ?</div><div><br></div><div>Cheers</div><div><br></div><div><div>
<div style="word-wrap:break-word"><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">—</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Cédric Lemarchand</div></div>
</div><div><div class="h5">
<br><div><blockquote type="cite"><div>On 18 Apr 2018, at 07:53, Sweta Anandpara <<a href="mailto:sanandpa@redhat.com" target="_blank">sanandpa@redhat.com</a>> wrote:</div><br class="m_-972653206206289369Apple-interchange-newline"><div><div>Hi Cedric,<br><br>Any file is picked up for signing by the bitd process after the predetermined wait of 120 seconds. This default value is captured in the volume option 'features.expiry-time' and is configurable - in your case, it can be set to 0 or 1.<br><br>Point 2 is correct. A file corrupted before the bitrot signature is generated will not be successfully detected by the scrubber. That would require admin/manual intervention to explicitly heal the corrupted file.<br><br>-Sweta<br><br>On 04/16/2018 10:42 PM, Cedric Lemarchand wrote:<br><blockquote type="cite">Hello,<br><br>I am playing around with the bitrot feature and have some questions:<br><br>1. when a file is created, the "trusted.bit-rot.signature” attribute<br>seems only created approximatively 120 seconds after its creations<br>(the cluster is idle and there is only one file living on it). Why ?<br>Is there a way to make this attribute generated at the same time of<br>the file creation ?<br><br>2. corrupting a file (adding a 0 locally on a brick) before the<br>creation of the "trusted.bit-rot.signature” do not provide any<br>warning: its signature is different than the 2 others copies on other<br>bricks. Starting a scrub did not show up anything. I would think that<br>Gluster compares signature between bricks for this particular use<br>cases, but it seems the check is only local, so a file corrupted<br>before it’s bitrot signature creation stay corrupted, and thus could<br>be served to clients whith bad data ?<br><br>Gluster 3.12.8 on Debian Stretch, bricks on ext4.<br><br>Volume Name: vol1<br>Type: Replicate<br>Volume ID: 85ccfaf2-5793-46f2-bd20-<wbr>3f823b0a2232<br>Status: Started<br>Snapshot Count: 0<br>Number of Bricks: 1 x 3 = 3<br>Transport-type: tcp<br>Bricks:<br>Brick1: gluster-01:/data/brick1<br>Brick2: gluster-02:/data/brick2<br>Brick3: gluster-03:/data/brick3<br>Options Reconfigured:<br>storage.build-pgfid: on<br>performance.client-io-threads: off<br>nfs.disable: on<br>transport.address-family: inet<br>features.bitrot: on<br>features.scrub: Active<br>features.scrub-throttle: aggressive<br>features.scrub-freq: hourly<br><br>Cheers,<br><br>Cédric<br>______________________________<wbr>_________________<br>Gluster-users mailing list<br><a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br><a href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br></blockquote><br></div></div></blockquote></div><br></div></div></div></div><br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>