<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi David,<div class=""><br class=""></div><div class="">What is it reporting for brick’s `df` output?</div><div class=""><br class=""></div><div class="">```</div><div class="">df /nodirectwritedata/gluster/gvol0</div><div class="">```</div><div class=""><br class=""></div><div class="">—</div><div class="">regards</div><div class=""><div>Aravinda Vishwanathapura</div><div><a href="https://kadalu.io" class="">https://kadalu.io</a></div><div><br class=""><blockquote type="cite" class=""><div class="">On 06-Mar-2020, at 2:52 AM, David Cunningham <<a href="mailto:dcunningham@voisonics.com" class="">dcunningham@voisonics.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hello,</div><div class=""><br class=""></div><div class="">A major concern we have is that "df" was reporting only 54% used and yet GlusterFS was giving "No space left on device" errors. We rely on "df" to report the correct result to monitor the system and ensure stability. Does anyone know what might have been going on here?</div><div class=""><br class=""></div><div class="">Thanks in advance.</div><div class=""><br class=""></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 5 Mar 2020 at 21:35, David Cunningham <<a href="mailto:dcunningham@voisonics.com" class="">dcunningham@voisonics.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class="">Hi Aravinda,</div><div class=""><br class=""></div><div class="">Thanks for the reply. This test server is indeed the master server for geo-replication to a slave.</div><div class=""><br class=""></div><div class="">I'm really surprised that geo-replication simply keeps writing logs until all space is consumed, without cleaning them up itself. I didn't see any warning about it in the geo-replication install documentation which is unfortunate. We'll come up with a solution to delete log files older than the LAST_SYNCED time in the geo-replication status. Is anyone aware of any other potential gotchas like this?<br class=""></div><div class=""><br class=""></div><div class="">Does anyone have an idea why in my previous note some space in the 2GB GlusterFS partition apparently went missing? We had 0.47GB of data, 1GB reported used by .glusterfs, which even if they were separate files would only add up to 1.47GB used, meaning 0.53GB should have been left in the partition. If less space is actually being used because of the hard links then it's even harder to understand where the other 1.53GB went. So why would GlusterFS report "No space left on device"?</div><div class=""><br class=""></div><div class="">Thanks again for any assistance.</div><div class=""><br class=""></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 5 Mar 2020 at 17:31, Aravinda VK <<a href="mailto:aravinda@kadalu.io" target="_blank" class="">aravinda@kadalu.io</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="">Hi David,<div class=""><br class=""></div><div class="">Is this Volume is uses Geo-replication? Geo-replication feature enables Changelog to identify the latest changes happening in the GlusterFS volume. </div><div class=""><br class=""></div><div class="">Content of .glusterfs directory also includes hardlinks to the actual data, so the size shown in .glusterfs is including data. Please refer the comment by Xavi <a href="https://github.com/gluster/glusterfs/issues/833#issuecomment-594436009" target="_blank" class="">https://github.com/gluster/glusterfs/issues/833#issuecomment-594436009</a></div><div class=""><br class=""></div><div class="">If Changelogs files are causing issue, you can use archival tool to remove processed changelogs.</div><div class=""><a href="https://github.com/aravindavk/archive_gluster_changelogs" target="_blank" class="">https://github.com/aravindavk/archive_gluster_changelogs</a></div><div class=""><br class=""></div><div class="">—</div><div class="">regards</div><div class=""><div class="">Aravinda Vishwanathapura</div><div class=""><a href="https://kadalu.io/" target="_blank" class="">https://kadalu.io</a></div><div class=""><br class=""></div><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 05-Mar-2020, at 9:02 AM, David Cunningham <<a href="mailto:dcunningham@voisonics.com" target="_blank" class="">dcunningham@voisonics.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div class="">Hello,</div><div class=""><br class=""></div><div class="">We are looking for some advice on disk use. This is on a single node GlusterFS test server.<br class=""></div><div class=""><br class=""></div><div class="">There's a 2GB partition for GlusterFS. Of that, 470MB is used for actual data, and 1GB is used by the .glusterfs directory. The .glusterfs directory is mostly used by the two-character directories and the "changelogs" directory. Why is so much used by .glusterfs, and can we reduce that overhead?</div><div class=""><br class=""></div><div class="">We also have a problem with this test system where GlusterFS is giving "No space left on device" errors. That's despite "df" reporting only 54% used, and even if we add the 470MB to 1GB used above, that still comes out to less than the 2GB available, so there should be some spare.<br class=""></div><div class=""><br class=""></div><div class="">Would anyone be able to advise on these please? Thank you in advance.<br class=""></div><div class=""><br class=""></div><div class="">The GlusterFS version is 5.11 and here is the volume information:</div><div class=""><br class=""></div><div class="">Volume Name: gvol0<br class="">Type: Distribute<br class="">Volume ID: 33ed309b-0e63-4f9a-8132-ab1b0fdcbc36<br class="">Status: Started<br class="">Snapshot Count: 0<br class="">Number of Bricks: 1<br class="">Transport-type: tcp<br class="">Bricks:<br class="">Brick1: myhost:/nodirectwritedata/gluster/gvol0<br class="">Options Reconfigured:<br class="">transport.address-family: inet<br class="">nfs.disable: on<br class="">geo-replication.indexing: on<br class="">geo-replication.ignore-pid-check: on<br class="">changelog.changelog: on<br class=""></div><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">David Cunningham, Voisonics Limited<br class=""><a href="http://voisonics.com/" target="_blank" class="">http://voisonics.com/</a><br class="">USA: +1 213 221 1092<br class="">New Zealand: +64 (0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div></div>
________<br class=""><br class=""><br class=""><br class="">Community Meeting Calendar:<br class=""><br class="">Schedule -<br class="">Every Tuesday at 14:30 IST / 09:00 UTC<br class="">Bridge: <a href="https://bluejeans.com/441850968" target="_blank" class="">https://bluejeans.com/441850968</a><br class=""><br class="">Gluster-users mailing list<br class=""><a href="mailto:Gluster-users@gluster.org" target="_blank" class="">Gluster-users@gluster.org</a><br class=""><a href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank" class="">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br class=""></div></blockquote></div><br class=""><div class="">
<div class=""><br class=""></div><div class=""><br class=""></div><br class="">
</div>
<br class=""></div></div></blockquote></div><br clear="all" class=""><br class="">-- <br class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">David Cunningham, Voisonics Limited<br class=""><a href="http://voisonics.com/" target="_blank" class="">http://voisonics.com/</a><br class="">USA: +1 213 221 1092<br class="">New Zealand: +64 (0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all" class=""><br class="">-- <br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">David Cunningham, Voisonics Limited<br class=""><a href="http://voisonics.com/" target="_blank" class="">http://voisonics.com/</a><br class="">USA: +1 213 221 1092<br class="">New Zealand: +64 (0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div>
</div></blockquote></div><br class=""><div class="">
<div><br class=""></div><div class=""><br class=""></div><br class="Apple-interchange-newline">
</div>
<br class=""></div></body></html>