<div dir="ltr">Please check the rebalance log to see why those files were not migrated.<div><br></div><div>Regards,</div><div>Nithya</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 24 September 2018 at 21:21, Jeevan Patnaik <span dir="ltr"><<a href="mailto:g1patnaik@gmail.com" target="_blank">g1patnaik@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Hi,<div dir="auto"><br></div><div dir="auto">Yes, I did. The status shows all as completed.<div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Jeevan.</div></div></div><div class="HOEnZb"><div class="h5"><br><div class="gmail_quote"><div dir="ltr">On Mon 24 Sep, 2018, 9:20 PM Nithya Balachandran, <<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 24 September 2018 at 21:01, Jeevan Patnaik <span dir="ltr"><<a href="mailto:g1patnaik@gmail.com" rel="noreferrer" target="_blank">g1patnaik@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Hi,<div dir="auto"><br></div><div dir="auto"><span style="font-family:sans-serif">What do you mean by expanding volume?</span><br></div><div dir="auto"><span style="font-family:sans-serif"><br></span></div><div dir="auto"><span style="font-family:sans-serif">I mean adding new bricks to extend storage. It is mentioned that detaching tier is must for adding or removing bricks.</span></div><span><div dir="auto"><span style="font-family:sans-serif"><br></span></div><div dir="auto"><div style="font-family:sans-serif" dir="auto">What command did you use to detach the tier? And how did you check that the files still exist on the hot tier?</div><br></div></span><div dir="auto">gluster volume tier volume_name detach start</div><div dir="auto"><br></div><div dir="auto">I check the files directly from backend filesystem on disk and there are still left over files on all hot tier disks.</div><div dir="auto"><br></div><div dir="auto">Anyhow I proceeded with detach commit to see what would happen..so after commit, I can't access those files from client filesystems..so they're gone with the hot tier and were not moved to cold tier.</div></div></blockquote><div><br></div><div>Did you check that the detach tier operation had completed before running detach commit? </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div dir="auto"><br></div><div dir="auto">And even if I try to attach the hot tier bricks again to bring back the files, attach won't work with existing files.</div><div dir="auto"><br></div><div dir="auto">So, it seems we shouldn't detach hot tier without making sure all files are moved to cold tier..but in my case, some files just won't move, even though they aren't being accessed anywhere.</div></div></blockquote><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div dir="auto"><br></div><div dir="auto">There's a statement saying POSIX locks can cause files to be still left in hot tier and in that case, either the application causing posix lock has to be closed or files should be moved manually (but where?)... however, I don't think in my case there's any posix locks as the files are not accessed by any application except gluster itself.</div><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Jeevan.</div><div><div class="m_7462052381991830084m_8330735216703424778h5"><br><br><div class="gmail_quote" dir="auto"><div dir="ltr">On Mon 24 Sep, 2018, 8:45 PM Nithya Balachandran, <<a href="mailto:nbalacha@redhat.com" rel="noreferrer" target="_blank">nbalacha@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It has been a while since I worked on tier but here is what I remember:<div><br><div class="gmail_extra"><br><div class="gmail_quote">On 24 September 2018 at 19:15, Jeevan Patnaik <span dir="ltr"><<a href="mailto:g1patnaik@gmail.com" rel="noreferrer noreferrer" target="_blank">g1patnaik@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div>Hi Nithya,<div dir="auto"><br></div><div dir="auto">No..the files are not being accessed. And tiering mode is in cache mode and what I understood is in cache mode every file is moved to hot tier until it reaches low watermark, then it only promotes highly accessed files.</div><div dir="auto"><br></div></div></div></blockquote><div>Not quite . All files will initially exist on the cold tier. Any new files will be created on the hot tier. Any old files that are accessed will be promoted ot the hot tier.</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div><div dir="auto"></div><div dir="auto">And do you mean everytime we expand volume, we are virtually flushing the tier and it regenerates only when a files are accessed?</div><div dir="auto"><br></div></div></div></blockquote><div>What do you mean by expanding the volume?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div><div dir="auto"></div><div dir="auto">Also I see one more issue now, I hav generated another 1000 files after tiering is started and I see all 1000 files are now created on hot tier. Now, I detached again..in detach status it shows completed..but when I checked, around 260 files are still located on hot tier..those files are not being used anywhere..ca you tell why are they still not moved to cold tier and why the detach status is showing a wrong status.</div></div></div></blockquote><div><br></div><div>What command did you use to detach the tier? And how did you check that the files still exist on the hot tier?</div><div><br></div><div>Regards,</div><div>Nithya </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div><div dir="auto"><br></div><div dir="auto">I am using gluster 3.12.3 and server hosts include RHEL 6.7 and 7.2 hosts also.</div><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Jeevan.</div><div><div class="m_7462052381991830084m_8330735216703424778m_-744891376683826568m_1603412331074834329h5"><br><div class="gmail_quote"><div dir="ltr">On Mon 24 Sep, 2018, 7:00 PM Nithya Balachandran, <<a href="mailto:nbalacha@redhat.com" rel="noreferrer noreferrer" target="_blank">nbalacha@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Are those files being accessed? Tiering will only promote those that have been accessed recently.<div><br></div><div>Regards,</div><div>Nithya</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 24 September 2018 at 18:32, Jeevan Patnaik <span dir="ltr"><<a href="mailto:g1patnaik@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">g1patnaik@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">I see it still not promoting any files. Do we need to run any command to force the movement?<div dir="auto"><br></div><div dir="auto">This would be an issue if we need to expand disk as then we also need to detach existing tier and attach again and expect the data to be promoted to the hot tier.<br><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Jeevan.</div></div></div><div class="m_7462052381991830084m_8330735216703424778m_-744891376683826568m_1603412331074834329m_-8405063923353280846m_4003440502909294073HOEnZb"><div class="m_7462052381991830084m_8330735216703424778m_-744891376683826568m_1603412331074834329m_-8405063923353280846m_4003440502909294073h5"><br><div class="gmail_quote"><div dir="ltr">On Mon 24 Sep, 2018, 6:17 PM Jeevan Patnaik, <<a href="mailto:g1patnaik@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">g1patnaik@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Hi,<div dir="auto"><br></div><div dir="auto">I have created a 18 disk replica 3 volume and created 1000 random files each of 10M. </div><div dir="auto"><br></div><div dir="auto">Later I have attached 15 disk replica 3 tier with each hot tier disk coming from cold tier host, except on 3 (as we don't have hot tier disk)</div><div dir="auto"><br></div><div dir="auto">After attach, I see no files are promoted at even after 15 minutes. Status is showing as In progress. </div><div dir="auto"><br></div><div dir="auto">Is it always this much slow? For 10G data, 15 minutes is too much</div><div dir="auto">Also will the data try to be promoted to a hot tier coming from the same host to reduce the time taken to move the files?</div><div dir="auto"><br></div><div dir="auto">Cold tier is made of RAID supported SSD and hot tier is made of NVMe SSDs.</div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Jeevan.</div></div>
</blockquote></div>
</div></div><br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" rel="noreferrer noreferrer noreferrer" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>
</blockquote></div></div></div></div></div>
</blockquote></div><br></div></div></div>
</blockquote></div></div></div></div>
</blockquote></div><br></div></div>
</blockquote></div>
</div></div></blockquote></div><br></div>