<div dir="ltr"><div>Hello all,</div><div><br></div><div>Apparently we don't want to "kill -HUP" the two processes that have rotated log file still open:<br></div><div>root 4495 1 0 Aug10 ? 00:00:59 /usr/bin/python2 /usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py --path=/nodirectwritedata/gluster/gvol0 --monitor -c /var/lib/glusterd/geo-replication/gvol0_nvfs10_gvol0/gsyncd.conf --iprefix=/var :gvol0 --glusterd-uuid=b7521445-ee93-4fed-8ced-6a609fa8c7d4 nvfs10::gvol0<br>root 4508 4495 0 Aug10 ? 00:01:56 python2 /usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py agent gvol0 nvfs10::gvol0 --local-path /nodirectwritedata/gluster/gvol0 --local-node cafs30 --local-node-id b7521445-ee93-4fed-8ced-6a609fa8c7d4 --slave-id cdcdb210-839c-4306-a4dc-e696b165ed17 --rpc-fd 9,12,11,10</div><div>... a kill -HUP on those processes stops them rather than re-opening the log file.</div><div><br></div><div>Does anyone know if these processes are supposed to have gsyncd.log open? If so, how do we tell them to close and re-open their file handle?</div><div><br></div><div>Thanks in advance!</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 25 Aug 2020 at 15:24, David Cunningham <<a href="mailto:dcunningham@voisonics.com">dcunningham@voisonics.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello,</div><div><br></div><div>We're having an issue with the rotated gsyncd.log not being released. Here's the output of 'lsof':<br></div><div><br></div><div># lsof | grep 'gsyncd.log.1'<br>python2 4495 root 3w REG 8,1 991675023 4332241 /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)<br>python2 4495 4496 root 3w REG 8,1 991675023 4332241 /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)<br>python2 4495 4507 root 3w REG 8,1 991675023 4332241 /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)<br>python2 4508 root 3w REG 8,1 991675023 4332241 /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)<br>python2 4508 root 5w REG 8,1 991675023 4332241 /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)<br>python2 4508 4511 root 3w REG 8,1 991675023 4332241 /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)</div><div>... etc...</div><div><br></div><div>Those processes are:</div><div># ps -ef | egrep '4495|4508'<br>root 4495 1 0 Aug10 ? 00:00:59 /usr/bin/python2 /usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py --path=/nodirectwritedata/gluster/gvol0 --monitor -c /var/lib/glusterd/geo-replication/gvol0_nvfs10_gvol0/gsyncd.conf --iprefix=/var :gvol0 --glusterd-uuid=b7521445-ee93-4fed-8ced-6a609fa8c7d4 nvfs10::gvol0<br>root 4508 4495 0 Aug10 ? 00:01:56 python2 /usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py agent gvol0 nvfs10::gvol0 --local-path /nodirectwritedata/gluster/gvol0 --local-node cafs30 --local-node-id b7521445-ee93-4fed-8ced-6a609fa8c7d4 --slave-id cdcdb210-839c-4306-a4dc-e696b165ed17 --rpc-fd 9,12,11,10<br></div><div><br></div><div>And here's the relevant part of the /etc/logrotate.d/glusterfs-georep script:<br></div><div><br></div><div>/var/log/glusterfs/geo-replication/*/*.log {<br> sharedscripts<br> rotate 52<br> missingok<br> compress<br> delaycompress<br> notifempty<br> postrotate<br> for pid in `ps -aef | grep glusterfs | egrep "\-\-aux-gfid-mount" | awk '{print $2}'`; do<br> /usr/bin/kill -HUP $pid > /dev/null 2>&1 || true<br> done<br> endscript<br>}<br></div><div><br></div><div>If I run the postrotate part manually:</div><div># ps -aef | grep glusterfs | egrep "\-\-aux-gfid-mount" | awk '{print $2}'<br>4520</div><div><br></div><div># ps -aef | grep 4520<br>root 4520 1 0 Aug10 ? 01:24:23 /usr/sbin/glusterfs --aux-gfid-mount --acl --log-level=INFO --log-file=/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/mnt-nodirectwritedata-gluster-gvol0.log --volfile-server=localhost --volfile-id=gvol0 --client-pid=-1 /tmp/gsyncd-aux-mount-Tq_3sU<br></div><div><br></div><div></div><div>Perhaps the problem is that the kill -HUP in the logrotate script doesn't act on the right process? If so, does anyone have a command to get the right PID?</div><div><br></div><div>Thanks in advance for any help.<br></div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>David Cunningham, Voisonics Limited<br><a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>USA: +1 213 221 1092<br>New Zealand: +64 (0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>David Cunningham, Voisonics Limited<br><a href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br>USA: +1 213 221 1092<br>New Zealand: +64 (0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div>