[Gluster-users] Geo-replication log file not closed

[David Cunningham]

Hello,

We're having an issue with the rotated gsyncd.log not being released.
Here's the output of 'lsof':

# lsof | grep 'gsyncd.log.1'
python2    4495                  root    3w      REG                8,1
 991675023    4332241
/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)
python2    4495  4496            root    3w      REG                8,1
 991675023    4332241
/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)
python2    4495  4507            root    3w      REG                8,1
 991675023    4332241
/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)
python2    4508                  root    3w      REG                8,1
 991675023    4332241
/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)
python2    4508                  root    5w      REG                8,1
 991675023    4332241
/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)
python2    4508  4511            root    3w      REG                8,1
 991675023    4332241
/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log.1 (deleted)
... etc...

Those processes are:
# ps -ef | egrep '4495|4508'
root      4495     1  0 Aug10 ?        00:00:59 /usr/bin/python2
/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py
--path=/nodirectwritedata/gluster/gvol0  --monitor -c
/var/lib/glusterd/geo-replication/gvol0_nvfs10_gvol0/gsyncd.conf
--iprefix=/var :gvol0 --glusterd-uuid=b7521445-ee93-4fed-8ced-6a609fa8c7d4
nvfs10::gvol0
root      4508  4495  0 Aug10 ?        00:01:56 python2
/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py agent gvol0
nvfs10::gvol0 --local-path /nodirectwritedata/gluster/gvol0 --local-node
cafs30 --local-node-id b7521445-ee93-4fed-8ced-6a609fa8c7d4 --slave-id
cdcdb210-839c-4306-a4dc-e696b165ed17 --rpc-fd 9,12,11,10

And here's the relevant part of the /etc/logrotate.d/glusterfs-georep
script:

/var/log/glusterfs/geo-replication/*/*.log {
    sharedscripts
    rotate 52
    missingok
    compress
    delaycompress
    notifempty
    postrotate
    for pid in `ps -aef | grep glusterfs | egrep "\-\-aux-gfid-mount" | awk
'{print $2}'`; do
        /usr/bin/kill -HUP $pid > /dev/null 2>&1 || true
    done
     endscript
}

If I run the postrotate part manually:
# ps -aef | grep glusterfs | egrep "\-\-aux-gfid-mount" | awk '{print $2}'
4520

# ps -aef | grep 4520
root      4520     1  0 Aug10 ?        01:24:23 /usr/sbin/glusterfs
--aux-gfid-mount --acl --log-level=INFO
--log-file=/var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/mnt-nodirectwritedata-gluster-gvol0.log
--volfile-server=localhost --volfile-id=gvol0 --client-pid=-1
/tmp/gsyncd-aux-mount-Tq_3sU

Perhaps the problem is that the kill -HUP in the logrotate script doesn't
act on the right process? If so, does anyone have a command to get the
right PID?

Thanks in advance for any help.

-- 
David Cunningham, Voisonics Limited
http://voisonics.com/
USA: +1 213 221 1092
New Zealand: +64 (0)28 2558 3782
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200825/fa5213f6/attachment.html>