[Gluster-users] writing small 16Mb files many times a min from postgresql to Glusterfs client file system

[Sophie.Loewenthal at smals.be]

Hi,

        We have a postgresdb that streams archive logs into a glusterfs 
mount point.  Their writing process shows continuous writes but not any 
flush or close system calls.   During this writing the virtual memory of 
the glusterfs process gradually increases that it eventually surpasses the 
amount of system memory.  And then other processes cannot start because 
shared memory cannot be allocated.

Each file 'chunk' written is 16Mb and writes are up to 100 a minuet.

The postgresql process is /opt/PostgresPlus/9.4AS/bin/pg_receivexlog

My work around is to stop the archive log writer pg_receivexlog and 
unmount the glusterfs file systems, which frees the virtual memory, mount 
the file system and start pg_receivexlog. 

We plan to replace our RedHat gluster setup with community Glusterfs, but 
I would like to know if gluster is suitable for this type of writing.

Or did we hit a bug?




Client
3.10.0-514.el7.x86_64 / RHEL 7.3
# rpm -qa|grep -i gluster
glusterfs-3.7.9-12.el7.x86_64
glusterfs-client-xlators-3.7.9-12.el7.x86_64
glusterfs-fuse-3.7.9-12.el7.x86_64
glusterfs-libs-3.7.9-12.el7.x86_64
Process: 
# ps -eaf |grep glusterfs
root     12593     1  0 Mar30 ?        00:13:27 /usr/sbin/glusterfs --acl 
--volfile-server=glustersrv1.int.local 
--volfile-server=glustersrv1.int.local --volfile-id=/db_backup_111 
/backup_111_2

Server
2.6.32-573.12.1.el6.x86_64  /  RHEL 6.8
# rpm -qa |grep -i gluster
glusterfs-3.6.0.29-1.el6rhs.x86_64
glusterfs-cli-3.6.0.29-1.el6rhs.x86_64
glusterfs-geo-replication-3.6.0.29-1.el6rhs.x86_64
glusterfs-libs-3.6.0.29-1.el6rhs.x86_64
glusterfs-fuse-3.6.0.29-1.el6rhs.x86_64
glusterfs-server-3.6.0.29-1.el6rhs.x86_64
gluster-nagios-addons-0.1.10-2.el6rhs.x86_64
vdsm-gluster-4.14.7.2-1.el6rhs.noarch
samba-glusterfs-3.6.23-35.el6_8.x86_64
glusterfs-api-3.6.0.29-1.el6rhs.x86_64
gluster-nagios-common-0.1.3-2.el6rhs.noarch
glusterfs-rdma-3.6.0.29-1.el6rhs.x86_64

Kind regards,
Sophie Loewenthal

Server Infrastructure
Smals.be
















Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in 
haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur 
of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit 
onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans 
ses statuts, seul l'administrateur délégué, le directeur général ou son 
mandataire exprès est habilité à souscrire des engagements au nom de 
Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le 
signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit 
association in its bylaws, only the chief executive officer, the general 
manager or his explicit agent can enter into engagements on behalf of 
Smals.
If you are not the addressee of this message, we kindly ask you to signal 
this to us immediately and to delete the message.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170410/6a8dcbec/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 17394 bytes
Desc: not available
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170410/6a8dcbec/attachment.jpe>