[Gluster-devel] Testing SEEK_HOLE performance

[Niels de Vos]

On Sat, May 09, 2015 at 05:50:56AM -0300, Christopher Pereira wrote:
> Hi,
> 
> I did a local sparse image copy benchmark copying from a gluster mount (of a
> local gluster-server) vs copying directly from the gluster-brick to test
> SEEK_HOLE support and performance.
> Motivation : VM's image files are sparse.
> Benchmark:
> 
>    cp /mnt/gluster-mount/image to /tmp/
>    => takes 1m31.586s
> 
>    cp /mnt/ext4/gluster-brick/image to /tmp/
>    => takes 0m29.215s (3 times faster!)
> 
> Target: sparse qcow2 image file
> 
> logic size: 30G
> physic size: 1.9G
> 
> Commands like 'cp' call lseek(NEXT_HOLE) and lssek(SEEK_DATA) to skip
> useless zeros on modern filesystems [1].
> Is this supported on the gluster client? Is the server sending some kind of
> holes-index-buffer to the client?

I wanted to have a check about this and tried it on a Fedora 21 system
with its default glusterfs (3.5.x):

    # mount -t glusterfs localhost:/seek_hole /mnt
    # cd /mnt
    # qemu-img create -f qcow2 -o size=5G image.qcow2

Flush the caches from Gluster:

    # cd /
    # umount /mnt

Run the test:

    # tcpdump -i any -s 2048 -w /var/tmp/cp-qcow2.pcap tcp and not port 22 &
    # mount -t glusterfs localhost:/seek_hole /mnt
    # cp -f /mnt/image.qcow2 /var/tmp
    # killall tcpdump

This shows two read calls over the network:
1. offset: 0;      size: 131072
2. offset: 131072; size: 131072

There are no lseek() requests for SEEK_HOLE/SEEK_DATA sent over the
network. Looking into the operations (FOPs) that are available to the
Gluster xlators, it seems that there is no lseek() FOP. For normal
lseek() requests, the position of the file-descriptor is kept
client-side, so there is no need for the server to support it. However,
for SEEK_HOLE/SEEK_DATA, the server needs to do some actions.

For all I can see there is currently no support for SEEK_HOLE/SEEK_DATA
in Gluster, but it seems not too difficult to add. Please file a feature
request in Bugzilla:

    https://bugzilla.redhat.com/enter_bug.cgi?product=GlusterFS&component=core&keywords=FutureFeature

Include the details and a link to this email too. Gmane is one place
where the archives are kept:

    http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10884

Once done, we should also add a page for this in our wiki:

    http://www.gluster.org/community/documentation/index.php/Features

Do you have an interest do implement this, or could it be a project that
we can give to new contributors that would like to develop something for
Gluster?

Thanks,
Niels


> 
> [1] lseek & SEEK_HOLE: http://linux.die.net/man/2/lseek
> 

> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150510/52efa232/attachment.sig>