[Gluster-users] Shard file size (gluster 3.7.5)

[Krutika Dhananjay]

You are right! 

So this is what I ran on the volume with bricks in ext4 partition: dd if=/dev/urandom of=file bs=1024 seek=3072 count=2048 conv=notrunc 

with shard-block-size being 4M. As you can see, the command creates a sparse 5M file with holes in first 3M. 
This means the first block file will mostly be sparse, with the second block file (the one holding the last 1M of data) looking like a normal file. 

[root at calvin mnt]# dd if=/dev/urandom of=file bs=1024 seek=3072 count=2048 conv=notrunc 
2048+0 records in 
2048+0 records out 
2097152 bytes (2.1 MB) copied, 0.875319 s, 2.4 MB/s 
[root at calvin mnt]# du file 
2052 file 

... while on the volume with xfs bricks, the number reads 3012. 
I added trace logs to see what his happening in the latter case. The posix translator in gluster seems to return more blocks than actually written to if the file is sparse (in this case for the first block file). 
And for the second file, it is returning blocks as 1 block per 512bytes of data written. And sharding relies on the values returned by posix translator to keep an account of the size and block count. 

I will need some more time to understand why this is so. I will let you know soon as I've figured it out. 

Thanks for the report. 

-Krutika 

----- Original Message -----

> From: "Lindsay Mathieson" <lindsay.mathieson at gmail.com>
> To: "Krutika Dhananjay" <kdhananj at redhat.com>
> Cc: "gluster-users" <gluster-users at gluster.org>
> Sent: Friday, November 6, 2015 7:58:42 PM
> Subject: Re: [Gluster-users] Shard file size (gluster 3.7.5)

> On 6 November 2015 at 17:22, Krutika Dhananjay < kdhananj at redhat.com > wrote:

> > Sure. So far I've just been able to figure that GlusterFS counts blocks in
> > multiples of 512B while XFS seems to count them in multiples of 4.0KB.
> 
> > Let me again try creating sparse files on xfs, sharded and non-sharded
> > gluster volumes and compare the results. I'll let you know what I find.
> 

> I repeated the tests with a single gluster brick on a ext4 partition - disk
> usage (du) and file size were exactly right.

> --
> Lindsay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151109/f413d550/attachment.html>