[Gluster-devel] Fragment size in Systematic erasure code

[Xavier Hernandez]

Hi Ashish,

On 14/03/16 12:31, Ashish Pandey wrote:
> Hi Xavi,
>
> I think for Systematic erasure coded volume you are going to take fragment size of 512 Bytes.
> Will there be any CLI option to configure this block size?
> We were having a discussion and Manoj was suggesting to have this option which might improve performance for some workload.
> For example- If we can configure it to 8K, all the read can be served only from one brick in case a file size is less than 8K.

I already considered to use a configurable fragment size, and I plan to 
have it. However the benefits of larger block sizes are not so clear. 
Having a fragment size of 8KB in a 4+2 configuration will use a stripe 
of 32KB. Any write smaller, or not aligned, or not multiple of this 
value will need a read-modify-write cycle, causing a performance 
degradation for some workloads. It's also slower to encode/decode a 
block of 32KB because it might not fully fit into processor caches, 
making the computation slower.

On the other side, a small read on multiple bricks should, in theory, be 
executed in parallel, not causing a noticeable performance drop.

Anyway many of these things depend on the workload, so having a 
configurable fragment size will give enough control to choose the best 
solution for each environment.

Xavi