[Gluster-devel] Fragment size in Systematic erasure code

[Manoj Pillai]

Hi Xavi,

----- Original Message -----
> From: "Xavier Hernandez" <xhernandez at datalab.es>
> To: "Ashish Pandey" <aspandey at redhat.com>
> Cc: "Gluster Devel" <gluster-devel at gluster.org>, "Manoj Pillai" <mpillai at redhat.com>
> Sent: Monday, March 14, 2016 5:25:53 PM
> Subject: Re: Fragment size in Systematic erasure code
> 
> Hi Ashish,
> 
> On 14/03/16 12:31, Ashish Pandey wrote:
> > Hi Xavi,
> >
> > I think for Systematic erasure coded volume you are going to take fragment
> > size of 512 Bytes.
> > Will there be any CLI option to configure this block size?
> > We were having a discussion and Manoj was suggesting to have this option
> > which might improve performance for some workload.
> > For example- If we can configure it to 8K, all the read can be served only
> > from one brick in case a file size is less than 8K.
> 
> I already considered to use a configurable fragment size, and I plan to
> have it. 

Good to hear!

> However the benefits of larger block sizes are not so clear.
> Having a fragment size of 8KB in a 4+2 configuration will use a stripe
> of 32KB. Any write smaller, or not aligned, or not multiple of this
> value will need a read-modify-write cycle, causing a performance
> degradation for some workloads. It's also slower to encode/decode a
> block of 32KB because it might not fully fit into processor caches,
> making the computation slower.
> 
> On the other side, a small read on multiple bricks should, in theory, be
> executed in parallel, not causing a noticeable performance drop.

Yes, we're primarily looking to see if certain read-intensive workloads 
can benefit from a larger fragment size.

> 
> Anyway many of these things depend on the workload, so having a
> configurable fragment size will give enough control to choose the best
> solution for each environment.

Right. Once the option is available, we can do some testing with different 
workloads, varying the fragment size, and post our findings.

Thanks,
Manoj

> Xavi
>