[Gluster-devel] Glusterfs and Structured data

[Raghavendra G]

On Tue, Feb 6, 2018 at 8:15 PM, Vijay Bellur <vbellur at redhat.com> wrote:

>
>
> On Sun, Feb 4, 2018 at 3:39 AM, Raghavendra Gowdappa <rgowdapp at redhat.com>
> wrote:
>
>> All,
>>
>> One of our users pointed out to the documentation that glusterfs is not
>> good for storing "Structured data" [1], while discussing an issue [2].
>
>
>
> As far as I remember, the content around structured data in the Install
> Guide is from a FAQ that was being circulated in Gluster, Inc. indicating
> the startup's market positioning. Most of that was based on not wanting to
> get into performance based comparisons of storage systems that are
> frequently seen in the structured data space.
>
>
>> Does any of you have more context on the feasibility of storing
>> "structured data" on Glusterfs? Is one of the reasons for such a suggestion
>> "staleness of metadata" as encountered in bugs like [3]?
>>
>
>
> There are challenges that distributed storage systems face when exposed to
> applications that were written for a local filesystem interface. We have
> encountered problems with applications like tar [4] that are not in the
> realm of "Structured data". If we look at the common theme across all these
> problems, it is related to metadata & read after write consistency issues
> with the default translator stack that gets exposed on the client side.
> While the default stack is optimal for other scenarios, it does seem that a
> category of applications needing strict metadata consistency is not well
> served by that. We have observed that disabling a few performance
> translators and tuning cache timeouts for VFS/FUSE have helped to overcome
> some of them. The WIP effort on timestamp consistency across the translator
> stack, patches that have been merged as a result of the bugs that you
> mention & other fixes for outstanding issues should certainly help in
> catering to these workloads better with the file interface.
>
> There are deployments that I have come across where glusterfs is used for
> storing structured data. gluster-block  & qemu-libgfapi overcome the
> metadata consistency problem by exposing a file as a block device & by
> disabling most of the performance translators in the default stack.
> Workloads that have been deemed problematic with the file interface for the
> reasons alluded above, function well with the block interface.
>

I agree that gluster-block due to its usage of a subset of glusterfs fops
(mostly reads/writes I guess), runs into less number of consistency issues.
However, as you've mentioned we seem to disable perf xlator stack in our
tests/use-cases till now. Note that perf xlator stack is one of worst
offenders as far as the metadata consistency is concerned (relatively less
scenarios of data inconsistency). So, I wonder,
* what would be the scenario if we enable perf xlator stack for
gluster-block?
* Is performance on gluster-block satisfactory so that we don't need these
xlators?
  - Or is it that these xlators are not useful for the workload usually run
on gluster-block (For random read/write workload, read/write caching
xlators offer less or no advantage)?
  - Or theoretically the workload is ought to benefit from perf xlators,
but we don't see them in our results (there are open bugs to this effect)?

I am asking these questions to ascertain priority on fixing perf xlators
for (meta)data inconsistencies. If we offer a different solution for these
workloads, the need for fixing these issues will be less.

I feel that we have come a long way from the time the install guide was
> written and an update for removing the "staleness of content" might be in
> order there :-).
>
> Regards,
> Vijay
>
> [4] https://bugzilla.redhat.com/show_bug.cgi?id=1058526
>
>
>>
>> [1] http://docs.gluster.org/en/latest/Install-Guide/Overview/
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1512691
>> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1390050
>>
>> regards,
>> Raghavendra
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Raghavendra G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20180208/17dedfdf/attachment.html>