[Gluster-devel] trusted.glusterfs.version xattr

[Gordan Bobic]

I suspect this isn't a problem that can be solved without having a 
proper journal of metadata per directory, so that upon connection, the 
whole journal can be replayed.

You could sort of bodge it and use timestamps as the primary version and 
the xattr version as secondary, bit that is no less dangerous - it only 
takes one machine to be out of sync, and we are again looking at massive 
scope for data loss.

You could bodge the bodge further to work around this by ensuring that 
the nodes are heartbeating current times to sync between them and 
without the sync no data exchange takes place. But that then complicates 
things because what do you do when a node connects and is out of sync, 
but in the future? Who wins on time sync? Who has the latest 
authoritative copy?

I think the most sane way of addressing this is to have a fully logged 
directory metadata journal. But then we are back to the journalling for 
fast updates issue with a journal shadow volume, which is non-trivial to 
implement.

Unless there is some kind of a major mitigating circumstance, it seems 
that between this and the race condition that Martin is talking about on 
the other thread, GlusterFS in it's current is just too dangerous to use 
in most environments that I can think of. And unlike Gareth a few days 
ago, I'm not talking about performance issues - I'm talking about scope 
for data loss in very valid and very common use cases. :'(

Gordan

Amar S. Tumballi wrote:
> I still need time to understand it. But my doubt is that, vi moves the 
> old file to 'filename~' and then create a new file 'filename'. If this 
> is the case it will have version 2, as its a new file.
> But still, I can think of all disasters it can cause. Let me check with 
> Avati/Krishna about these.
> 
> Regards,
> Amar
> 
> On Tue, May 6, 2008 at 12:07 PM, <gordan at bobich.net 
> <mailto:gordan at bobich.net>> wrote:
> 
>     Hi,
> 
>     Maybe I'm missing something important here, but something isn't
>     making sense to me.
> 
>     When I create a file on the glusterfs mount, it gets version of "2".
>     All well and good. If I >> a line to it, it's version ticks up. So
>     far so good.
> 
>     If I vi the file, add a line and save it, it's version resets back
>     to "2"!
> 
>     What exactly is going on here? I'm sure that there is a perfectly
>     good explanation here (i.e. that vi removes the file and re-creates
>     it anew), but I just wanted to make sure this is a sane condition.
>     Also, would this not mean that following a disconnection, all vi-ed
>     files would get clobbered when the server reconnects and the file
>     gets read?! This strikes me as _INCREDIBLY_ dangerous. Too dangerous
>     for any use, in fact.
> 
>     Can somebody please tell me that I'm wrong here? Because I really
>     hope that I am.