[Gluster-devel] Re; Load balancing ...

[Gareth Bult]

>You're expecting a bit much here - for any shared/clustered FS. DRBD 
>might come close if your extents are big enough, but that's a whole 
>different ball game...

I was quoting a real-world / live data scenario, DRBD handles it just fine.
.. but it is a different mechanism to gluster.

>Sounds like a reasonably sane solution to me.

It is. It also makes Gluster useless in this scenario.

>Why would the cluster effectively be down? Other nodes would still be 
>able to server that file.

Nope, it won't replicate the file while another node has it locked .. which means you effectively need to close all files in order to kick off the replication process, and the OPEN call will not complete until the file has replicated .. so effectively (a) you need to restart all your processes to make then close and re-open their files (or HUP them.. or whatever), then those processes will all freeze until the files they are trying to open have replicated.

>Or are you talking about the client-side AFR? 

Mmm, it's been a while, I'm not entirely sure I've tested the issue on client side and server side.
Are you telling me that server-side will work quite happily and it's only client-side that has all these issues?

>I have to say, a one-client/multiple-servers scenario sounds odd. 
>If you don't care about downtime (you have just one client node so that's 
>the only conclusion that can be reached), then what's the problem with a bit more downtime?

My live scenario was 4 (2x2) AFR servers with ~ 12 clients.

Obviously this setup is no longer available to me as it proved to be useless in practice.

I'm currently revisiting Gluster with another "new" requirement (as per my last email) .. currently I'm testing a 2 x server + 1 x client setup with regards to load balancing and use over a slow line. Obviously (!) both servers can also act as clients so I guess to be pedantic you'd call it 2 servers + 3 clients. My point was I have 1 machine with no server.

Gareth.

--
Managing Director, Encryptec Limited
Tel: 0845 5082719, Mob: 0785 3305393
Email: gareth at encryptec.net 
Statements made are at all times subject to Encryptec's Terms and Conditions of Business, which are available upon request.

----- Original Message -----
From: "Gordan Bobic" <gordan at bobich.net>
To: gluster-devel at nongnu.org
Sent: Friday, April 25, 2008 9:40:00 PM GMT +00:00 GMT Britain, Ireland, Portugal
Subject: Re: [Gluster-devel] Re; Load balancing ...

Gareth Bult wrote:
>> If you have two nodes and the 20 GB file
>> only got written to node A while node B was down and
>> node B comes up the whole 20 GB is resynced to node B;
>> is that more network usage than if the 20 GB file were
>> written immediately to node A & node B.
> 
> Ah. Let's say you have both nodes running with a 20Gb file synced.
> Then you have to restart one glusterfs on one of the nodes.
> While it's down, let's say the other node appends 1 byte to the file.
> When it comes back up and looks a the file, the other node will see it's out of date and re-copy the entire 20Gb.

You're expecting a bit much here - for any shared/clustered FS. DRBD 
might come close if your extents are big enough, but that's a whole 
different ball game...

>> Perhaps the issue is really that the cost comes at an
>> unexpected time, on node startup instead of when the
>> file was originally written?  Would a startup
>> throttling mechanism help here on resyncs?
> 
> Yes, unfortunately you can't open a file while it's syncing .. so when you reboot your gluster server, downtime is the length of time it takes to restart glusterfs (or the machine, either way..) PLUS the amount of time it takes to recopy every file that was written to while one node was down ...

Sounds like a reasonably sane solution to me.

> Take a Xen server for example serving disk images off a gluster partition.
> 10 Images at 10G each gives you a 100G copy to do.

If they are static images why would they have changed? What you are 
describing would really be much better accomplished with a SAN+GFS or 
Coda which is specifically designed to handle disconnected operation at 
the expense of other things.

> Wait, it get's better .. it will only re-sync the file on opening, so you actually have to close all the files, then try to re-open them , then wait while it re-syncs the data (during this time your cluster is effectively down), then the file open completes and you are back up again.

Why would the cluster effectively be down? Other nodes would still be 
able to server that file. Or are you talking about the client-side AFR? 
I have to say, a one-client/multiple-servers scenario sounds odd. If you 
don't care about downtime (you have just one client node so that's the 
only conclusion that can be reached), then what's the problem with a bit 
more downtime?

> Yet there is a claim in the FAQ that there is no single point of failure .. yet to upgrade gluster for example you effectively need to shut down the entire cluster in order to get all files to re-sync ...

Wire protocol incompatibilities are, indeed unfortunate. But on one hand 
you speak of manual failover and SPOF clients and on the other you speak 
of unwanted downtime. If this bothers you, have enough nodes that you 
could shut down half (leaving half running), upgrade the downed ones, 
bring them up and migrade the IPs (heartbeat, RHCS, etc) to the upgraded 
ones and upgrade the remaining nodes. The downtime should be seconds at 
most.

> Effectively storing anything like a large file on AFR is pretty unworkable and makes split-brian issues pale into insignificance ... or at least that's my experience of trying to use it...

I can't help but think that you're trying to use the wrong tool for the 
job here. A SAN/GFS solution sounds like it would fit your use case better.

Gordan


_______________________________________________
Gluster-devel mailing list
Gluster-devel at nongnu.org
http://lists.nongnu.org/mailman/listinfo/gluster-devel