[Gluster-devel] Glusterfs and Debian 8 installation

[Niels de Vos]

On Sat, Jul 18, 2015 at 02:42:37PM +0300, Roman wrote:
> But I've got some questions about options
> 
> are these options are normal for only 2 nodes in replica?
> 
> cluster.server-quorum-type.server
> cluster.quorum-type
> according to description seems like useless if not running at least 3
> nodes? and won't it even harm my installation in the future?

Quorum helps a lot to prevent split brains. If one of your two servers
is offline, quorum can help to prevent the need for manual split-brain
resolving. And yes, 3 nodes or more would be advised, otherwise you can
get a full (quorum intended) outage if one server does down.

> network.remote-dio I did not understand from description, what actually
> this one does and how it could be useful for qemu?

    Description: If enabled, in open() and creat() calls, O_DIRECT flag
    will be filtered at the client protocol level so server will still
    continue to cache the file. This works similar to NFS's behavior of
    O_DIRECT

O_DIRECT instructs the functions in the I/O path to cache the I/O as
little as possible. When 'network.remote-dio' is enabled, the O_DIRECT
flag is used on the client side, until Gluster comes into play. Gluster
will filter the O_DIRECT flag and does not pass it on to the bricks.
This allows the bricks to cache I/O anyway, even if the application
tries not to do so.

> And general question about remaining performance options.. they all set
> some kind of caching off.. Does it affects performance in some way?

The performance options are intended to affect performance. How they
affect performance is *very* workload and environment dependent. In
general we try to use most senseful defaults. There are some
standardized options for certain workloads, like the "virt" group
options for VM storage. Those are a recommendation, but it may be
possible to tune your environment better for your workload.

Tuning is extremely time-consuming and a speciallized area of expertise.
You likely will need to change one option at the time, and combine each
single option with others, capture all of the test results in a huge
matrix. The tests you do, should be matching your intended workload. For
VMs, this would include the workload inside the VMs too.

HTH,
Niels


> 2015-07-18 7:03 GMT+03:00 Roman <romeo.r at gmail.com>:
> 
> > Hi,
> >
> > Niels, its turned out that I had one of the options you told me to set
> > already on (performance.read-ahead: off). After I added the second one
> > (performance.write-behind: off) it seem like this problem does not exists
> > any more. At least I installed twice D8 with with Mate. I will do more
> > tests to be certain about that. As for now, can we speak, that recommended
> > options for qemu VM storage are:
> >
> > server.allow-insecure: on
> > cluster.server-quorum-type: server
> > cluster.quorum-type: auto
> > network.remote-dio: enable
> > cluster.eager-lock: enable
> > performance.stat-prefetch: off
> > performance.io-cache: off
> > performance.read-ahead: off
> > performance.quick-read: off
> > performance.write-behind: off
> >
> > How bad this affects volume performance (if it does, of course) ?
> >
> >
> > 2015-07-14 13:38 GMT+03:00 Roman <romeo.r at gmail.com>:
> >
> >> here is one of the errors example. its like files that debian installer
> >> copies to the virtual disk that is located on glusterfs storage are getting
> >> corrupted.
> >> in-target is /dev/vda1
> >>
> >>
> >> 2015-07-14 11:49 GMT+03:00 Roman <romeo.r at gmail.com>:
> >>
> >>> Ubuntu 14.04 LTS base install and then mate install was fine!
> >>>
> >>> Forgot to add, that debian8 installs OK on local disks.
> >>>
> >>> 2015-07-14 9:30 GMT+03:00 Roman <romeo.r at gmail.com>:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've reported a lot on gluster-users list, but seems, like noone is
> >>>> excited because of this bug. I think devs should try this out in their lab.
> >>>>
> >>>> In short:
> >>>>
> >>>> Running latest proxmox VE with these versions:
> >>>>
> >>>> Linux services 2.6.32-37-pve
> >>>> QEMU emulator version 2.2.1
> >>>>
> >>>> Running 2 gluster servers with Replica and Distributed volumes, both
> >>>> server and client side of gluster are updated to glusterfs 3.6.4
> >>>> yesterday. Before that was 3.5.4 with same behavior.
> >>>>
> >>>> Problem:
> >>>> I'm not able to install Debian 8 on gluster volumes. It just randomly
> >>>> aborts installation on random step:
> >>>> If we talk about base system, it aborts on mirror selecting step (just
> >>>> keeps saying, there is networking problems and unable to connect to any
> >>>> mirror (while i do can ping them all from console). Or it aborts (if got
> >>>> downloaded packages) on installation steps complaining about not solved
> >>>> depndecies due to unsuccessfull install one of them (the most often is
> >>>> python-gtk2 pkg, when I install with Mate)
> >>>>
> >>>> Once I was lucky enough to insall the base system, but after reboot it
> >>>> was not usable, i wasn't able to run even apache, it always said there are
> >>>> no modules, while they were on their place. So i think they were corrupted.
> >>>>
> >>>> And such behavior is only with debian8, debian7 installs and runs
> >>>> perfectly, same as centos6 on all of my 4 proxmox nodes.
> >>>>
> >>>> I did md5sum of the installation iso, it is ok.
> >>>>
> >>>> I'll try today Latest Centos and Ubuntu just to check.
> >>>>
> >>>> But please, test the same in your lab or tell me what should i check
> >>>> for to help you to trace the bug.. There are no bad logs in gluster log
> >>>> files also.
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best regards,
> >>>> Roman.
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>> Roman.
> >>>
> >>
> >>
> >>
> >> --
> >> Best regards,
> >> Roman.
> >>
> >
> >
> >
> > --
> > Best regards,
> > Roman.
> >
> 
> 
> 
> -- 
> Best regards,
> Roman.

> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel