[Gluster-users] KVM guest I/O errors with xfs backed gluster volumes

[Stefan Hajnoczi]

On Tue, Oct 29, 2013 at 8:51 AM, Anand Avati <avati at gluster.org> wrote:
> Looks like what is happening is that qemu performs ioctls() on the backend
> to query logical_block_size (for direct IO alignment). That works on XFS,
> but fails on FUSE (hence qemu ends up performing IO with default 512
> alignment rather than 4k).
>
> Looks like this might be something we can enhance gluster driver in qemu.
> Note that glusterfs does not have an ioctl() FOP, but we could probably wire
> up a virtual xattr call for this purpose.
>
> Copying Bharata to check if he has other solutions in mind.

Hi Avati and Bharata,
QEMU's 4 KB sector status is as follows:

QEMU does not autodetect sector size but you can tell the guest by
setting -drive logical_block_size=,physical_block_size=,min_io_size=
properties on the QEMU command-line.  I'm not aware of an XFS sector
size ioctl() probe that you mentioned.

Note that QEMU can issue disk I/O internally when starting up (to
detect the image file format) or to access image format metadata
(qcow2, qed, etc).  This is usually not a problem if you explicitly
set -drive format=<format> on the command-line to skip file format
probe.  qcow2 operates on 64 KB clusters by default so 4 KB disks
work.

Finally, the BIOS running inside the guest is not 4 KB sector-aware.
Therefore, 4 KB sector disks can be used as data disks but attempting
to boot from them will hit I/O errors.

I'm happy to help if you want to discuss improvements and integration
with GlusterFS.

Stefan