[Gluster-users] Gluster terror loss data !

[higkoohk]

Hello everyone,

   It's a terror thing that meat data lose...  The env like this :
pveversion
pve-manager/3.1-3/dc0e9b0e (running kernel: 2.6.32-23-pve)

pvesm status
file-storage-c2  glusterfs 1       787924096        35682688
716371200 5.24%
vm-images-c2     glusterfs 1       364029184        17849088
331842688 5.60%
vm-images-c9     glusterfs 1      1766111360        42094336
1634303360 3.01%

gluster --version
glusterfs 3.4.0 built on Jul 19 2013 03:56:40

gluster volume info
Volume Name: vm-images-c9
Type: Striped-Replicate
Volume ID: 6a30d329-1ea2-42de-8c7d-b13964941a64
Status: Started
Number of Bricks: 1 x 14 x 2 = 28
Transport-type: tcp
Bricks:
Brick1: agent21.kisops.org:/d2/vm-images-c9
Brick2: agent22.kisops.org:/d2/vm-images-c9
Brick3: agent23.kisops.org:/d1/vm-images-c9
Brick4: agent24.kisops.org:/d1/vm-images-c9
Brick5: agent25.kisops.org:/d1/vm-images-c9
Brick6: agent26.kisops.org:/d1/vm-images-c9
Brick7: agent27.kisops.org:/d1/vm-images-c9
Brick8: agent28.kisops.org:/d1/vm-images-c9
Brick9: agent29.kisops.org:/d0/vm-images-c9
Brick10: agent30.kisops.org:/d1/vm-images-c9
Brick11: agent31.kisops.org:/d1/vm-images-c9
Brick12: agent32.kisops.org:/d1/vm-images-c9
Brick13: agent21.kisops.org:/d3/vm-images-c9
Brick14: agent22.kisops.org:/d3/vm-images-c9
Brick15: agent23.kisops.org:/d2/vm-images-c9
Brick16: agent24.kisops.org:/d2/vm-images-c9
Brick17: agent25.kisops.org:/d2/vm-images-c9
Brick18: agent26.kisops.org:/d2/vm-images-c9
Brick19: agent27.kisops.org:/d2/vm-images-c9
Brick20: agent30.kisops.org:/d2/vm-images-c9
Brick21: agent31.kisops.org:/d2/vm-images-c9
Brick22: agent23.kisops.org:/d3/vm-images-c9
Brick23: agent24.kisops.org:/d3/vm-images-c9
Brick24: agent25.kisops.org:/d3/vm-images-c9
Brick25: agent26.kisops.org:/d3/vm-images-c9
Brick26: agent27.kisops.org:/d3/vm-images-c9
Brick27: agent30.kisops.org:/d3/vm-images-c9
Brick28: agent31.kisops.org:/d3/vm-images-c9

/usr/bin/kvm -id 107 -chardev
socket,id=qmp,path=/var/run/qemu-server/107.qmp,server,nowait -mon
chardev=qmp,mode=control -vnc
unix:/var/run/qemu-server/107.vnc,x509,password -pidfile
/var/run/qemu-server/107.pid -daemonize -name ngrinder-Console -smp
sockets=1,cores=3 -nodefaults -boot menu=on -vga cirrus -cpu
kvm64,+x2apic,+sep -k en-us -m 6144 -device
piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device
usb-tablet,id=tablet,bus=uhci.0,port=1 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -drive
if=none,id=drive-ide2,media=cdrom,aio=native -device
ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -drive
file=gluster://
agent21.kisops.org/vm-images-c9/images/107/vm-107-disk-1.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=native-device
virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100
-netdev
type=tap,id=net0,ifname=tap107i0,script=/var/lib/qemu-server/pve-bridge,vhost=on
-device
virtio-net-pci,mac=0E:17:15:37:D4:83,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300

cat /etc/pve/nodes/agent21/qemu-server/107.conf
#Ngrinder-Controller
balloon: 6
bootdisk: virtio0
cores: 3
ide2: none,media=cdrom
memory: 6144
name: ngrinder-Console
net0: virtio=0E:17:15:37:D4:83,bridge=vmbr0
ostype: l26
sockets: 1
virtio0:
vm-images-c9:107/vm-107-disk-1.qcow2,format=qcow2,cache=writeback,backup=no,size=88G


   When on node 'agent24.kisops.org' shutdown by errors ,then the VMs
errors :
[image: 内嵌图片 1]
reboot after errors :
[image: 内嵌图片 2]
some VMs could be OK after exec `fsck /vda1`,others couldn't up.
kvm: -drive file=gluster://
agent21.kisops.org/vm-images-c9/images/109/vm-109-disk-2.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=native:
could not open disk image gluster://
agent21.kisops.org/vm-images-c9/images/109/vm-109-disk-2.qcow2:
Input/output error
TASK ERROR: start failed: command '/usr/bin/kvm -id 109 -chardev
'socket,id=qmp,path=/var/run/qemu-server/109.qmp,server,nowait' -mon
'chardev=qmp,mode=control' -vnc
unix:/var/run/qemu-server/109.vnc,x509,password -pidfile
/var/run/qemu-server/109.pid -daemonize -name Ngrinder-Cient-26 -smp
'sockets=1,cores=2' -nodefaults -boot 'menu=on' -vga cirrus -cpu
kvm64,+x2apic,+sep -k en-us -m 4096 -device
'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device
'usb-tablet,id=tablet,bus=uhci.0,port=1' -device
'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -drive
'if=none,id=drive-ide2,media=cdrom,aio=native' -device
'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive
'file=gluster://
10.20.197.21/vm-images-c9/images/109/vm-109-disk-2.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=native'
-device
'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100'
-netdev
'type=tap,id=net0,ifname=tap109i0,script=/var/lib/qemu-server/pve-bridge,vhost=on'
-device
'virtio-net-pci,mac=0A:0E:47:39:76:23,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300''
failed: exit code 1

I see there are io error on GlusterFS even 'agent24.kisops.org' is up :
du -sh /mnt/pve/vm-images-c9/images/109/*
1.9G     /mnt/pve/vm-images-c9/images/109/vm-109-disk-1.raw
du: cannot access `/mnt/pve/vm-images-c9/images/109/vm-109-disk-2.qcow2':
Input/output error


If I shut down 'agent24.kisops.org' , the GlusterFS dir will be Empty.

What's wrong with me ? Help !
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131008/90b8833a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 57511 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131008/90b8833a/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 64720 bytes
Desc: not available
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131008/90b8833a/attachment-0001.png>