[Gluster-users] Performance
Todd Daugherty
todd at fotokem.hu
Mon Jun 7 21:36:59 UTC 2010
I included a bunch of other info. If there is anything else that will
help solve this problem I am very interested in getting to the bottom
of this. Side note I built a GlusterFS cluster (2.0.9) from RAM disks
(10 gig /dev/ram0 bricks) That performs quite well but still much less
than the backend /dev/ram0 bricks.
Todd
InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s
- IB QDR / 10GigE] (rev a0)
Atto Technology SAS RAID Adapter Subsystem: Atto Technology Device
002d Kernel modules: esasraid
16 SATA 1 TB Disks on two of those Atto Cards so 8 disks each
controller. (JMR PE SAN)
dd if=/dev/zero of=/node01/zero bs=1M count=16386 oflag=direct
16386+0 records in
16386+0 records out
17181966336 bytes (17 GB) copied, 17.0376 s, 1.0 GB/s
6 Gigabytes of RAM
glusterfs -V
glusterfs 2.0.9 built on Apr 11 2010 20:40:04
Repository revision: v2.0.9
Copyright (c) 2006-2009 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.77
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 1
cpu cores : 4
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.07
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 2
cpu cores : 4
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.07
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
apicid : 6
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.07
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 4
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.07
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 5
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 2668.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 1
cpu cores : 4
apicid : 3
initial apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.07
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 6
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 2
cpu cores : 4
apicid : 5
initial apicid : 5
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.08
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
stepping : 5
cpu MHz : 1600.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips : 5319.07
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
cat /proc/meminfo
MemTotal: 6060364 kB
MemFree: 137120 kB
Buffers: 176488 kB
Cached: 5225760 kB
SwapCached: 0 kB
Active: 319020 kB
Inactive: 5200428 kB
Active(anon): 13216 kB
Inactive(anon): 104236 kB
Active(file): 305804 kB
Inactive(file): 5096192 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 8110072 kB
SwapFree: 8110072 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 117264 kB
Mapped: 17232 kB
Slab: 208168 kB
SReclaimable: 177444 kB
SUnreclaim: 30724 kB
PageTables: 5492 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 11140252 kB
Committed_AS: 290736 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 320548 kB
VmallocChunk: 34359397567 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 7744 kB
DirectMap2M: 6275072 kB
uname -a
Linux axon07 2.6.29.4-167.fc11.x86_64 #1 SMP Wed May 27 17:27:08 EDT
2009 x86_64 x86_64 x86_64 GNU/Linux
lshw
axon07
description: Desktop Computer
product: X8SAX
vendor: Supermicro
version: 1234567890
serial: 1234567890
width: 64 bits
capabilities: vsyscall64 vsyscall32
configuration: boot=normal chassis=desktop
uuid=00020003-0004-0005-0006-000700080009
*-core
description: Motherboard
product: X8SAX
vendor: Supermicro
physical id: 0
version: 1234567890
serial: 1234567890
slot: To Be Filled By O.E.M.
*-firmware
description: BIOS
vendor: American Megatrends Inc.
physical id: 0
version: 1.0c (08/05/2009)
size: 64KiB
capacity: 4032KiB
capabilities: isa pci pnp upgrade shadowing escd cdboot
bootselect socketedrom edd int13floppy1200 int13floppy720
int13floppy2880 int5printscreen int9keyboard int14serial int17printer
int10video acpi usb ls120boot zipboot biosbootspecification
*-cpu
description: CPU
product: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
vendor: Intel Corp.
physical id: 4
bus info: cpu at 0
version: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz
serial: To Be Filled By O.E.M.
slot: CPU 1
size: 1600MHz
capacity: 1600MHz
width: 64 bits
clock: 133MHz
capabilities: fpu fpu_exception wp vme de pse tsc msr pae
mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse
sse2 ss ht tm pbe syscall nx rdtscp x86-64 constant_tsc arch_perfmon
pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx
est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi
flexpriority ept vpid cpufreq
*-cache:0
description: L1 cache
physical id: 5
slot: L1-Cache
size: 256KiB
capacity: 256KiB
capabilities: internal write-back data
*-cache:1
description: L2 cache
physical id: 6
slot: L2-Cache
size: 1MiB
capacity: 1MiB
capabilities: internal write-back unified
*-cache:2
description: L3 cache
physical id: 7
slot: L3-Cache
size: 8MiB
capacity: 8MiB
capabilities: internal write-back unified
*-memory:0
description: System Memory
physical id: 28
slot: System board or motherboard
*-bank:0
description: DIMM 1333 MHz (0.8 ns)
vendor: Manufacturer00
physical id: 0
serial: 00000000
slot: DIMM0
size: 2GiB
width: 64 bits
clock: 1333MHz (0.8ns)
*-bank:1
description: DIMM 1333 MHz (0.8 ns) [empty]
product: ModulePartNumber01
vendor: Manufacturer01
physical id: 1
serial: SerNum01
slot: DIMM1
width: 64 bits
clock: 1333MHz (0.8ns)
*-bank:2
description: DIMM 1333 MHz (0.8 ns)
vendor: Manufacturer02
physical id: 2
serial: 00000000
slot: DIMM2
size: 2GiB
width: 64 bits
clock: 1333MHz (0.8ns)
*-bank:3
description: DIMM 1333 MHz (0.8 ns) [empty]
product: ModulePartNumber03
vendor: Manufacturer03
physical id: 3
serial: SerNum03
slot: DIMM3
width: 64 bits
clock: 1333MHz (0.8ns)
*-bank:4
description: DIMM 1333 MHz (0.8 ns)
vendor: Manufacturer04
physical id: 4
serial: 00000000
slot: DIMM4
size: 2GiB
width: 64 bits
clock: 1333MHz (0.8ns)
*-bank:5
description: DIMM 1333 MHz (0.8 ns) [empty]
product: ModulePartNumber05
vendor: Manufacturer05
physical id: 5
serial: SerNum05
slot: DIMM5
width: 64 bits
clock: 1333MHz (0.8ns)
*-memory:1 UNCLAIMED
description: Flash Memory
physical id: 36
slot: System board or motherboard
capacity: 4MiB
*-bank UNCLAIMED
description: FLASH Non-volatile 33 MHz (30.3 ns)
product: 26DF321
vendor: ATMEL
physical id: 0
size: 4MiB
width: 8 bits
clock: 33MHz (30.3ns)
*-memory:2 UNCLAIMED
physical id: 1
*-memory:3 UNCLAIMED
physical id: 2
*-pci
description: Host bridge
product: X58 I/O Hub to ESI Port
vendor: Intel Corporation
physical id: 100
bus info: pci at 0000:00:00.0
version: 13
width: 32 bits
clock: 33MHz
*-pci:0
description: PCI bridge
product: X58 I/O Hub PCI Express Root Port 1
vendor: Intel Corporation
physical id: 1
bus info: pci at 0000:00:01.0
version: 13
width: 32 bits
clock: 33MHz
capabilities: pci msi pciexpress pm normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:24
*-pci
description: PCI bridge
product: 6702PXH PCI Express-to-PCI Bridge A
vendor: Intel Corporation
physical id: 0
bus info: pci at 0000:01:00.0
version: 09
width: 32 bits
clock: 33MHz
capabilities: pci pciexpress msi pm pcix normal_decode
bus_master cap_list
*-pci:1
description: PCI bridge
product: X58 I/O Hub PCI Express Root Port 3
vendor: Intel Corporation
physical id: 3
bus info: pci at 0000:00:03.0
version: 13
width: 32 bits
clock: 33MHz
capabilities: pci msi pciexpress pm normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:25 ioport:a000(size=8192) memory:fb500000-fb9fffff
*-pci
description: PCI bridge
product: PEX 8533 32-lane, 6-port PCI Express Switch
vendor: PLX Technology, Inc.
physical id: 0
bus info: pci at 0000:03:00.0
version: aa
width: 32 bits
clock: 33MHz
capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:30 memory:fb5e0000-fb5fffff
ioport:a000(size=8192) memory:fb600000-fb9fffff
*-pci:0
description: PCI bridge
product: PEX 8533 32-lane, 6-port PCI Express Switch
vendor: PLX Technology, Inc.
physical id: 0
bus info: pci at 0000:04:00.0
version: aa
width: 64 bits
clock: 33MHz
capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: iomemory:1f10-1f0f irq:31
*-pci:1
description: PCI bridge
product: PEX 8533 32-lane, 6-port PCI Express Switch
vendor: PLX Technology, Inc.
physical id: 1
bus info: pci at 0000:04:01.0
version: aa
width: 64 bits
clock: 33MHz
capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: iomemory:a1a10-a1a0f irq:32
ioport:a000(size=4096) memory:fb600000-fb7fffff
*-storage
description: RAID bus controller
product: SAS RAID Adapter
vendor: Atto Technology
physical id: 0
bus info: pci at 0000:06:00.0
logical name: scsi4
version: 00
width: 64 bits
clock: 33MHz
capabilities: storage pm msi pciexpress
bus_master cap_list rom scsi-host
configuration: driver=esasraid latency=0
resources: irq:17 memory:fb780000-fb7fffff
memory:fb700000-fb77ffff ioport:a800(size=256)
memory:fb600000-fb6fffff(prefetchable)
*-enclosure UNCLAIMED
description: SCSI Enclosure
product: PSAN SAS Exp
vendor: JMR Elec
physical id: 0.0.0
bus info: scsi at 4:0.0.0
version: 2.17
configuration: ansiversion=3
*-disk
description: SCSI Disk
product: test0100
vendor: ATTO
physical id: 0.1.0
bus info: scsi at 4:0.1.0
logical name: /dev/sdb
version: 0001
serial: c3da063f774df7d71d996b020ac16ca400000000
size: 7451GiB (8001GB)
configuration: ansiversion=5
*-pci:2
description: PCI bridge
product: PEX 8533 32-lane, 6-port PCI Express Switch
vendor: PLX Technology, Inc.
physical id: 9
bus info: pci at 0000:04:09.0
version: aa
width: 64 bits
clock: 33MHz
capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: iomemory:b1b10-b1b0f irq:33
ioport:b000(size=4096) memory:fb800000-fb9fffff
*-storage
description: RAID bus controller
product: SAS RAID Adapter
vendor: Atto Technology
physical id: 0
bus info: pci at 0000:07:00.0
logical name: scsi5
version: 00
width: 64 bits
clock: 33MHz
capabilities: storage pm msi pciexpress
bus_master cap_list rom scsi-host
configuration: driver=esasraid latency=0
resources: irq:17 memory:fb980000-fb9fffff
memory:fb900000-fb97ffff ioport:b800(size=256)
memory:fb800000-fb8fffff(prefetchable)
*-enclosure UNCLAIMED
description: SCSI Enclosure
product: PSAN SAS Exp
vendor: JMR Elec
physical id: 0.0.0
bus info: scsi at 5:0.0.0
version: 2.17
configuration: ansiversion=3
*-disk
description: SCSI Disk
product: test0200
vendor: ATTO
physical id: 0.1.0
bus info: scsi at 5:0.1.0
logical name: /dev/sdc
version: 0001
serial: cad720b12a0778fe517f60962c34285400000000
size: 7451GiB (8001GB)
configuration: ansiversion=5
*-pci:2
description: PCI bridge
product: X58 I/O Hub PCI Express Root Port 7
vendor: Intel Corporation
physical id: 7
bus info: pci at 0000:00:07.0
version: 13
width: 32 bits
clock: 33MHz
capabilities: pci msi pciexpress pm normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:26 ioport:c000(size=4096)
memory:fba00000-fbafffff ioport:d0000000(size=268435456)
*-display
description: VGA compatible controller
product: RV710 [Radeon HD 4350]
vendor: ATI Technologies Inc
physical id: 0
bus info: pci at 0000:08:00.0
version: 00
width: 64 bits
clock: 33MHz
capabilities: pm pciexpress msi vga_controller
bus_master cap_list rom
configuration: driver=radeon latency=0
resources: irq:16
memory:d0000000-dfffffff(prefetchable) memory:fbaf0000-fbafffff
ioport:c000(size=256) memory:fbac0000-fbadffff(prefetchable)
*-multimedia
description: Audio device
product: R700 Audio Device [Radeon HD 4000 Series]
vendor: ATI Technologies Inc
physical id: 0.1
bus info: pci at 0000:08:00.1
version: 00
width: 64 bits
clock: 33MHz
capabilities: pm pciexpress msi bus_master cap_list
configuration: driver=HDA Intel latency=0
resources: irq:17 memory:fbaec000-fbaeffff
*-generic:0 UNCLAIMED
description: PIC
product: X58 I/O Hub System Management Registers
vendor: Intel Corporation
physical id: 14
bus info: pci at 0000:00:14.0
version: 13
width: 32 bits
clock: 33MHz
capabilities: pciexpress 8259 cap_list
configuration: latency=0
*-generic:1 UNCLAIMED
description: PIC
product: X58 I/O Hub GPIO and Scratch Pad Registers
vendor: Intel Corporation
physical id: 14.1
bus info: pci at 0000:00:14.1
version: 13
width: 32 bits
clock: 33MHz
capabilities: pciexpress 8259 cap_list
configuration: latency=0
*-generic:2 UNCLAIMED
description: PIC
product: X58 I/O Hub Control Status and RAS Registers
vendor: Intel Corporation
physical id: 14.2
bus info: pci at 0000:00:14.2
version: 13
width: 32 bits
clock: 33MHz
capabilities: pciexpress 8259 cap_list
configuration: latency=0
*-generic:3 UNCLAIMED
description: PIC
product: X58 I/O Hub Throttle Registers
vendor: Intel Corporation
physical id: 14.3
bus info: pci at 0000:00:14.3
version: 13
width: 32 bits
clock: 33MHz
capabilities: 8259
configuration: latency=0
*-generic:4 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16
bus info: pci at 0000:00:16.0
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4fc000-fb4fffff
*-generic:5 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16.1
bus info: pci at 0000:00:16.1
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4f8000-fb4fbfff
*-generic:6 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16.2
bus info: pci at 0000:00:16.2
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4f4000-fb4f7fff
*-generic:7 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16.3
bus info: pci at 0000:00:16.3
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4f0000-fb4f3fff
*-generic:8 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16.4
bus info: pci at 0000:00:16.4
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4ec000-fb4effff
*-generic:9 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16.5
bus info: pci at 0000:00:16.5
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4e8000-fb4ebfff
*-generic:10 UNCLAIMED
description: System peripheral
product: X58 Chipset QuickData Technology Device
vendor: Intel Corporation
physical id: 16.6
bus info: pci at 0000:00:16.6
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4e4000-fb4e7fff
*-generic:11 UNCLAIMED
description: System peripheral
product: DMA Engine
vendor: Intel Corporation
physical id: 16.7
bus info: pci at 0000:00:16.7
version: 13
width: 64 bits
clock: 33MHz
capabilities: msix pciexpress pm bus_master cap_list
configuration: latency=0
resources: memory:fb4e0000-fb4e3fff
*-usb:0
description: USB Controller
product: 82801JI (ICH10 Family) USB UHCI Controller #4
vendor: Intel Corporation
physical id: 1a
bus info: pci at 0000:00:1a.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: uhci bus_master cap_list
configuration: driver=uhci_hcd latency=0
resources: irq:20 ioport:9c00(size=32)
*-usbhost
product: UHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
physical id: 1
bus info: usb at 3
logical name: /dev/usb3
version: 2.06
capabilities: usb-1.10
configuration: driver=hub slots=2 speed=12.0MB/s
*-usb:1
description: USB Controller
product: 82801JI (ICH10 Family) USB UHCI Controller #5
vendor: Intel Corporation
physical id: 1a.1
bus info: pci at 0000:00:1a.1
version: 00
width: 32 bits
clock: 33MHz
capabilities: uhci bus_master cap_list
configuration: driver=uhci_hcd latency=0
resources: irq:21 ioport:9880(size=32)
*-usbhost
product: UHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
physical id: 1
bus info: usb at 4
logical name: /dev/usb4
version: 2.06
capabilities: usb-1.10
configuration: driver=hub slots=2 speed=12.0MB/s
*-usb:2
description: USB Controller
product: 82801JI (ICH10 Family) USB UHCI Controller #6
vendor: Intel Corporation
physical id: 1a.2
bus info: pci at 0000:00:1a.2
version: 00
width: 32 bits
clock: 33MHz
capabilities: uhci bus_master cap_list
configuration: driver=uhci_hcd latency=0
resources: irq:23 ioport:9800(size=32)
*-usbhost
product: UHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
physical id: 1
bus info: usb at 5
logical name: /dev/usb5
version: 2.06
capabilities: usb-1.10
configuration: driver=hub slots=2 speed=12.0MB/s
*-usb:3
description: USB Controller
product: 82801JI (ICH10 Family) USB2 EHCI Controller #2
vendor: Intel Corporation
physical id: 1a.7
bus info: pci at 0000:00:1a.7
version: 00
width: 32 bits
clock: 33MHz
capabilities: pm debug ehci bus_master cap_list
configuration: driver=ehci_hcd latency=0
resources: irq:22 memory:fb4de000-fb4de3ff
*-usbhost
product: EHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 ehci_hcd
physical id: 1
bus info: usb at 1
logical name: /dev/usb1
version: 2.06
capabilities: usb-2.00
configuration: driver=hub slots=6 speed=480.0MB/s
*-multimedia
description: Audio device
product: 82801JI (ICH10 Family) HD Audio Controller
vendor: Intel Corporation
physical id: 1b
bus info: pci at 0000:00:1b.0
version: 00
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress bus_master cap_list
configuration: driver=HDA Intel latency=0
resources: irq:22 memory:fb4d8000-fb4dbfff
*-pci:3
description: PCI bridge
product: 82801JI (ICH10 Family) PCI Express Port 1
vendor: Intel Corporation
physical id: 1c
bus info: pci at 0000:00:1c.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: pci pciexpress msi pm normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:27 memory:fbb00000-fbbfffff
ioport:fa800000(size=8388608)
*-serial DISABLED
description: interface
product: MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE]
vendor: Mellanox Technologies
physical id: 0
bus info: pci at 0000:09:00.0
logical name: ib0
version: a0
serial: 80:00:00:48:fe:80
width: 64 bits
clock: 33MHz
capabilities: pm vpd msix pciexpress bus_master
cap_list physical
configuration: broadcast=yes driver=ipoib latency=0
multicast=yes
resources: irq:16 memory:fbb00000-fbbfffff
memory:fa800000-faffffff(prefetchable)
*-pci:4
description: PCI bridge
product: 82801JI (ICH10 Family) PCI Express Port 5
vendor: Intel Corporation
physical id: 1c.4
bus info: pci at 0000:00:1c.4
version: 00
width: 32 bits
clock: 33MHz
capabilities: pci pciexpress msi pm normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:28 ioport:d000(size=4096) memory:fbc00000-fbcfffff
*-network
description: Ethernet interface
product: 82574L Gigabit Network Connection
vendor: Intel Corporation
physical id: 0
bus info: pci at 0000:0a:00.0
logical name: eth0
version: 00
serial: 00:30:48:f9:b3:9a
size: 1GB/s
capacity: 1GB/s
width: 32 bits
clock: 33MHz
capabilities: pm msi pciexpress msix bus_master
cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd
autonegotiation
configuration: autonegotiation=on broadcast=yes
driver=e1000e driverversion=0.3.3.3-k6 duplex=full firmware=1.8-0
ip=192.168.0.117 latency=0 link=yes multicast=yes port=twisted pair
speed=1GB/s
resources: irq:16 memory:fbce0000-fbcfffff
ioport:dc00(size=32) memory:fbcdc000-fbcdffff
*-pci:5
description: PCI bridge
product: 82801JI (ICH10 Family) PCI Express Port 6
vendor: Intel Corporation
physical id: 1c.5
bus info: pci at 0000:00:1c.5
version: 00
width: 32 bits
clock: 33MHz
capabilities: pci pciexpress msi pm normal_decode
bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:29 ioport:e000(size=4096) memory:fbd00000-fbdfffff
*-network
description: Ethernet interface
product: 82574L Gigabit Network Connection
vendor: Intel Corporation
physical id: 0
bus info: pci at 0000:0b:00.0
logical name: eth1
version: 00
serial: 00:30:48:f9:b3:9b
capacity: 1GB/s
width: 32 bits
clock: 33MHz
capabilities: pm msi pciexpress msix bus_master
cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd
autonegotiation
configuration: autonegotiation=on broadcast=yes
driver=e1000e driverversion=0.3.3.3-k6 firmware=1.8-0 latency=0
link=no multicast=yes port=twisted pair
resources: irq:17 memory:fbde0000-fbdfffff
ioport:ec00(size=32) memory:fbddc000-fbddffff
*-usb:4
description: USB Controller
product: 82801JI (ICH10 Family) USB UHCI Controller #1
vendor: Intel Corporation
physical id: 1d
bus info: pci at 0000:00:1d.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: uhci bus_master cap_list
configuration: driver=uhci_hcd latency=0
resources: irq:20 ioport:9480(size=32)
*-usbhost
product: UHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
physical id: 1
bus info: usb at 6
logical name: /dev/usb6
version: 2.06
capabilities: usb-1.10
configuration: driver=hub slots=2 speed=12.0MB/s
*-usb:5
description: USB Controller
product: 82801JI (ICH10 Family) USB UHCI Controller #2
vendor: Intel Corporation
physical id: 1d.1
bus info: pci at 0000:00:1d.1
version: 00
width: 32 bits
clock: 33MHz
capabilities: uhci bus_master cap_list
configuration: driver=uhci_hcd latency=0
resources: irq:21 ioport:9400(size=32)
*-usbhost
product: UHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
physical id: 1
bus info: usb at 7
logical name: /dev/usb7
version: 2.06
capabilities: usb-1.10
configuration: driver=hub slots=2 speed=12.0MB/s
*-usb:6
description: USB Controller
product: 82801JI (ICH10 Family) USB UHCI Controller #3
vendor: Intel Corporation
physical id: 1d.2
bus info: pci at 0000:00:1d.2
version: 00
width: 32 bits
clock: 33MHz
capabilities: uhci bus_master cap_list
configuration: driver=uhci_hcd latency=0
resources: irq:22 ioport:9080(size=32)
*-usbhost
product: UHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
physical id: 1
bus info: usb at 8
logical name: /dev/usb8
version: 2.06
capabilities: usb-1.10
configuration: driver=hub slots=2 speed=12.0MB/s
*-usb:7
description: USB Controller
product: 82801JI (ICH10 Family) USB2 EHCI Controller #1
vendor: Intel Corporation
physical id: 1d.7
bus info: pci at 0000:00:1d.7
version: 00
width: 32 bits
clock: 33MHz
capabilities: pm debug ehci bus_master cap_list
configuration: driver=ehci_hcd latency=0
resources: irq:20 memory:fb4dc000-fb4dc3ff
*-usbhost
product: EHCI Host Controller
vendor: Linux 2.6.29.4-167.fc11.x86_64 ehci_hcd
physical id: 1
bus info: usb at 2
logical name: /dev/usb2
version: 2.06
capabilities: usb-2.00
configuration: driver=hub slots=6 speed=480.0MB/s
*-pci:6
description: PCI bridge
product: 82801 PCI Bridge
vendor: Intel Corporation
physical id: 1e
bus info: pci at 0000:00:1e.0
version: 90
width: 32 bits
clock: 33MHz
capabilities: pci subtractive_decode bus_master cap_list
resources: memory:fbe00000-fbefffff
*-firewire
description: FireWire (IEEE 1394)
product: TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)
vendor: Texas Instruments
physical id: 8
bus info: pci at 0000:0c:08.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: pm ohci bus_master cap_list
configuration: driver=firewire_ohci latency=64
maxlatency=4 mingnt=2
resources: irq:17 memory:fbeff800-fbefffff
memory:fbef8000-fbefbfff
*-isa
description: ISA bridge
product: 82801JIR (ICH10R) LPC Interface Controller
vendor: Intel Corporation
physical id: 1f
bus info: pci at 0000:00:1f.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: isa bus_master cap_list
configuration: latency=0
*-ide:0
description: IDE interface
product: 82801JI (ICH10 Family) 4 port SATA IDE Controller
vendor: Intel Corporation
physical id: 1f.2
bus info: pci at 0000:00:1f.2
logical name: scsi0
logical name: scsi1
version: 00
width: 32 bits
clock: 66MHz
capabilities: ide pm bus_master cap_list emulated
configuration: driver=ata_piix latency=0
resources: irq:19 ioport:1f0(size=8) ioport:3f6
ioport:170(size=8) ioport:376 ioport:ff90(size=16)
ioport:ffa0(size=16)
*-disk
description: ATA Disk
product: SAMSUNG HD753LJ
physical id: 0.0.0
bus info: scsi at 0:0.0.0
logical name: /dev/sda
version: 1AA0
serial: S13UJ9FZ101808
size: 698GiB (750GB)
capabilities: partitioned partitioned:dos
configuration: ansiversion=5 signature=0007d825
*-volume:0
description: EXT3 volume
vendor: Linux
physical id: 1
bus info: scsi at 0:0.0.0,1
logical name: /dev/sda1
logical name: /boot
version: 1.0
serial: fc544031-e0c9-4cc9-86c0-583efd5b263d
size: 200MiB
capacity: 200MiB
capabilities: primary bootable journaled
extended_attributes ext3 ext2 initialized
configuration: created=2010-05-19 21:18:54
filesystem=ext3 modified=2010-06-05 13:26:21 mount.fstype=ext3
mount.options=rw,relatime,errors=continue,data=ordered
mounted=2010-06-02 18:15:25 state=mounted
*-volume:1
description: Linux LVM Physical Volume partition
physical id: 2
bus info: scsi at 0:0.0.0,2
logical name: /dev/sda2
serial: eOHevd-eVLc-aqpu-LN8x-ariz-JPI2-jb0Tkr
size: 698GiB
capacity: 698GiB
capabilities: primary multi lvm2
*-cdrom
description: DVD-RAM writer
product: DVDRAM GH22NS40
vendor: HL-DT-ST
physical id: 0.1.0
bus info: scsi at 1:0.1.0
logical name: /dev/cdrom
logical name: /dev/cdrw
logical name: /dev/dvd
logical name: /dev/dvdrw
logical name: /dev/scd0
logical name: /dev/sr0
version: NL02
capabilities: removable audio cd-r cd-rw dvd dvd-r dvd-ram
configuration: ansiversion=5 status=nodisc
*-serial
description: SMBus
product: 82801JI (ICH10 Family) SMBus Controller
vendor: Intel Corporation
physical id: 1f.3
bus info: pci at 0000:00:1f.3
version: 00
width: 64 bits
clock: 33MHz
configuration: driver=i801_smbus latency=0
resources: irq:18 memory:fb4d6000-fb4d60ff ioport:400(size=32)
*-ide:1
description: IDE interface
product: 82801JI (ICH10 Family) 2 port SATA IDE Controller
vendor: Intel Corporation
physical id: 1f.5
bus info: pci at 0000:00:1f.5
version: 00
width: 32 bits
clock: 66MHz
capabilities: ide pm bus_master cap_list
configuration: driver=ata_piix latency=0
resources: irq:19 ioport:8c00(size=8) ioport:8880(size=4)
ioport:8800(size=8) ioport:8480(size=4) ioport:8400(size=16)
ioport:8080(size=16)
On Mon, Jun 7, 2010 at 10:52 PM, Harshavardhana <harsha at gluster.com> wrote:
> On 06/07/2010 06:22 AM, Todd Daugherty wrote:
>>
>> I have run iozone on the same machine with GlusterFS and without
>> GlusterFS. (Local storage and Local storage via GlusterFS). I also
>> included the results from a remote machine. The remote machine and the
>> local glusterfs are the same. I am getting around 200 megabytes per
>> second on reads. My local storage will do around 1400 megabytes per
>> second. Any ideas why?
>>
>> Todd
>>
>>
>
> To objectively understand the variables here, you need to provide us details
> you the hardware including infiniband card, RAM size. Backend disk
> performance and type of disks (Raid controller).
>
> Backend disk performance should be with "dd if=/dev/zero
> of=/<disk_mountpoint> bs=1M count=16386 oflag=direct"
>
> Regards
>
> --
> Harshavardhana
> Gluster Inc - http://www.gluster.com
> +1(408)-770-1887, Ext-113
> +1(408)-480-1730
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
More information about the Gluster-users
mailing list