[Gluster-users] Gluster 2.6 and infiniband

[bxmatus at gmail.com]

Hello,

at first it was tcp then tcp,rdma.

You are right that without tcp definition ".rdma" is not working. But
now i have another problem.
I'm trying tcp / rdma, im trying even tcp/rdma using normal network
card ( not using infiniband IP but normal 1gbit network
card and i have still same speed, upload about 30mb/s and download
about 200mb/s .. so i'm not sure if rdma is even working.

Native infiniband is giving me 3500mb/s speed with benchmark tests
(ib_rdma_bw ).

thanks

Matus

2012/6/7 Amar Tumballi <amarts at redhat.com>:
> On 06/07/2012 02:04 PM, bxmatus at gmail.com wrote:
>>
>> Hello,
>>
>> i have a problem with gluster 3.2.6 and infiniband. With gluster 3.3
>> its working ok but with 3.2.6 i have following problems:
>>
>> when i'm trying to mount rdma volume using command mount -t glusterfs
>> 192.168.100.1:/atlas1.rdma mount  i get:
>>
>> [2012-06-07 04:30:18.894337] I [glusterfsd.c:1493:main]
>> 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs
>> version 3.2.6
>> [2012-06-07 04:30:18.907499] E
>> [glusterfsd-mgmt.c:628:mgmt_getspec_cbk] 0-glusterfs: failed to get
>> the 'volume file' from server
>> [2012-06-07 04:30:18.907592] E
>> [glusterfsd-mgmt.c:695:mgmt_getspec_cbk] 0-mgmt: failed to fetch
>> volume file (key:/atlas1.rdma)
>> [2012-06-07 04:30:18.907995] W [glusterfsd.c:727:cleanup_and_exit]
>> (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0xc9)
>> [0x7f784e2c8bc9] (-->/usr/local/
>> lib/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5) [0x7f784e2c8975]
>> (-->/usr/local/sbin/glusterfs(mgmt_getspec_cbk+0x28b) [0x40861b])))
>> 0-: received signum (0)
>> , shutting down
>> [2012-06-07 04:30:18.908049] I [fuse-bridge.c:3727:fini] 0-fuse:
>> Unmounting 'mount'.
>>
>> Same command without ".rdma" works ok.
>>
>
> Is the volume's transport type only 'rdma' ? or 'tcp,rdma' ? If its only
> 'rdma', then appending ".rdma" to volume name is not required. The appending
> of ".rdma" is only required when there are both type of transports on a
> volume (ie, 'tcp,rdma'), as from the client you can decide which transport
> you want to mount.
>
> default volume name would point to 'tcp' transport type, and appending
> ".rdma", will point to rdma transport type.
>
> Hope that is clear now.
>
> Regards,
> Amar