[Gluster-users] pnfs(glusterfs-3.7.1 + ganesha-2.2) problem for clients layout commit

Tue Jun 30 08:39:30 UTC 2015

On 29/06/15 15:49, 莊尚豪 wrote:
>
> Hi Jiffin,
>
> If you have time, could you test multiple client to ganesha for pnfs?
>
> I  test a case that two client connect and translate data, and they 
> will suspend to translate.
>
> Something like CLIENT ID get conflict to each other, and these are 
> found wireshark in client.
>

Hi Ben,

I created new pnfs s with latest code for nfs-ganesha(V2.3-dev-9) and 
glusterfs (master)

I have three servers and two clients , all running with fedora20 (kernel 
3.19.3-100.fc20.x86_64) and having 1G network card in it.

I created  a volume "test" with following configuration in my enviroment
Volume Name: test
Type: Distribute
Volume ID: 77da4906-526c-4f0f-bd81-23eba2053e36
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: 10.19.96.123:/home/brick/b1
Brick2: 10.19.96.125:/home/brick/b2
Brick3: 10.19.96.127:/home/brick/b3
Options Reconfigured:
features.cache-invalidation: on
nfs.disable: on

I used following  dd command for 1gb and 10gb file in both clients

echo 3 > /proc/sys/vm/drop_caches;dd if=/dev/zero of=<mount point>/file 
bs=1000K count=1000 conv=sync ;
1000+0 records in
1000+0 records out
10240000000 bytes (10 GB) copied, 97.0755 s, 105 MB/s

echo 3 > /proc/sys/vm/drop_caches;dd if=/dev/zero of=<mount_point>file 
bs=10000K count=1000 conv=sync ;
1000+0 records in
1000+0 records out
1024000000 bytes (1.0 GB) copied, 10.013 s, 102 MB/s

It didn't hang for me. Also i can read file from client1 which is 
written from client2.

Also i tried to run distributed iozone sequential write test in which  
four files{iozone.DUMMY.[1-4] } of size 2G is written to two clients 
concurrently.
Still it didn't hang for me.
I used following command for that.
iozone -+m ./clients.ioz -+h 10.19.96.131 -i 0 -w -+n -c -C -e -s 8G -r 
64k -t 4 (clients.ioz -- iozone configuration file.)

It seems to be i can't reproduce your issue

Can u send ur packet trace in a file to me, so that i can analyze same 
in a wireshark?

And also send your test case in step by step, so that i can try the same 
in my environment.

I am attaching my iozone result with this mail

With regards,
Jiffin

>
> Thanks,
>
> Ben
>
> *From:*Jiffin Tony Thottan [mailto:jthottan at redhat.com]
> *Sent:* Friday, June 26, 2015 2:34 PM
> *To:* 莊尚豪; gluster-users at gluster.org
> *Subject:* Re: [Gluster-users] pnfs(glusterfs-3.7.1 + ganesha-2.2) 
> problem for clients layout commit
>
> On 26/06/15 08:25, 莊尚豪wrote:
>
>     Hi Jiffin,
>
>     Thanks your advice, but I still have these errors.
>
>     I suspect the pnfs clients causes errors because client can pass
>     successfully sometimes.
>
>     Can you tell me what the linux or nfs-utils version you test? Or
>     the requirement for client?
>
>
> As I said before, I ran perf tests (using dd, iozone) for large files 
> upto 10G two or three months back. I used fedora 20 clients and done 
> basic testing in RHEL6.5 clients
> Initially I had some issues with rhel clients, with new updates i 
> don't anything. Anyway I will run again same with latest sources and 
> give my updates on the same.
>
> Just make sure your all ganesha server is  up and exporting the 
> required volume. And more thing  , every servers should be resolvable 
> from m.d.s  (there is should information about all other server in 
> /etc/host).
>
> Regards,
> Jiffin
>
>
> Many thanks,
>
> Ben
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150630/2cbfc29b/attachment.html>
-------------- next part --------------
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.429 $
		Compiled for 64 bit mode.
		Build: linux-AMD64 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa.

	Run began: Tue Jun 30 02:16:42 2015

	Network distribution mode enabled.
	Hostname = 10.19.96.131
	Setting no_unlink
	No retest option selected
	Include close in write timing
	Include fsync in write timing
	File size set to 8388608 kB
	Record Size 64 kB
	Command line used: iozone -+m ./clients.ioz -+h 10.19.96.131 -i 0 -w -+n -c -C -e -s 8G -r 64k -t 4
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
	Throughput test with 4 processes
	Each process writes a 8388608 kByte file in 64 kByte records

	Test running:
	Children see throughput for  4 initial writers 	=  214398.12 kB/sec
	Min throughput per process 			=   53125.08 kB/sec 
	Max throughput per process 			=   53983.92 kB/sec
	Avg throughput per process 			=   53599.53 kB/sec
	Min xfer 					= 8108352.00 kB
	Child[0] xfer count = 8269312.00 kB, Throughput =   53983.92 kB/sec
	Child[1] xfer count = 8388608.00 kB, Throughput =   53616.62 kB/sec
	Child[2] xfer count = 8108352.00 kB, Throughput =   53125.08 kB/sec
	Child[3] xfer count = 8311232.00 kB, Throughput =   53672.50 kB/sec

iozone test complete.