[Gluster-users] Problems to work with mounted directory in Gluster 3.2.7 -> switch to 3.2.4 ; -)

Targino Silveira targinosilveira at gmail.com
Wed Feb 19 19:32:53 UTC 2014


Sure,

I will use XFS, as I sayd before it's for old data, so we don't need a
great performance, we only need to store data.

regards,

Targino Silveira
+55-85-8626-7297
www.twitter.com/targinosilveira


2014-02-19 16:11 GMT-03:00 BGM <bernhard.glomm at ecologic.eu>:

> well, note:
> - you don't need zfs on the hardeware machines, xfs or ext3 or ext4 would
> do it too
> - for production you wouldn't use a glusterfs on top of a glusterfs but
> rather giving the vm access to a real blockdevice, like a whole harddisk or
> at least a partition of it although migration of the vm wouldn't be
> possible than...
> therefor: a VM as a glusterserver might not be the best idea.
> - remember to peer probe the glusterserver partner from both sides! as
> mentioned below
>
> for a first setup you should be fine with that.
>
> regards
>
> On 19.02.2014, at 19:32, Targino Silveira <targinosilveira at gmail.com>
> wrote:
>
> Thanks Bernhard I will do this.
>
> Regards,
>
>
> Targino Silveira
> +55-85-8626-7297
> www.twitter.com/targinosilveira
>
>
> 2014-02-19 14:43 GMT-03:00 Bernhard Glomm <bernhard.glomm at ecologic.eu>:
>
>> I would strongly recommend to restart fresh with gluster 3.2.4 from
>> http://download.gluster.org/pub/gluster/glusterfs/3.4/
>> It works totally fine for me.
>> (reinstall the vms as slim as possible if you can.)
>>
>> As a quick howto consider this:
>>
>> - We have 2 Hardware machines (just desktop machines for dev-env)
>> - both running zol
>> - create a zpool and zfs filesystem
>> - create a gluster replica 2 volume between hostA and hostB
>> - installe 3 VM vmachine0{4,5,6}
>> - vmachine0{4,5} each have a 100GB diskimage file as /dev/vdb which also
>> resides on the glustervolume
>> - create ext3 filesystem on vmachine0{4,5}:/dev/vdb1
>> - create gluster replica 2 between vmachine04 and vmachine05 as shown
>> below
>>
>> (!!!obviously nobody would do that in any serious environment,
>> just to show that even a setup like that _would_ be possible!!!)
>>
>> - run some benchmarks on that volume and compare the results to other
>>
>> So:
>>
>> root at vmachine04[/0]:~ # mkdir -p /srv/vdb1/gf_brick
>> root at vmachine04[/0]:~ # mount /dev/vdb1 /srv/vdb1/
>> root at vmachine04[/0]:~ # gluster peer probe vmachine05
>> peer probe: success
>>
>> # now switch over to vmachine05 and do
>>
>> root at vmachine05[/1]:~ # mkdir -p /srv/vdb1/gf_brick
>> root at vmachine05[/1]:~ # mount /dev/vdb1 /srv/vdb1/
>> root at vmachine05[/1]:~ # gluster peer probe vmachine04
>> peer probe: success
>> root at vmachine05[/1]:~ # gluster peer probe vmachine04
>> peer probe: success: host vmachine04 port 24007 already in peer list
>>
>> # the peer probe from BOTH sides ist often forgotten
>> # switch back to vmachine04 and continue with
>>
>> root at vmachine04[/0]:~ # gluster peer status
>> Number of Peers: 1
>>
>> Hostname: vmachine05
>> Port: 24007
>> Uuid: 085a1489-dabf-40bb-90c1-fbfe66539953
>> State: Peer in Cluster (Connected)
>> root at vmachine04[/0]:~ # gluster volume info layer_cake_volume
>>
>> Volume Name: layer_cake_volume
>> Type: Replicate
>> Volume ID: ef5299db-2896-4631-a2a8-d0082c1b25be
>> Status: Started
>> Number of Bricks: 1 x 2 = 2
>> Transport-type: tcp
>> Bricks:
>> Brick1: vmachine04:/srv/vdb1/gf_brick
>> Brick2: vmachine05:/srv/vdb1/gf_brick
>> root at vmachine04[/0]:~ # gluster volume status layer_cake_volume
>> Status of volume: layer_cake_volume
>> Gluster process                                         Port    Online
>>  Pid
>>
>> ------------------------------------------------------------------------------
>> Brick vmachine04:/srv/vdb1/gf_brick                         49152   Y
>>   12778
>> Brick vmachine05:/srv/vdb1/gf_brick                         49152   Y
>>   16307
>> NFS Server on localhost                                 2049    Y
>> 12790
>> Self-heal Daemon on localhost                           N/A     Y
>> 12791
>> NFS Server on vmachine05                                    2049    Y
>>   16320
>> Self-heal Daemon on vmachine05                              N/A     Y
>>   16319
>>
>> There are no active volume tasks
>>
>> # set any option you might like
>>
>> root at vmachine04[/1]:~ # gluster volume set layer_cake_volume
>> network.remote-dio enable
>> volume set: success
>>
>> # go to vmachine06 and mount the volume
>> root at vmachine06[/1]:~ # mkdir /srv/layer_cake
>> root at vmachine06[/1]:~ # mount -t glusterfs -o
>> backupvolfile-server=vmachine05 vmachine04:/layer_cake_volume
>> /srv/layer_cake
>> root at vmachine06[/1]:~ # mount
>> vmachine04:/layer_cake_volume on /srv/layer_cake type fuse.glusterfs
>> (rw,default_permissions,allow_other,max_read=131072)
>> root at vmachine06[/1]:~ # df -h
>> Filesystem                 Size  Used Avail Use% Mounted on
>> ...
>> vmachine04:/layer_cake_volume   97G  188M   92G   1% /srv/layer_cake
>>
>> All fine and stable
>>
>>
>>
>> # now let's see how it tastes
>> # note this is postmark on  / NOT on the glustermounted layer_cake_volume!
>> # that postmark results might be available tomorrow ;-)))
>> root at vmachine06[/1]:~ # postmark
>> PostMark v1.51 : 8/14/01
>> pm>set transactions 500000
>> pm>set number 200000
>> pm>set subdirectories 10000
>> pm>run
>> Creating subdirectories...Done
>> Creating files...Done
>> Performing transactions..........Done
>> Deleting files...Done
>> Deleting subdirectories...Done
>> Time:
>>         2314 seconds total
>>         2214 seconds of transactions (225 per second)
>> Files:
>>         450096 created (194 per second)
>>                 Creation alone: 200000 files (4166 per second)
>>                 Mixed with transactions: 250096 files (112 per second)
>>         249584 read (112 per second)
>>         250081 appended (112 per second)
>>         450096 deleted (194 per second)
>>                 Deletion alone: 200192 files (3849 per second)
>>                 Mixed with transactions: 249904 files (112 per second)
>>
>> Data:
>>         1456.29 megabytes read (644.44 kilobytes per second)
>>         2715.89 megabytes written (1.17 megabytes per second)
>>
>> # reference
>> # running postmark on the hardware machine directly on zfs
>> #
>> #           /test # postmark
>> #           PostMark v1.51 : 8/14/01
>> #           pm>set transactions 500000
>> #           pm>set number 200000
>> #           pm>set subdirectories 10000
>> #           pm>run
>> #           Creating subdirectories...Done
>> #           Creating files...Done
>> #           Performing transactions..........Done
>> #           Deleting files...Done
>> #           Deleting subdirectories...Done
>> #           Time:
>> #           605 seconds total
>> #           549 seconds of transactions (910 per second)
>> #
>> #           Files:
>> #           450096 created (743 per second)
>> #           Creation alone: 200000 files (4255 per second)
>> #           Mixed with transactions: 250096 files (455 per second)
>> #           249584 read (454 per second)
>> #           250081 appended (455 per second)
>> #           450096 deleted (743 per second)
>> #           Deletion alone: 200192 files (22243 per second)
>> #           Mixed with transactions: 249904 files (455 per second)
>> #
>> #           Data:
>> #           1456.29 megabytes read (2.41 megabytes per second)
>> #           2715.89 megabytes written (4.49 megabytes per second)
>>
>> dbench -D /srv/layer_cake 5
>>
>>  Operation      Count    AvgLat    MaxLat
>>  ----------------------------------------
>>  NTCreateX     195815     5.159   333.296
>>  Close         143870     0.793    93.619
>>  Rename          8310    10.922   123.096
>>  Unlink         39525     2.428   203.753
>>  Qpathinfo     177736     2.551   220.605
>>  Qfileinfo      31030     2.057   175.565
>>  Qfsinfo        32545     1.393   174.045
>>  Sfileinfo      15967     2.691   129.028
>>  Find           68664     9.629   185.739
>>  WriteX         96860     0.841   108.863
>>  ReadX         307834     0.511   213.602
>>  LockX            642     1.511    10.578
>>  UnlockX          642     1.541    10.137
>>  Flush          13712    12.853   405.383
>>
>> Throughput 10.1832 MB/sec  5 clients  5 procs  max_latency=405.405 ms
>>
>>
>> # reference
>> dbench -D /tmp 5
>> # reference
>> dbench -D /tmp 5
>>
>>  Operation      Count    AvgLat    MaxLat
>>  ----------------------------------------
>>  NTCreateX    3817455     0.119   499.847
>>  Close        2804160     0.005    16.000
>>  Rename        161655     0.322   459.790
>>  Unlink        770906     0.556   762.314
>>  Deltree           92    20.647    81.619
>>  Mkdir             46     0.003     0.012
>>  Qpathinfo    3460227     0.017    18.388
>>  Qfileinfo     606258     0.003    11.652
>>  Qfsinfo       634444     0.006    14.976
>>  Sfileinfo     310990     0.155   604.585
>>  Find         1337732     0.056    18.466
>>  WriteX       1902611     0.245   503.604
>>  ReadX        5984135     0.008    16.154
>>  LockX          12430     0.008     9.111
>>  UnlockX        12430     0.004     4.551
>>  Flush         267557     4.505   902.093
>>
>> Throughput 199.664 MB/sec  5 clients  5 procs  max_latency=902.099 ms
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140219/242011de/attachment.html>


More information about the Gluster-users mailing list