<font size=2 face="sans-serif">Andy does the disconnect appear mostly
from a CLI change? I'm trying to investigate the same issue with
"disconnects". Several of my clients report rpc_ping_timer_check
of not responding in 10secs. which is the limit i've set, but I am not
fully understanding what trips that limit?</font><br><br><font size=2 face="sans-serif">Prasanth is the gluster update going
to be made available for gluster 3.4.4 ?</font><br><font size=2 face="sans-serif"><br><br>Khoi Mai<br><br><br></font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">gluster-users-request@gluster.org</font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gluster-users@gluster.org</font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">06/03/2014 06:58 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Gluster-users
Digest, Vol 74, Issue 3</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gluster-users-bounces@gluster.org</font><br><hr noshade><br><br><br><tt><font size=2>Send Gluster-users mailing list submissions to<br>
gluster-users@gluster.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br>
</font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>or, via email, send a message with subject or body 'help' to<br>
gluster-users-request@gluster.org<br><br>You can reach the person managing the list at<br>
gluster-users-owner@gluster.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Gluster-users digest..."<br><br><br>Today's Topics:<br><br> 1. Distributed volumes (yalla.gnan.kumar@accenture.com)<br> 2. Re: Distributed volumes (Michael DePaulo)<br> 3. Re: Distributed volumes (Franco Broi)<br> 4. Re: recommended upgrade procedure from gluster-3.2.7 to<br> gluster-3.5.0 (Todd Pfaff)<br> 5. Re: Unavailability during self-heal for large volumes<br> (Laurent Chouinard)<br> 6. Re: [Gluster-devel] autodelete in snapshots (M S Vishwanath
Bhat)<br> 7. Brick on just one host constantly going offline (Andrew Lau)<br> 8. Re: Unavailability during self-heal for large volumes<br> (Pranith Kumar Karampuri)<br> 9. Re: Brick on just one host constantly going offline<br> (Pranith Kumar Karampuri)<br> 10. Re: Brick on just one host constantly going offline (Andrew
Lau)<br> 11. Re: Brick on just one host constantly going offline<br> (Pranith Kumar Karampuri)<br> 12. Re: Distributed volumes (yalla.gnan.kumar@accenture.com)<br> 13. Re: Distributed volumes (Franco Broi)<br> 14. Re: Distributed volumes (yalla.gnan.kumar@accenture.com)<br> 15. Re: [Gluster-devel] autodelete in snapshots (Kaushal M)<br> 16. Re: Distributed volumes (Franco Broi)<br> 17. Re: Distributed volumes (yalla.gnan.kumar@accenture.com)<br> 18. Re: Distributed volumes (Kaushal M)<br> 19. Re: Distributed volumes (yalla.gnan.kumar@accenture.com)<br> 20. Re: Distributed volumes (Franco Broi)<br> 21. Re: Distributed volumes (Vijay Bellur)<br> 22. NFS ACL Support in Gluster 3.4 (Indivar Nair)<br> 23. Re: NFS ACL Support in Gluster 3.4 (Santosh Pradhan)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Mon, 2 Jun 2014 12:26:09 +0000<br>From: <yalla.gnan.kumar@accenture.com><br>To: <gluster-users@gluster.org><br>Subject: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<67765C71374B974FBFD2AD05AF438EFF0BD564B6@048-CH1MPN3-362.048d.mgd.msft.net><br>
<br>Content-Type: text/plain; charset="us-ascii"<br><br>Hi All,<br><br>I have created a distributed volume of 1 GB , using two bricks from
two different servers.<br>I have written 7 files whose sizes are a total of 1 GB.<br>How can I check that files are distributed on both the bricks ?<br><br><br>Thanks<br>Kumar<br><br>________________________________<br><br>This message is for the designated recipient only and may contain privileged,
proprietary, or otherwise confidential information. If you have received
it in error, please notify the sender immediately and delete the original.
Any other use of the e-mail by you is prohibited. Where allowed by local
law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our
systems for the purposes of information security and assessment of internal
compliance with Accenture policy.<br>______________________________________________________________________________________<br><br></font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140602/fef11c67/attachment-0001.html"><tt><font size=2>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140602/fef11c67/attachment-0001.html</font></tt></a><tt><font size=2>><br><br>------------------------------<br><br>Message: 2<br>Date: Mon, 2 Jun 2014 09:04:44 -0400<br>From: Michael DePaulo <mikedep333@gmail.com><br>To: yalla.gnan.kumar@accenture.com<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<CAMKht8gtijHHZhza3T5zv5hRij9PzwrVvkxtSHDXYRkmGDYpEA@mail.gmail.com><br>Content-Type: text/plain; charset=UTF-8<br><br>On Mon, Jun 2, 2014 at 8:26 AM, <yalla.gnan.kumar@accenture.com>
wrote:<br>> Hi All,<br>><br>><br>><br>> I have created a distributed volume of 1 GB , using two bricks
from two<br>> different servers.<br>><br>> I have written 7 files whose sizes are a total of 1 GB.<br>><br>> How can I check that files are distributed on both the bricks ?<br>><br>><br>><br>><br>><br>> Thanks<br>><br>> Kumar<br><br>Hi Kumar,<br><br>You can use standard file browsing commands like "cd" and "ls"
on both<br>of the bricks. The volume's files will show up as regular files on the<br>underlying filesystem. You can manually verify that files that exist<br>on brick 1 do not exist on brick 2, and vica-versa.<br><br>For example, here's me running file browsing commands on my replicated<br>volume's brick:<br><br>mike@nostromo:/data1/brick1/gv1 :( [7] $ ls -latr<br>total 24<br>drwxr-xr-x. 3 root root 4096 Dec 19 22:21 homes<br>drwxr-xr-x. 3 root root 4096 May 3 17:55 ..<br>drw-------. 261 root root 4096 May 3 18:38 .glusterfs<br>drwxr-xr-x. 4 root root 4096 May 3 21:02 .<br>mike@nostromo:/data1/brick1/gv1 :) [8] $ sudo du -s -h homes/ .glusterfs/<br>[sudo] password for mike:<br>34G homes/<br>252M .glusterfs/<br><br>-Mike<br><br><br>------------------------------<br><br>Message: 3<br>Date: Mon, 02 Jun 2014 21:05:07 +0800<br>From: Franco Broi <franco.broi@iongeo.com><br>To: yalla.gnan.kumar@accenture.com<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID: <1401714307.17051.23.camel@tc1><br>Content-Type: text/plain; charset="UTF-8"<br><br>Just do an ls on the bricks, the paths are the same as the mounted<br>filesystem.<br><br>On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>wrote: <br>> Hi All,<br>> <br>> <br>> <br>> I have created a distributed volume of 1 GB , using two bricks
from<br>> two different servers.<br>> <br>> I have written 7 files whose sizes are a total of 1 GB. <br>> <br>> How can I check that files are distributed on both the bricks ?<br>> <br>> <br>> <br>> <br>> <br>> Thanks<br>> <br>> Kumar<br>> <br>> <br>> <br>> <br>> ______________________________________________________________________<br>> <br>> <br>> This message is for the designated recipient only and may contain<br>> privileged, proprietary, or otherwise confidential information. If
you<br>> have received it in error, please notify the sender immediately and<br>> delete the original. Any other use of the e-mail by you is prohibited.<br>> Where allowed by local law, electronic communications with Accenture<br>> and its affiliates, including e-mail and instant messaging (including<br>> content), may be scanned by our systems for the purposes of<br>> information security and assessment of internal compliance with<br>> Accenture policy. <br>> ______________________________________________________________________________________<br>> <br>> </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> _______________________________________________<br>> Gluster-users mailing list<br>> Gluster-users@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br><br><br><br><br>------------------------------<br><br>Message: 4<br>Date: Mon, 2 Jun 2014 11:56:11 -0400 (EDT)<br>From: Todd Pfaff <pfaff@rhpcs.mcmaster.ca><br>To: Pranith Kumar Karampuri <pkarampu@redhat.com><br>Cc: Susant Palai <spalai@redhat.com>, Venkatesh Somyajulu<br>
<vsomyaju@redhat.com>,
gluster-users@gluster.org<br>Subject: Re: [Gluster-users] recommended upgrade procedure from<br>
gluster-3.2.7 to gluster-3.5.0<br>Message-ID:<br>
<alpine.LMD.2.02.1406021149500.1729@rhpcserv.rhpcs.mcmaster.ca><br>Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed<br><br>On Sun, 1 Jun 2014, Pranith Kumar Karampuri wrote:<br><br>><br>><br>> ----- Original Message -----<br>>> From: "Todd Pfaff" <pfaff@rhpcs.mcmaster.ca><br>>> To: "Pranith Kumar Karampuri" <pkarampu@redhat.com><br>>> Cc: gluster-users@gluster.org<br>>> Sent: Saturday, May 31, 2014 7:18:23 PM<br>>> Subject: Re: [Gluster-users] recommended upgrade procedure from
gluster-3.2.7 to gluster-3.5.0<br>>><br>>> thanks, pranith, that was very helpful!<br>>><br>>> i followed your advice, it ran and completed, and now i'm left
with these<br>>> results on the removed brick (before commit):<br>>><br>>> find /2/scratch/ | wc -l<br>>> 83083<br>>><br>>> find /2/scratch/ -type f | wc -l<br>>> 16<br>>><br>>> find /2/scratch/ -type d | wc -l<br>>> 70243<br>>><br>>> find /2/scratch/ -type l | wc -l<br>>> 12824<br>>><br>>> find /2/scratch/ ! -type d -a ! -type f | wc -l<br>>> 12824<br>>><br>>> find /2/scratch/.glusterfs -type l | wc -l<br>>> 12824<br>>><br>>> find /2/scratch/* | wc -l<br>>> 12873<br>>><br>>> find /2/scratch/* -type d | wc -l<br>>> 12857<br>>><br>>> so it looks like i have 16 files and 12857 directories left in
/2/scratch,<br>>> and 12824 links under /2/scratch/.glusterfs/.<br>>><br>>> my first instinct is to ignore (and remove) the many remaining
directories<br>>> that are empty and only look closer at those that contain the
16 remaining<br>>> files.<br>>><br>>> can i ignore the links under /2/scratch/.glusterfs?<br>>><br>>> as for the 16 files that remain, i can migrate them manually if
necessary<br>>> but i'll first look at all the brick filesystems to see if they
already<br>>> exist elsewhere in some form.<br>>><br>>> do you recommend i do anything else?<br>><br>> Your solutions are good :-). Could you please send us the<br>> configuration, logs of the setup so that we can debug why those files<br>> didn't move? It would be good if we can find the reason for it and
fix<br>> it in the next releases so that this issue is prevented.<br><br><br>sure, i'd be happy to help. what exactly should i send you in terms
of<br>configuration? just my /etc/glusterfs/glusterd.vol? output
of some<br>gluster commands? other?<br><br>in terms of logs, what do you want to see? do you want this file
in its<br>entirety?<br><br> -rw------- 1 root root 145978018 May 31 08:10<br> /var/log/glusterfs/scratch-rebalance.log<br><br>anything else?<br><br><br>><br>> CC developers who work on this feature to look into the issue.<br>><br>> Just curious, did the remove-brick status output say if any failures<br>> happened?<br><br><br>i don't recall seeing anything in the remove-brick status command output<br>that indicated any failures.<br><br>tp<br><br><br>><br>> Pranith<br>><br>>><br>>> thanks,<br>>> tp<br>>><br>>><br>>> On Fri, 30 May 2014, Pranith Kumar Karampuri wrote:<br>>><br>>>><br>>>><br>>>> ----- Original Message -----<br>>>>> From: "Todd Pfaff" <pfaff@rhpcs.mcmaster.ca><br>>>>> To: gluster-users@gluster.org<br>>>>> Sent: Saturday, May 31, 2014 1:58:33 AM<br>>>>> Subject: Re: [Gluster-users] recommended upgrade procedure
from<br>>>>> gluster-3.2.7 to gluster-3.5.0<br>>>>><br>>>>> On Sat, 24 May 2014, Todd Pfaff wrote:<br>>>>><br>>>>>> I have a gluster distributed volume that has been
running nicely with<br>>>>>> gluster-3.2.7 for the past two years and I now want
to upgrade this to<br>>>>>> gluster-3.5.0.<br>>>>>><br>>>>>> What is the recommended procedure for such an upgrade?
Is it necessary<br>>>>>> to<br>>>>>> upgrade from 3.2.7 to 3.3 to 3.4 to 3.5, or can I
safely transition from<br>>>>>> 3.2.7 directly to 3.5.0?<br>>>>><br>>>>> nobody responded so i decided to wing it and hope for
the best.<br>>>>><br>>>>> i also decided to go directly from 3.2.7 to 3.4.3 and
not bother with<br>>>>> 3.5 yet.<br>>>>><br>>>>> the volume is distributed across 13 bricks. formerly
these were in 13<br>>>>> nodes, 1 brick per node, but i recently lost one of these
nodes.<br>>>>> i've moved the brick from the dead node to be a second
brick in one of<br>>>>> the remaining 12 nodes. i currently have this state:<br>>>>><br>>>>> gluster volume status<br>>>>> Status of volume: scratch<br>>>>> Gluster process
Port Online Pid<br>>>>> ------------------------------------------------------------------------------<br>>>>> Brick 172.16.1.1:/1/scratch
49152 Y
6452<br>>>>> Brick 172.16.1.2:/1/scratch
49152 Y
10783<br>>>>> Brick 172.16.1.3:/1/scratch
49152 Y
10164<br>>>>> Brick 172.16.1.4:/1/scratch
49152 Y
10465<br>>>>> Brick 172.16.1.5:/1/scratch
49152 Y
10186<br>>>>> Brick 172.16.1.6:/1/scratch
49152 Y
10388<br>>>>> Brick 172.16.1.7:/1/scratch
49152 Y
10386<br>>>>> Brick 172.16.1.8:/1/scratch
49152 Y
10215<br>>>>> Brick 172.16.1.9:/1/scratch
49152 Y
11059<br>>>>> Brick 172.16.1.10:/1/scratch
49152 Y
9238<br>>>>> Brick 172.16.1.11:/1/scratch
49152 Y
9466<br>>>>> Brick 172.16.1.12:/1/scratch
49152 Y
10777<br>>>>> Brick 172.16.1.1:/2/scratch
49153 Y
6461<br>>>>><br>>>>><br>>>>> what i want to do next is remove Brick 172.16.1.1:/2/scratch
and have<br>>>>> all files it contains redistributed across the other 12
bricks.<br>>>>><br>>>>> what's the correct procedure for this? is it as
simple as:<br>>>>><br>>>>> gluster volume remove-brick scratch 172.16.1.1:/2/scratch
start<br>>>>><br>>>>> and then wait for all files to be moved off that brick?
or do i also<br>>>>> have to do:<br>>>>><br>>>>> gluster volume remove-brick scratch 172.16.1.1:/2/scratch
commit<br>>>>><br>>>>> and then wait for all files to be moved off that brick?
or do i also<br>>>>> have to do something else, such as a rebalance, to cause
the files to<br>>>>> be moved?<br>>>><br>>>> 'gluster volume remove-brick scratch 172.16.1.1:/2/scratch
start' does<br>>>> start the process of migrating all the files to the other
bricks. You need<br>>>> to observe the progress of the process using 'gluster volume
remove-brick<br>>>> scratch 172.16.1.1:/2/scratch status' Once this command
says 'completed'<br>>>> You should execute 'gluster volume remove-brick scratch<br>>>> 172.16.1.1:/2/scratch commit' to completely remove this brick
from the<br>>>> volume. I am a bit paranoid so I would check that no files
are left behind<br>>>> by doing a find on the brick 172.16.1.1:/2/scratch just before
issuing the<br>>>> 'commit' :-).<br>>>><br>>>> Pranith.<br>>>><br>>>>><br>>>>> how do i know when everything has been moved safely to
other bricks and<br>>>>> the then-empty brick is no longer involved in the cluster?<br>>>>><br>>>>> thanks,<br>>>>> tp<br>>>>><br>>>>> _______________________________________________<br>>>>> Gluster-users mailing list<br>>>>> Gluster-users@gluster.org<br>>>>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>>>>><br>>>><br>>>><br>>><br>><br>><br><br><br>------------------------------<br><br>Message: 5<br>Date: Mon, 2 Jun 2014 19:26:40 +0000<br>From: Laurent Chouinard <laurent.chouinard@ubisoft.com><br>To: Pranith Kumar Karampuri <pkarampu@redhat.com><br>Cc: "gluster-users@gluster.org" <gluster-users@gluster.org><br>Subject: Re: [Gluster-users] Unavailability during self-heal for large<br>
volumes<br>Message-ID:<br>
<95ea1865fac2484980d020c6a3b7f0cd@MSR-MAIL-EXCH02.ubisoft.org><br>Content-Type: text/plain; charset="utf-8"<br><br>> Laurent,<br>> This has been improved significantly in afr-v2 (enhanced
version of replication<br>> translator in gluster) which will be released with 3.6 I believe.
The issue happens <br>> because of the directory self-heal in the older versions. In the new
version per file<br>> healing in a directory is performed instead of Full directory heal
at-once which was<br>> creating a lot of traffic. Unfortunately This is too big a change
to backport to older <br>> releases :-(.<br>><br>> Pranith<br><br><br>Hi Pranith,<br><br>Thank you for this information. <br><br>Do you think there is a way to limit/throttle the current directory self-heal
then? I don't mind if it takes a long time.<br><br>Alternatively, is there a way to completely disable the complete healing
system? I would consider running a manual healing operation by STAT'ing
every file, which would allow me to throttle the speed to a more manageable
level.<br><br>Thanks,<br><br>Laurent Chouinard<br><br>------------------------------<br><br>Message: 6<br>Date: Tue, 3 Jun 2014 01:23:48 +0530<br>From: M S Vishwanath Bhat <msvbhat@gmail.com><br>To: Vijay Bellur <vbellur@redhat.com><br>Cc: Seema Naik <senaik@redhat.com>, Gluster Devel<br>
<gluster-devel@gluster.org>,
gluster-users@gluster.org<br>Subject: Re: [Gluster-users] [Gluster-devel] autodelete in snapshots<br>Message-ID:<br>
<CA+H6b3MyWZruhrrykB6Vmv6B1++cgPwb=f+iSqdHnECJQvvmEQ@mail.gmail.com><br>Content-Type: text/plain; charset="utf-8"<br><br>On 3 June 2014 01:02, M S Vishwanath Bhat <msvbhat@gmail.com> wrote:<br><br>><br>><br>><br>> On 2 June 2014 20:22, Vijay Bellur <vbellur@redhat.com> wrote:<br>><br>>> On 04/23/2014 05:50 AM, Vijay Bellur wrote:<br>>><br>>>> On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote:<br>>>><br>>>>> On 04/16/2014 11:39 AM, Avra Sengupta wrote:<br>>>>><br>>>>>> The whole purpose of introducing the soft-limit is,
that at any point<br>>>>>> of time the number of<br>>>>>> snaps should not exceed the hard limit. If we trigger
auto-delete on<br>>>>>> hitting hard-limit, then<br>>>>>> the purpose itself is lost, because at that point
we would be taking a<br>>>>>> snap, making the limit<br>>>>>> hard-limit + 1, and then triggering auto-delete, which
violates the<br>>>>>> sanctity of the hard-limit.<br>>>>>> Also what happens when we are at hard-limit + 1, and
another snap is<br>>>>>> issued, while auto-delete<br>>>>>> is yet to process the first delete. At that point
we end up at<br>>>>>> hard-limit + 1. Also what happens<br>>>>>> if for a particular snap the auto-delete fails.<br>>>>>><br>>>>>> We should see the hard-limit, as something set by
the admin keeping in<br>>>>>> mind the resource consumption<br>>>>>> and at no-point should we cross this limit, come what
may. If we hit<br>>>>>> this limit, the create command<br>>>>>> should fail asking the user to delete snaps using
the "snapshot<br>>>>>> delete" command.<br>>>>>><br>>>>>> The two options Raghavendra mentioned are applicable
for the<br>>>>>> soft-limit only, in which cases on<br>>>>>> hitting the soft-limit<br>>>>>><br>>>>>> 1. Trigger auto-delete<br>>>>>><br>>>>>> or<br>>>>>><br>>>>>> 2. Log a warning-message, for the user saying the
number of snaps is<br>>>>>> exceeding the snap-limit and<br>>>>>> display the number of available snaps<br>>>>>><br>>>>>> Now which of these should happen also depends on the
user, because the<br>>>>>> auto-delete option<br>>>>>> is configurable.<br>>>>>><br>>>>>> So if the auto-delete option is set as true, auto-delete
should be<br>>>>>> triggered and the above message<br>>>>>> should also be logged.<br>>>>>><br>>>>>> But if the option is set as false, only the message
should be logged.<br>>>>>><br>>>>>> This is the behaviour as designed. Adding Rahul, and
Seema in the<br>>>>>> mail, to reflect upon the<br>>>>>> behaviour as well.<br>>>>>><br>>>>>> Regards,<br>>>>>> Avra<br>>>>>><br>>>>><br>>>>> This sounds correct. However we need to make sure that
the usage or<br>>>>> documentation around this should be good enough , so that
users<br>>>>> understand the each of the limits correctly.<br>>>>><br>>>>><br>>>> It might be better to avoid the usage of the term "soft-limit".<br>>>> soft-limit as used in quota and other places generally has
an alerting<br>>>> connotation. Something like "auto-deletion-limit"
might be better.<br>>>><br>>>><br>>> I still see references to "soft-limit" and auto deletion
seems to get<br>>> triggered upon reaching soft-limit.<br>>><br>>> Why is the ability to auto delete not configurable? It does seem
pretty<br>>> nasty to go about deleting snapshots without obtaining explicit
consent<br>>> from the user.<br>>><br>><br>> I agree with Vijay here. It's not good to delete a snap (even though
it is<br>> oldest) without the explicit consent from user.<br>><br>> FYI It took me more than 2 weeks to figure out that my snaps were
getting<br>> autodeleted after reaching "soft-limit". For all I know
I had not done<br>> anything and my snap restore were failing.<br>><br>> I propose to remove the terms "soft" and "hard"
limit. I believe there<br>> should be a limit (just "limit") after which all snapshot
creates should<br>> fail with proper error messages. And there can be a water-mark after
which<br>> user should get warning messages. So below is my proposal.<br>><br>> *auto-delete + snap-limit: *If the snap-limit is set to *n*,
next snap<br>> create (n+1th) will succeed *only if* *if auto-delete is set to on/true/1*<br>> and oldest snap will get deleted automatically. If autodelete is set
to<br>> off/false/0 , (n+1)th snap create will fail with proper error message
from<br>> gluster CLI command. But again by default autodelete should
be off.<br>><br>> *snap-water-mark*: This should come in picture only if autodelete
is<br>> turned off. It should not have any meaning if auto-delete is turned
ON.<br>> Basically it's usage is to give the user warning that limit almost
being<br>> reached and it is time for admin to decide which snaps should be deleted<br>> (or which should be kept)<br>><br>> *my two cents*<br>><br>Adding gluster-users as well.<br><br>-MS<br><br>><br>> -MS<br>><br>><br>>><br>>> Cheers,<br>>><br>>> Vijay<br>>><br>>> _______________________________________________<br>>> Gluster-devel mailing list<br>>> Gluster-devel@gluster.org<br>>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-devel</font></tt></a><tt><font size=2><br>>><br>><br>><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140603/58732995/attachment-0001.html"><tt><font size=2>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140603/58732995/attachment-0001.html</font></tt></a><tt><font size=2>><br><br>------------------------------<br><br>Message: 7<br>Date: Tue, 3 Jun 2014 08:40:25 +1000<br>From: Andrew Lau <andrew@andrewklau.com><br>To: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>Subject: [Gluster-users] Brick on just one host constantly going<br>
offline<br>Message-ID:<br>
<CAD7dF9dCb-f_pNkiu51P7BsPqbeonE+OOuXH84ni4fD_poR0kA@mail.gmail.com><br>Content-Type: text/plain; charset=UTF-8<br><br>Hi,<br><br>Just a short post as I've since nuked the test environment.<br></font></tt><br><tt><font size=2>I've had this case where in a 2 node gluster replica,
the brick of the<br>first host is constantly going offline.<br><br>gluster volume status<br><br>would report host 1's brick is offline. The quorum would kick in,<br>putting the whole cluster into a read only state. This has only<br>recently been happening w/ gluster 3.5 and it normally happens after<br>about 3-4 days of 500GB or so data transfer.<br><br>Has anyone noticed this before? The only way to bring it back was to:<br><br>killall glusterfsd ; killall -9 glusterfsd ; killall glusterd ; glusterd<br><br><br>Thanks,<br>Andrew<br><br><br>------------------------------<br><br>Message: 8<br>Date: Mon, 2 Jun 2014 20:42:36 -0400 (EDT)<br>From: Pranith Kumar Karampuri <pkarampu@redhat.com><br>To: Laurent Chouinard <laurent.chouinard@ubisoft.com><br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Unavailability during self-heal for large<br>
volumes<br>Message-ID:<br>
<1921256894.15836933.1401756156535.JavaMail.zimbra@redhat.com><br>Content-Type: text/plain; charset=utf-8<br><br><br><br>----- Original Message -----<br>> From: "Laurent Chouinard" <laurent.chouinard@ubisoft.com><br>> To: "Pranith Kumar Karampuri" <pkarampu@redhat.com><br>> Cc: gluster-users@gluster.org<br>> Sent: Tuesday, June 3, 2014 12:56:40 AM<br>> Subject: RE: [Gluster-users] Unavailability during self-heal for large
volumes<br>> <br>> > Laurent,<br>> > This has been improved significantly in afr-v2 (enhanced
version of<br>> > replication<br>> > translator in gluster) which will be released with 3.6 I believe.
The issue<br>> > happens<br>> > because of the directory self-heal in the older versions. In
the new<br>> > version per file<br>> > healing in a directory is performed instead of Full directory
heal at-once<br>> > which was<br>> > creating a lot of traffic. Unfortunately This is too big a change
to<br>> > backport to older<br>> > releases :-(.<br>> ><br>> > Pranith<br>> <br>> <br>> Hi Pranith,<br>> <br>> Thank you for this information.<br>> <br>> Do you think there is a way to limit/throttle the current directory
self-heal<br>> then? I don't mind if it takes a long time.<br>> <br>> Alternatively, is there a way to completely disable the complete healing<br>> system? I would consider running a manual healing operation by STAT'ing<br>> every file, which would allow me to throttle the speed to a more manageable<br>> level.<br><br>gluster volume set <volume-name> cluster.self-heal-daemon off would
disable glustershd performing automatic healing.<br><br>Pranith<br>> <br>> Thanks,<br>> <br>> Laurent Chouinard<br>> <br><br><br>------------------------------<br><br>Message: 9<br>Date: Mon, 2 Jun 2014 20:56:14 -0400 (EDT)<br>From: Pranith Kumar Karampuri <pkarampu@redhat.com><br>To: Andrew Lau <andrew@andrewklau.com><br>Cc: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>Subject: Re: [Gluster-users] Brick on just one host constantly going<br>
offline<br>Message-ID:<br>
<1543691713.15838872.1401756974277.JavaMail.zimbra@redhat.com><br>Content-Type: text/plain; charset=utf-8<br><br><br><br>----- Original Message -----<br>> From: "Andrew Lau" <andrew@andrewklau.com><br>> To: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>> Sent: Tuesday, June 3, 2014 4:10:25 AM<br>> Subject: [Gluster-users] Brick on just one host constantly going offline<br>> <br>> Hi,<br>> <br>> Just a short post as I've since nuked the test environment.<br>> <br>> I've had this case where in a 2 node gluster replica, the brick of
the<br>> first host is constantly going offline.<br>> <br>> gluster volume status<br>> <br>> would report host 1's brick is offline. The quorum would kick in,<br>> putting the whole cluster into a read only state. This has only<br>> recently been happening w/ gluster 3.5 and it normally happens after<br>> about 3-4 days of 500GB or so data transfer.<br><br>Could you check mount logs to see if there are ping timer expiry messages
for disconnects?<br>If you see them, then it is very likely that you are hitting throttling
problem fixed by </font></tt><a href=http://review.gluster.org/7531><tt><font size=2>http://review.gluster.org/7531</font></tt></a><tt><font size=2><br><br>Pranith<br><br>> <br>> Has anyone noticed this before? The only way to bring it back was
to:<br>> <br>> killall glusterfsd ; killall -9 glusterfsd ; killall glusterd ; glusterd<br>> <br>> <br>> Thanks,<br>> Andrew<br>> _______________________________________________<br>> Gluster-users mailing list<br>> Gluster-users@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> <br><br><br>------------------------------<br><br>Message: 10<br>Date: Tue, 3 Jun 2014 11:12:44 +1000<br>From: Andrew Lau <andrew@andrewklau.com><br>To: Pranith Kumar Karampuri <pkarampu@redhat.com><br>Cc: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>Subject: Re: [Gluster-users] Brick on just one host constantly going<br>
offline<br>Message-ID:<br>
<CAD7dF9c0005xCo_RKZW9L-cqP_TqGPpNd1B7-2FWRzEXZ_Rvvw@mail.gmail.com><br>Content-Type: text/plain; charset=UTF-8<br><br>Hi Pranith,<br><br>On Tue, Jun 3, 2014 at 10:56 AM, Pranith Kumar Karampuri<br><pkarampu@redhat.com> wrote:<br>><br>><br>> ----- Original Message -----<br>>> From: "Andrew Lau" <andrew@andrewklau.com><br>>> To: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>>> Sent: Tuesday, June 3, 2014 4:10:25 AM<br>>> Subject: [Gluster-users] Brick on just one host constantly going
offline<br>>><br>>> Hi,<br>>><br>>> Just a short post as I've since nuked the test environment.<br>>><br>>> I've had this case where in a 2 node gluster replica, the brick
of the<br>>> first host is constantly going offline.<br>>><br>>> gluster volume status<br>>><br>>> would report host 1's brick is offline. The quorum would kick
in,<br>>> putting the whole cluster into a read only state. This has only<br>>> recently been happening w/ gluster 3.5 and it normally happens
after<br>>> about 3-4 days of 500GB or so data transfer.<br>><br>> Could you check mount logs to see if there are ping timer expiry messages
for disconnects?<br>> If you see them, then it is very likely that you are hitting throttling
problem fixed by </font></tt><a href=http://review.gluster.org/7531><tt><font size=2>http://review.gluster.org/7531</font></tt></a><tt><font size=2><br>><br><br>Ah, that makes sense as it was the only volume which had that ping<br>timeout setting. I also did see the timeout messages in the logs when<br>I was checking. So is this merged in 3.5.1 ?<br><br>> Pranith<br>><br>>><br>>> Has anyone noticed this before? The only way to bring it back
was to:<br>>><br>>> killall glusterfsd ; killall -9 glusterfsd ; killall glusterd
; glusterd<br>>><br>>><br>>> Thanks,<br>>> Andrew<br>>> _______________________________________________<br>>> Gluster-users mailing list<br>>> Gluster-users@gluster.org<br>>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>>><br><br><br>------------------------------<br><br>Message: 11<br>Date: Mon, 2 Jun 2014 22:14:53 -0400 (EDT)<br>From: Pranith Kumar Karampuri <pkarampu@redhat.com><br>To: Andrew Lau <andrew@andrewklau.com><br>Cc: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>Subject: Re: [Gluster-users] Brick on just one host constantly going<br>
offline<br>Message-ID:<br>
<1518143519.15897096.1401761693764.JavaMail.zimbra@redhat.com><br>Content-Type: text/plain; charset=utf-8<br><br><br><br>----- Original Message -----<br>> From: "Andrew Lau" <andrew@andrewklau.com><br>> To: "Pranith Kumar Karampuri" <pkarampu@redhat.com><br>> Cc: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>> Sent: Tuesday, June 3, 2014 6:42:44 AM<br>> Subject: Re: [Gluster-users] Brick on just one host constantly going
offline<br>> <br>> Hi Pranith,<br>> <br>> On Tue, Jun 3, 2014 at 10:56 AM, Pranith Kumar Karampuri<br>> <pkarampu@redhat.com> wrote:<br>> ><br>> ><br>> > ----- Original Message -----<br>> >> From: "Andrew Lau" <andrew@andrewklau.com><br>> >> To: "gluster-users@gluster.org List" <gluster-users@gluster.org><br>> >> Sent: Tuesday, June 3, 2014 4:10:25 AM<br>> >> Subject: [Gluster-users] Brick on just one host constantly
going offline<br>> >><br>> >> Hi,<br>> >><br>> >> Just a short post as I've since nuked the test environment.<br>> >><br>> >> I've had this case where in a 2 node gluster replica, the
brick of the<br>> >> first host is constantly going offline.<br>> >><br>> >> gluster volume status<br>> >><br>> >> would report host 1's brick is offline. The quorum would
kick in,<br>> >> putting the whole cluster into a read only state. This has
only<br>> >> recently been happening w/ gluster 3.5 and it normally happens
after<br>> >> about 3-4 days of 500GB or so data transfer.<br>> ><br>> > Could you check mount logs to see if there are ping timer expiry
messages<br>> > for disconnects?<br>> > If you see them, then it is very likely that you are hitting
throttling<br>> > problem fixed by </font></tt><a href=http://review.gluster.org/7531><tt><font size=2>http://review.gluster.org/7531</font></tt></a><tt><font size=2><br>> ><br>> <br>> Ah, that makes sense as it was the only volume which had that ping<br>> timeout setting. I also did see the timeout messages in the logs when<br>> I was checking. So is this merged in 3.5.1 ?<br><br>Yes! </font></tt><a href=http://review.gluster.org/7570><tt><font size=2>http://review.gluster.org/7570</font></tt></a><tt><font size=2><br><br>Pranith<br>> <br>> > Pranith<br>> ><br>> >><br>> >> Has anyone noticed this before? The only way to bring it
back was to:<br>> >><br>> >> killall glusterfsd ; killall -9 glusterfsd ; killall glusterd
; glusterd<br>> >><br>> >><br>> >> Thanks,<br>> >> Andrew<br>> >> _______________________________________________<br>> >> Gluster-users mailing list<br>> >> Gluster-users@gluster.org<br>> >> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> >><br>> <br><br><br>------------------------------<br><br>Message: 12<br>Date: Tue, 3 Jun 2014 07:21:21 +0000<br>From: <yalla.gnan.kumar@accenture.com><br>To: <franco.broi@iongeo.com><br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<67765C71374B974FBFD2AD05AF438EFF0BD586A4@048-CH1MPN3-362.048d.mgd.msft.net><br>
<br>Content-Type: text/plain; charset="utf-8"<br><br>Hi,<br><br>I have created a distributed volume on my gluster node. I have attached
this volume to a VM on openstack. The size is 1 GB. I have written files
close to 1 GB onto the<br>Volume. But when I do a ls inside the brick directory , the volume
is present only on one gluster server brick. But it is empty on another
server brick. Files are meant to be<br>spread across both the bricks according to distributed volume definition.<br><br>On the VM:<br>--------------<br><br># ls -al<br>total 1013417<br>drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>-rw------- 1 root root 31478251520
Jun 1 21:52 file<br>-rw------- 1 root root 157391257600
Jun 1 21:54 file1<br>-rw------- 1 root root 629565030400
Jun 1 21:55 file2<br>-rw------- 1 root root 708260659200
Jun 1 21:59 file3<br>-rw------- 1 root root 6295650304
Jun 1 22:01 file4<br>-rw------- 1 root root 39333801984
Jun 1 22:01 file5<br>-rw------- 1 root root 78643200000
Jun 1 22:04 file6<br>drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>----------<br># du -sch *<br>20.0M file<br>100.0M file1<br>400.0M file2<br>454.0M file3<br>4.0M file4<br>11.6M file5<br>0 file6<br>16.0K lost+found<br>989.7M total<br>------------------------<br><br><br>On the gluster server nodes:<br>-----------------------<br>root@primary:/export/sdd1/brick# ll<br>total 12<br>drwxr-xr-x 2 root root 4096 Jun 2 04:08 ./<br>drwxr-xr-x 4 root root 4096 May 27 08:42 ../<br>root@primary:/export/sdd1/brick#<br>--------------------------<br><br>root@secondary:/export/sdd1/brick# ll<br>total 1046536<br>drwxr-xr-x 2 root root 4096 Jun 2 08:51 ./<br>drwxr-xr-x 4 root root 4096 May 27 08:43 ../<br>-rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35 volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>root@secondary:/export/sdd1/brick#<br>---------------------------------<br><br><br>Thanks<br>Kumar<br><br><br><br><br><br><br><br><br>-----Original Message-----<br>From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>Sent: Monday, June 02, 2014 6:35 PM<br>To: Gnan Kumar, Yalla<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br><br>Just do an ls on the bricks, the paths are the same as the mounted filesystem.<br><br>On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>wrote:<br>> Hi All,<br>><br>><br>><br>> I have created a distributed volume of 1 GB , using two bricks
from<br>> two different servers.<br>><br>> I have written 7 files whose sizes are a total of 1 GB.<br>><br>> How can I check that files are distributed on both the bricks ?<br>><br>><br>><br>><br>><br>> Thanks<br>><br>> Kumar<br>><br>><br>><br>><br>> ______________________________________________________________________<br>><br>><br>> This message is for the designated recipient only and may contain<br>> privileged, proprietary, or otherwise confidential information. If
you<br>> have received it in error, please notify the sender immediately and<br>> delete the original. Any other use of the e-mail by you is prohibited.<br>> Where allowed by local law, electronic communications with Accenture<br>> and its affiliates, including e-mail and instant messaging (including<br>> content), may be scanned by our systems for the purposes of<br>> information security and assessment of internal compliance with<br>> Accenture policy.<br>> ______________________________________________________________________<br>> ________________<br>><br>> </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> _______________________________________________<br>> Gluster-users mailing list<br>> Gluster-users@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br><br><br><br><br>________________________________<br><br>This message is for the designated recipient only and may contain privileged,
proprietary, or otherwise confidential information. If you have received
it in error, please notify the sender immediately and delete the original.
Any other use of the e-mail by you is prohibited. Where allowed by local
law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our
systems for the purposes of information security and assessment of internal
compliance with Accenture policy.<br>______________________________________________________________________________________<br><br></font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br><br>------------------------------<br><br>Message: 13<br>Date: Tue, 03 Jun 2014 15:25:46 +0800<br>From: Franco Broi <franco.broi@iongeo.com><br>To: yalla.gnan.kumar@accenture.com<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID: <1401780346.2236.299.camel@tc1><br>Content-Type: text/plain; charset="UTF-8"<br><br><br>What do gluster vol info and gluster vol status give you?<br><br>On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>wrote: <br>> Hi,<br>> <br>> I have created a distributed volume on my gluster node. I have
attached this volume to a VM on openstack. The size is 1 GB. I have written
files close to 1 GB onto the<br>> Volume. But when I do a ls inside the brick directory , the
volume is present only on one gluster server brick. But it is empty on
another server brick. Files are meant to be<br>> spread across both the bricks according to distributed volume definition.<br>> <br>> On the VM:<br>> --------------<br>> <br>> # ls -al<br>> total 1013417<br>> drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>> drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>> -rw------- 1 root root 31478251520
Jun 1 21:52 file<br>> -rw------- 1 root root 157391257600
Jun 1 21:54 file1<br>> -rw------- 1 root root 629565030400
Jun 1 21:55 file2<br>> -rw------- 1 root root 708260659200
Jun 1 21:59 file3<br>> -rw------- 1 root root 6295650304
Jun 1 22:01 file4<br>> -rw------- 1 root root 39333801984
Jun 1 22:01 file5<br>> -rw------- 1 root root 78643200000
Jun 1 22:04 file6<br>> drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>> ----------<br>> # du -sch *<br>> 20.0M file<br>> 100.0M file1<br>> 400.0M file2<br>> 454.0M file3<br>> 4.0M file4<br>> 11.6M file5<br>> 0 file6<br>> 16.0K lost+found<br>> 989.7M total<br>> ------------------------<br>> <br>> <br>> On the gluster server nodes:<br>> -----------------------<br>> root@primary:/export/sdd1/brick# ll<br>> total 12<br>> drwxr-xr-x 2 root root 4096 Jun 2 04:08 ./<br>> drwxr-xr-x 4 root root 4096 May 27 08:42 ../<br>> root@primary:/export/sdd1/brick#<br>> --------------------------<br>> <br>> root@secondary:/export/sdd1/brick# ll<br>> total 1046536<br>> drwxr-xr-x 2 root root 4096 Jun 2 08:51
./<br>> drwxr-xr-x 4 root root 4096 May 27 08:43 ../<br>> -rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35 volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>> root@secondary:/export/sdd1/brick#<br>> ---------------------------------<br>> <br>> <br>> Thanks<br>> Kumar<br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> -----Original Message-----<br>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> Sent: Monday, June 02, 2014 6:35 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>> <br>> Just do an ls on the bricks, the paths are the same as the mounted
filesystem.<br>> <br>> On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>> wrote:<br>> > Hi All,<br>> ><br>> ><br>> ><br>> > I have created a distributed volume of 1 GB , using two
bricks from<br>> > two different servers.<br>> ><br>> > I have written 7 files whose sizes are a total of 1 GB.<br>> ><br>> > How can I check that files are distributed on both the bricks
?<br>> ><br>> ><br>> ><br>> ><br>> ><br>> > Thanks<br>> ><br>> > Kumar<br>> ><br>> ><br>> ><br>> ><br>> > ______________________________________________________________________<br>> ><br>> ><br>> > This message is for the designated recipient only and may contain<br>> > privileged, proprietary, or otherwise confidential information.
If you<br>> > have received it in error, please notify the sender immediately
and<br>> > delete the original. Any other use of the e-mail by you is prohibited.<br>> > Where allowed by local law, electronic communications with Accenture<br>> > and its affiliates, including e-mail and instant messaging (including<br>> > content), may be scanned by our systems for the purposes of<br>> > information security and assessment of internal compliance with<br>> > Accenture policy.<br>> > ______________________________________________________________________<br>> > ________________<br>> ><br>> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> > _______________________________________________<br>> > Gluster-users mailing list<br>> > Gluster-users@gluster.org<br>> > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> <br>> <br>> <br>> <br>> ________________________________<br>> <br>> This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise confidential information. If you
have received it in error, please notify the sender immediately and delete
the original. Any other use of the e-mail by you is prohibited. Where allowed
by local law, electronic communications with Accenture and its affiliates,
including e-mail and instant messaging (including content), may be scanned
by our systems for the purposes of information security and assessment
of internal compliance with Accenture policy.<br>> ______________________________________________________________________________________<br>> <br>> </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br><br><br><br><br>------------------------------<br><br>Message: 14<br>Date: Tue, 3 Jun 2014 07:29:24 +0000<br>From: <yalla.gnan.kumar@accenture.com><br>To: <franco.broi@iongeo.com><br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<67765C71374B974FBFD2AD05AF438EFF0BD586B9@048-CH1MPN3-362.048d.mgd.msft.net><br>
<br>Content-Type: text/plain; charset="utf-8"<br><br>root@secondary:/export/sdd1/brick# gluster volume info<br><br>Volume Name: dst<br>Type: Distribute<br>Status: Started<br>Number of Bricks: 2<br>Transport-type: tcp<br>Bricks:<br>Brick1: primary:/export/sdd1/brick<br>Brick2: secondary:/export/sdd1/brick<br><br><br><br>-----Original Message-----<br>From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]
<br>Sent: Tuesday, June 03, 2014 12:56 PM<br>To: Gnan Kumar, Yalla<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br><br><br>What do gluster vol info and gluster vol status give you?<br></font></tt><br><tt><font size=2>On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>wrote: <br>> Hi,<br>> <br>> I have created a distributed volume on my gluster node. I have
attached this volume to a VM on openstack. The size is 1 GB. I have written
files close to 1 GB onto the<br>> Volume. But when I do a ls inside the brick directory , the
volume is present only on one gluster server brick. But it is empty on
another server brick. Files are meant to be<br>> spread across both the bricks according to distributed volume definition.<br>> <br>> On the VM:<br>> --------------<br>> <br>> # ls -al<br>> total 1013417<br>> drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>> drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>> -rw------- 1 root root 31478251520
Jun 1 21:52 file<br>> -rw------- 1 root root 157391257600
Jun 1 21:54 file1<br>> -rw------- 1 root root 629565030400
Jun 1 21:55 file2<br>> -rw------- 1 root root 708260659200
Jun 1 21:59 file3<br>> -rw------- 1 root root 6295650304
Jun 1 22:01 file4<br>> -rw------- 1 root root 39333801984
Jun 1 22:01 file5<br>> -rw------- 1 root root 78643200000
Jun 1 22:04 file6<br>> drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>> ----------<br>> # du -sch *<br>> 20.0M file<br>> 100.0M file1<br>> 400.0M file2<br>> 454.0M file3<br>> 4.0M file4<br>> 11.6M file5<br>> 0 file6<br>> 16.0K lost+found<br>> 989.7M total<br>> ------------------------<br>> <br>> <br>> On the gluster server nodes:<br>> -----------------------<br>> root@primary:/export/sdd1/brick# ll<br>> total 12<br>> drwxr-xr-x 2 root root 4096 Jun 2 04:08 ./ drwxr-xr-x 4 root
root <br>> 4096 May 27 08:42 ../ root@primary:/export/sdd1/brick#<br>> --------------------------<br>> <br>> root@secondary:/export/sdd1/brick# ll<br>> total 1046536<br>> drwxr-xr-x 2 root root 4096 Jun 2 08:51
./<br>> drwxr-xr-x 4 root root 4096 May 27 08:43 ../<br>> -rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35 <br>> volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>> root@secondary:/export/sdd1/brick#<br>> ---------------------------------<br>> <br>> <br>> Thanks<br>> Kumar<br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> -----Original Message-----<br>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> Sent: Monday, June 02, 2014 6:35 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>> <br>> Just do an ls on the bricks, the paths are the same as the mounted
filesystem.<br>> <br>> On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>> wrote:<br>> > Hi All,<br>> ><br>> ><br>> ><br>> > I have created a distributed volume of 1 GB , using two
bricks from <br>> > two different servers.<br>> ><br>> > I have written 7 files whose sizes are a total of 1 GB.<br>> ><br>> > How can I check that files are distributed on both the bricks
?<br>> ><br>> ><br>> ><br>> ><br>> ><br>> > Thanks<br>> ><br>> > Kumar<br>> ><br>> ><br>> ><br>> ><br>> > ____________________________________________________________________<br>> > __<br>> ><br>> ><br>> > This message is for the designated recipient only and may contain
<br>> > privileged, proprietary, or otherwise confidential information.
If <br>> > you have received it in error, please notify the sender immediately
<br>> > and delete the original. Any other use of the e-mail by you is
prohibited.<br>> > Where allowed by local law, electronic communications with Accenture
<br>> > and its affiliates, including e-mail and instant messaging <br>> > (including content), may be scanned by our systems for the purposes
<br>> > of information security and assessment of internal compliance
with <br>> > Accenture policy.<br>> > ____________________________________________________________________<br>> > __<br>> > ________________<br>> ><br>> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> > _______________________________________________<br>> > Gluster-users mailing list<br>> > Gluster-users@gluster.org<br>> > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> <br>> <br>> <br>> <br>> ________________________________<br>> <br>> This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise confidential information. If you
have received it in error, please notify the sender immediately and delete
the original. Any other use of the e-mail by you is prohibited. Where allowed
by local law, electronic communications with Accenture and its affiliates,
including e-mail and instant messaging (including content), may be scanned
by our systems for the purposes of information security and assessment
of internal compliance with Accenture policy.<br>> ______________________________________________________________________<br>> ________________<br>> <br>> </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br><br><br><br><br>------------------------------<br><br>Message: 15<br>Date: Tue, 3 Jun 2014 13:02:54 +0530<br>From: Kaushal M <kshlmster@gmail.com><br>To: Gluster Devel <gluster-devel@gluster.org><br>Cc: "gluster-users@gluster.org" <gluster-users@gluster.org><br>Subject: Re: [Gluster-users] [Gluster-devel] autodelete in snapshots<br>Message-ID:<br>
<CAOujamXNzzsNZW0jX9gNPzP7JzUnM1gGDCnNiu7R=ygWTc4oFQ@mail.gmail.com><br>Content-Type: text/plain; charset=UTF-8<br><br>I agree as well. We shouldn't be deleting any data without the<br>explicit consent of the user.<br><br>The approach proposed by MS is better than the earlier approach.<br><br>~kaushal<br><br>On Tue, Jun 3, 2014 at 1:02 AM, M S Vishwanath Bhat <msvbhat@gmail.com>
wrote:<br>><br>><br>><br>> On 2 June 2014 20:22, Vijay Bellur <vbellur@redhat.com> wrote:<br>>><br>>> On 04/23/2014 05:50 AM, Vijay Bellur wrote:<br>>>><br>>>> On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote:<br>>>>><br>>>>> On 04/16/2014 11:39 AM, Avra Sengupta wrote:<br>>>>>><br>>>>>> The whole purpose of introducing the soft-limit is,
that at any point<br>>>>>> of time the number of<br>>>>>> snaps should not exceed the hard limit. If we trigger
auto-delete on<br>>>>>> hitting hard-limit, then<br>>>>>> the purpose itself is lost, because at that point
we would be taking a<br>>>>>> snap, making the limit<br>>>>>> hard-limit + 1, and then triggering auto-delete, which
violates the<br>>>>>> sanctity of the hard-limit.<br>>>>>> Also what happens when we are at hard-limit + 1, and
another snap is<br>>>>>> issued, while auto-delete<br>>>>>> is yet to process the first delete. At that point
we end up at<br>>>>>> hard-limit + 1. Also what happens<br>>>>>> if for a particular snap the auto-delete fails.<br>>>>>><br>>>>>> We should see the hard-limit, as something set by
the admin keeping in<br>>>>>> mind the resource consumption<br>>>>>> and at no-point should we cross this limit, come what
may. If we hit<br>>>>>> this limit, the create command<br>>>>>> should fail asking the user to delete snaps using
the "snapshot<br>>>>>> delete" command.<br>>>>>><br>>>>>> The two options Raghavendra mentioned are applicable
for the<br>>>>>> soft-limit only, in which cases on<br>>>>>> hitting the soft-limit<br>>>>>><br>>>>>> 1. Trigger auto-delete<br>>>>>><br>>>>>> or<br>>>>>><br>>>>>> 2. Log a warning-message, for the user saying the
number of snaps is<br>>>>>> exceeding the snap-limit and<br>>>>>> display the number of available snaps<br>>>>>><br>>>>>> Now which of these should happen also depends on the
user, because the<br>>>>>> auto-delete option<br>>>>>> is configurable.<br>>>>>><br>>>>>> So if the auto-delete option is set as true, auto-delete
should be<br>>>>>> triggered and the above message<br>>>>>> should also be logged.<br>>>>>><br>>>>>> But if the option is set as false, only the message
should be logged.<br>>>>>><br>>>>>> This is the behaviour as designed. Adding Rahul, and
Seema in the<br>>>>>> mail, to reflect upon the<br>>>>>> behaviour as well.<br>>>>>><br>>>>>> Regards,<br>>>>>> Avra<br>>>>><br>>>>><br>>>>> This sounds correct. However we need to make sure that
the usage or<br>>>>> documentation around this should be good enough , so that
users<br>>>>> understand the each of the limits correctly.<br>>>>><br>>>><br>>>> It might be better to avoid the usage of the term "soft-limit".<br>>>> soft-limit as used in quota and other places generally has
an alerting<br>>>> connotation. Something like "auto-deletion-limit"
might be better.<br>>>><br>>><br>>> I still see references to "soft-limit" and auto deletion
seems to get<br>>> triggered upon reaching soft-limit.<br>>><br>>> Why is the ability to auto delete not configurable? It does seem
pretty<br>>> nasty to go about deleting snapshots without obtaining explicit
consent from<br>>> the user.<br>><br>><br>> I agree with Vijay here. It's not good to delete a snap (even though
it is<br>> oldest) without the explicit consent from user.<br>><br>> FYI It took me more than 2 weeks to figure out that my snaps were
getting<br>> autodeleted after reaching "soft-limit". For all I know
I had not done<br>> anything and my snap restore were failing.<br>><br>> I propose to remove the terms "soft" and "hard"
limit. I believe there<br>> should be a limit (just "limit") after which all snapshot
creates should<br>> fail with proper error messages. And there can be a water-mark after
which<br>> user should get warning messages. So below is my proposal.<br>><br>> auto-delete + snap-limit: If the snap-limit is set to n, next
snap create<br>> (n+1th) will succeed only if if auto-delete is set to on/true/1 and
oldest<br>> snap will get deleted automatically. If autodelete is set to off/false/0
,<br>> (n+1)th snap create will fail with proper error message from gluster
CLI<br>> command. But again by default autodelete should be off.<br>><br>> snap-water-mark: This should come in picture only if autodelete is
turned<br>> off. It should not have any meaning if auto-delete is turned ON. Basically<br>> it's usage is to give the user warning that limit almost being reached
and<br>> it is time for admin to decide which snaps should be deleted (or which<br>> should be kept)<br>><br>> *my two cents*<br>><br>> -MS<br>><br>>><br>>><br>>> Cheers,<br>>><br>>> Vijay<br>>><br>>> _______________________________________________<br>>> Gluster-devel mailing list<br>>> Gluster-devel@gluster.org<br>>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-devel</font></tt></a><tt><font size=2><br>><br>><br>><br>> _______________________________________________<br>> Gluster-devel mailing list<br>> Gluster-devel@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-devel"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-devel</font></tt></a><tt><font size=2><br>><br><br><br>------------------------------<br><br>Message: 16<br>Date: Tue, 03 Jun 2014 15:34:01 +0800<br>From: Franco Broi <franco.broi@iongeo.com><br>To: yalla.gnan.kumar@accenture.com<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID: <1401780841.2236.304.camel@tc1><br>Content-Type: text/plain; charset="UTF-8"<br><br><br>Ok, what you have is a single large file (must be filesystem image??).<br>Gluster will not stripe files, it writes different whole files to<br>different bricks. <br><br>On Tue, 2014-06-03 at 07:29 +0000, yalla.gnan.kumar@accenture.com<br>wrote: <br>> root@secondary:/export/sdd1/brick# gluster volume info<br>> <br>> Volume Name: dst<br>> Type: Distribute<br>> Status: Started<br>> Number of Bricks: 2<br>> Transport-type: tcp<br>> Bricks:<br>> Brick1: primary:/export/sdd1/brick<br>> Brick2: secondary:/export/sdd1/brick<br>> <br>> <br>> <br>> -----Original Message-----<br>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]
<br>> Sent: Tuesday, June 03, 2014 12:56 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>> <br>> <br>> What do gluster vol info and gluster vol status give you?<br>> <br>> On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>> wrote: <br>> > Hi,<br>> > <br>> > I have created a distributed volume on my gluster node.
I have attached this volume to a VM on openstack. The size is 1 GB. I have
written files close to 1 GB onto the<br>> > Volume. But when I do a ls inside the brick directory
, the volume is present only on one gluster server brick. But it is empty
on another server brick. Files are meant to be<br>> > spread across both the bricks according to distributed volume
definition.<br>> > <br>> > On the VM:<br>> > --------------<br>> > <br>> > # ls -al<br>> > total 1013417<br>> > drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>> > drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>> > -rw------- 1 root root
31478251520 Jun 1 21:52 file<br>> > -rw------- 1 root root
157391257600 Jun 1 21:54 file1<br>> > -rw------- 1 root root
629565030400 Jun 1 21:55 file2<br>> > -rw------- 1 root root
708260659200 Jun 1 21:59 file3<br>> > -rw------- 1 root root
6295650304 Jun 1 22:01 file4<br>> > -rw------- 1 root root
39333801984 Jun 1 22:01 file5<br>> > -rw------- 1 root root
78643200000 Jun 1 22:04 file6<br>> > drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>> > ----------<br>> > # du -sch *<br>> > 20.0M file<br>> > 100.0M file1<br>> > 400.0M file2<br>> > 454.0M file3<br>> > 4.0M file4<br>> > 11.6M file5<br>> > 0 file6<br>> > 16.0K lost+found<br>> > 989.7M total<br>> > ------------------------<br>> > <br>> > <br>> > On the gluster server nodes:<br>> > -----------------------<br>> > root@primary:/export/sdd1/brick# ll<br>> > total 12<br>> > drwxr-xr-x 2 root root 4096 Jun 2 04:08 ./ drwxr-xr-x 4
root root <br>> > 4096 May 27 08:42 ../ root@primary:/export/sdd1/brick#<br>> > --------------------------<br>> > <br>> > root@secondary:/export/sdd1/brick# ll<br>> > total 1046536<br>> > drwxr-xr-x 2 root root 4096 Jun 2
08:51 ./<br>> > drwxr-xr-x 4 root root 4096 May 27 08:43
../<br>> > -rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35
<br>> > volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>> > root@secondary:/export/sdd1/brick#<br>> > ---------------------------------<br>> > <br>> > <br>> > Thanks<br>> > Kumar<br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > -----Original Message-----<br>> > From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> > Sent: Monday, June 02, 2014 6:35 PM<br>> > To: Gnan Kumar, Yalla<br>> > Cc: gluster-users@gluster.org<br>> > Subject: Re: [Gluster-users] Distributed volumes<br>> > <br>> > Just do an ls on the bricks, the paths are the same as the mounted
filesystem.<br>> > <br>> > On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>> > wrote:<br>> > > Hi All,<br>> > ><br>> > ><br>> > ><br>> > > I have created a distributed volume of 1 GB , using
two bricks from <br>> > > two different servers.<br>> > ><br>> > > I have written 7 files whose sizes are a total of 1
GB.<br>> > ><br>> > > How can I check that files are distributed on both the bricks
?<br>> > ><br>> > ><br>> > ><br>> > ><br>> > ><br>> > > Thanks<br>> > ><br>> > > Kumar<br>> > ><br>> > ><br>> > ><br>> > ><br>> > > ____________________________________________________________________<br>> > > __<br>> > ><br>> > ><br>> > > This message is for the designated recipient only and may
contain <br>> > > privileged, proprietary, or otherwise confidential information.
If <br>> > > you have received it in error, please notify the sender
immediately <br>> > > and delete the original. Any other use of the e-mail by
you is prohibited.<br>> > > Where allowed by local law, electronic communications with
Accenture <br>> > > and its affiliates, including e-mail and instant messaging
<br>> > > (including content), may be scanned by our systems for the
purposes <br>> > > of information security and assessment of internal compliance
with <br>> > > Accenture policy.<br>> > > ____________________________________________________________________<br>> > > __<br>> > > ________________<br>> > ><br>> > > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> > > _______________________________________________<br>> > > Gluster-users mailing list<br>> > > Gluster-users@gluster.org<br>> > > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> > <br>> > <br>> > <br>> > <br>> > ________________________________<br>> > <br>> > This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise confidential information. If you
have received it in error, please notify the sender immediately and delete
the original. Any other use of the e-mail by you is prohibited. Where allowed
by local law, electronic communications with Accenture and its affiliates,
including e-mail and instant messaging (including content), may be scanned
by our systems for the purposes of information security and assessment
of internal compliance with Accenture policy.<br>> > ______________________________________________________________________<br>> > ________________<br>> > <br>> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> <br>> <br>> <br><br><br><br><br>------------------------------<br><br>Message: 17<br>Date: Tue, 3 Jun 2014 07:39:58 +0000<br>From: <yalla.gnan.kumar@accenture.com><br>To: <franco.broi@iongeo.com><br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<67765C71374B974FBFD2AD05AF438EFF0BD586DF@048-CH1MPN3-362.048d.mgd.msft.net><br>
<br>Content-Type: text/plain; charset="utf-8"<br><br>I have created distributed volume, created a 1 GB volume on it, and
attached it to the VM and created a filesystem on it. How to verify
that the files in the vm are<br>distributed across both the bricks on two servers ?<br><br><br><br>-----Original Message-----<br>From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]
<br>Sent: Tuesday, June 03, 2014 1:04 PM<br>To: Gnan Kumar, Yalla<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br><br><br>Ok, what you have is a single large file (must be filesystem image??).<br>Gluster will not stripe files, it writes different whole files to different
bricks. <br><br>On Tue, 2014-06-03 at 07:29 +0000, yalla.gnan.kumar@accenture.com<br>wrote: <br>> root@secondary:/export/sdd1/brick# gluster volume info<br>> <br>> Volume Name: dst<br>> Type: Distribute<br>> Status: Started<br>> Number of Bricks: 2<br>> Transport-type: tcp<br>> Bricks:<br>> Brick1: primary:/export/sdd1/brick<br>> Brick2: secondary:/export/sdd1/brick<br>> <br>> <br>> <br>> -----Original Message-----<br>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> Sent: Tuesday, June 03, 2014 12:56 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>> <br>> <br>> What do gluster vol info and gluster vol status give you?<br>> <br>> On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>> wrote: <br>> > Hi,<br>> > <br>> > I have created a distributed volume on my gluster node.
I have attached this volume to a VM on openstack. The size is 1 GB. I have
written files close to 1 GB onto the<br>> > Volume. But when I do a ls inside the brick directory
, the volume is present only on one gluster server brick. But it is empty
on another server brick. Files are meant to be<br>> > spread across both the bricks according to distributed volume
definition.<br>> > <br>> > On the VM:<br>> > --------------<br>> > <br>> > # ls -al<br>> > total 1013417<br>> > drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>> > drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>> > -rw------- 1 root root
31478251520 Jun 1 21:52 file<br>> > -rw------- 1 root root
157391257600 Jun 1 21:54 file1<br>> > -rw------- 1 root root
629565030400 Jun 1 21:55 file2<br>> > -rw------- 1 root root
708260659200 Jun 1 21:59 file3<br>> > -rw------- 1 root root
6295650304 Jun 1 22:01 file4<br>> > -rw------- 1 root root
39333801984 Jun 1 22:01 file5<br>> > -rw------- 1 root root
78643200000 Jun 1 22:04 file6<br>> > drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>> > ----------<br>> > # du -sch *<br>> > 20.0M file<br>> > 100.0M file1<br>> > 400.0M file2<br>> > 454.0M file3<br>> > 4.0M file4<br>> > 11.6M file5<br>> > 0 file6<br>> > 16.0K lost+found<br>> > 989.7M total<br>> > ------------------------<br>> > <br>> > <br>> > On the gluster server nodes:<br>> > -----------------------<br>> > root@primary:/export/sdd1/brick# ll<br>> > total 12<br>> > drwxr-xr-x 2 root root 4096 Jun 2 04:08 ./ drwxr-xr-x 4
root root<br>> > 4096 May 27 08:42 ../ root@primary:/export/sdd1/brick#<br>> > --------------------------<br>> > <br>> > root@secondary:/export/sdd1/brick# ll total 1046536<br>> > drwxr-xr-x 2 root root 4096 Jun 2
08:51 ./<br>> > drwxr-xr-x 4 root root 4096 May 27 08:43
../<br>> > -rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35<br>> > volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>> > root@secondary:/export/sdd1/brick#<br>> > ---------------------------------<br>> > <br>> > <br>> > Thanks<br>> > Kumar<br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > -----Original Message-----<br>> > From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> > Sent: Monday, June 02, 2014 6:35 PM<br>> > To: Gnan Kumar, Yalla<br>> > Cc: gluster-users@gluster.org<br>> > Subject: Re: [Gluster-users] Distributed volumes<br>> > <br>> > Just do an ls on the bricks, the paths are the same as the mounted
filesystem.<br>> > <br>> > On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>> > wrote:<br>> > > Hi All,<br>> > ><br>> > ><br>> > ><br>> > > I have created a distributed volume of 1 GB , using
two bricks from <br>> > > two different servers.<br>> > ><br>> > > I have written 7 files whose sizes are a total of 1
GB.<br>> > ></font></tt><br><tt><font size=2>> > > How can I check that files are distributed
on both the bricks ?<br>> > ><br>> > ><br>> > ><br>> > ><br>> > ><br>> > > Thanks<br>> > ><br>> > > Kumar<br>> > ><br>> > ><br>> > ><br>> > ><br>> > > ____________________________________________________________________<br>> > > __<br>> > ><br>> > ><br>> > > This message is for the designated recipient only and may
contain <br>> > > privileged, proprietary, or otherwise confidential information.
If <br>> > > you have received it in error, please notify the sender
immediately <br>> > > and delete the original. Any other use of the e-mail by
you is prohibited.<br>> > > Where allowed by local law, electronic communications with
Accenture <br>> > > and its affiliates, including e-mail and instant messaging
<br>> > > (including content), may be scanned by our systems for the
purposes <br>> > > of information security and assessment of internal compliance
with <br>> > > Accenture policy.<br>> > > ____________________________________________________________________<br>> > > __<br>> > > ________________<br>> > ><br>> > > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> > > _______________________________________________<br>> > > Gluster-users mailing list<br>> > > Gluster-users@gluster.org<br>> > > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> > <br>> > <br>> > <br>> > <br>> > ________________________________<br>> > <br>> > This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise confidential information. If you
have received it in error, please notify the sender immediately and delete
the original. Any other use of the e-mail by you is prohibited. Where allowed
by local law, electronic communications with Accenture and its affiliates,
including e-mail and instant messaging (including content), may be scanned
by our systems for the purposes of information security and assessment
of internal compliance with Accenture policy.<br>> > ______________________________________________________________________<br>> > ________________<br>> > <br>> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> <br>> <br>> <br><br><br><br><br>------------------------------<br><br>Message: 18<br>Date: Tue, 3 Jun 2014 13:19:08 +0530<br>From: Kaushal M <kshlmster@gmail.com><br>To: yalla.gnan.kumar@accenture.com<br>Cc: "gluster-users@gluster.org" <gluster-users@gluster.org><br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<CAOujamXxBoF3QAbYgyAOvGY7fimNgkGBv8Z-gotpchh0xgEePA@mail.gmail.com><br>Content-Type: text/plain; charset=UTF-8<br><br>You have only 1 file on the gluster volume, the 1GB disk image/volume<br>that you created. This disk image is attached to the VM as a file<br>system, not the gluster volume. So whatever you do in the VM's file<br>system, affects just the 1 disk image. The files, directories etc. you<br>created, are inside the disk image. So you still have just one file on<br>the gluster volume, not many as you are assuming.<br><br><br><br>On Tue, Jun 3, 2014 at 1:09 PM, <yalla.gnan.kumar@accenture.com>
wrote:<br>> I have created distributed volume, created a 1 GB volume on
it, and attached it to the VM and created a filesystem on it. How
to verify that the files in the vm are<br>> distributed across both the bricks on two servers ?<br>><br>><br>><br>> -----Original Message-----<br>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> Sent: Tuesday, June 03, 2014 1:04 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>><br>><br>> Ok, what you have is a single large file (must be filesystem image??).<br>> Gluster will not stripe files, it writes different whole files to
different bricks.<br>><br>> On Tue, 2014-06-03 at 07:29 +0000, yalla.gnan.kumar@accenture.com<br>> wrote:<br>>> root@secondary:/export/sdd1/brick# gluster volume info<br>>><br>>> Volume Name: dst<br>>> Type: Distribute<br>>> Status: Started<br>>> Number of Bricks: 2<br>>> Transport-type: tcp<br>>> Bricks:<br>>> Brick1: primary:/export/sdd1/brick<br>>> Brick2: secondary:/export/sdd1/brick<br>>><br>>><br>>><br>>> -----Original Message-----<br>>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>>> Sent: Tuesday, June 03, 2014 12:56 PM<br>>> To: Gnan Kumar, Yalla<br>>> Cc: gluster-users@gluster.org<br>>> Subject: Re: [Gluster-users] Distributed volumes<br>>><br>>><br>>> What do gluster vol info and gluster vol status give you?<br>>><br>>> On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>>> wrote:<br>>> > Hi,<br>>> ><br>>> > I have created a distributed volume on my gluster
node. I have attached this volume to a VM on openstack. The size is 1 GB.
I have written files close to 1 GB onto the<br>>> > Volume. But when I do a ls inside the brick directory
, the volume is present only on one gluster server brick. But it is empty
on another server brick. Files are meant to be<br>>> > spread across both the bricks according to distributed volume
definition.<br>>> ><br>>> > On the VM:<br>>> > --------------<br>>> ><br>>> > # ls -al<br>>> > total 1013417<br>>> > drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>>> > drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>>> > -rw------- 1 root root
31478251520 Jun 1 21:52 file<br>>> > -rw------- 1 root root
157391257600 Jun 1 21:54 file1<br>>> > -rw------- 1 root root
629565030400 Jun 1 21:55 file2<br>>> > -rw------- 1 root root
708260659200 Jun 1 21:59 file3<br>>> > -rw------- 1 root root
6295650304 Jun 1 22:01 file4<br>>> > -rw------- 1 root root
39333801984 Jun 1 22:01 file5<br>>> > -rw------- 1 root root
78643200000 Jun 1 22:04 file6<br>>> > drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>>> > ----------<br>>> > # du -sch *<br>>> > 20.0M file<br>>> > 100.0M file1<br>>> > 400.0M file2<br>>> > 454.0M file3<br>>> > 4.0M file4<br>>> > 11.6M file5<br>>> > 0 file6<br>>> > 16.0K lost+found<br>>> > 989.7M total<br>>> > ------------------------<br>>> ><br>>> ><br>>> > On the gluster server nodes:<br>>> > -----------------------<br>>> > root@primary:/export/sdd1/brick# ll<br>>> > total 12<br>>> > drwxr-xr-x 2 root root 4096 Jun 2 04:08 ./ drwxr-xr-x
4 root root<br>>> > 4096 May 27 08:42 ../ root@primary:/export/sdd1/brick#<br>>> > --------------------------<br>>> ><br>>> > root@secondary:/export/sdd1/brick# ll total 1046536<br>>> > drwxr-xr-x 2 root root 4096 Jun 2
08:51 ./<br>>> > drwxr-xr-x 4 root root 4096 May 27 08:43
../<br>>> > -rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35<br>>> > volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>>> > root@secondary:/export/sdd1/brick#<br>>> > ---------------------------------<br>>> ><br>>> ><br>>> > Thanks<br>>> > Kumar<br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> > -----Original Message-----<br>>> > From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>>> > Sent: Monday, June 02, 2014 6:35 PM<br>>> > To: Gnan Kumar, Yalla<br>>> > Cc: gluster-users@gluster.org<br>>> > Subject: Re: [Gluster-users] Distributed volumes<br>>> ><br>>> > Just do an ls on the bricks, the paths are the same as the
mounted filesystem.<br>>> ><br>>> > On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>>> > wrote:<br>>> > > Hi All,<br>>> > ><br>>> > ><br>>> > ><br>>> > > I have created a distributed volume of 1 GB , using
two bricks from<br>>> > > two different servers.<br>>> > ><br>>> > > I have written 7 files whose sizes are a total of 1
GB.<br>>> > ><br>>> > > How can I check that files are distributed on both the
bricks ?<br>>> > ><br>>> > ><br>>> > ><br>>> > ><br>>> > ><br>>> > > Thanks<br>>> > ><br>>> > > Kumar<br>>> > ><br>>> > ><br>>> > ><br>>> > ><br>>> > > ____________________________________________________________________<br>>> > > __<br>>> > ><br>>> > ><br>>> > > This message is for the designated recipient only and
may contain<br>>> > > privileged, proprietary, or otherwise confidential information.
If<br>>> > > you have received it in error, please notify the sender
immediately<br>>> > > and delete the original. Any other use of the e-mail
by you is prohibited.<br>>> > > Where allowed by local law, electronic communications
with Accenture<br>>> > > and its affiliates, including e-mail and instant messaging<br>>> > > (including content), may be scanned by our systems for
the purposes<br>>> > > of information security and assessment of internal compliance
with<br>>> > > Accenture policy.<br>>> > > ____________________________________________________________________<br>>> > > __<br>>> > > ________________<br>>> > ><br>>> > > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>>> > > _______________________________________________<br>>> > > Gluster-users mailing list<br>>> > > Gluster-users@gluster.org<br>>> > > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>>> ><br>>> ><br>>> ><br>>> ><br>>> > ________________________________<br>>> ><br>>> > This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise confidential information.
If you have received it in error, please notify the sender immediately
and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and
its affiliates, including e-mail and instant messaging (including content),
may be scanned by our systems for the purposes of information security
and assessment of internal compliance with Accenture policy.<br>>> > ______________________________________________________________________<br>>> > ________________<br>>> ><br>>> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>>><br>>><br>>><br>><br>><br>><br>> _______________________________________________<br>> Gluster-users mailing list<br>> Gluster-users@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br><br><br>------------------------------<br><br>Message: 19<br>Date: Tue, 3 Jun 2014 08:20:46 +0000<br>From: <yalla.gnan.kumar@accenture.com><br>To: <kshlmster@gmail.com><br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID:<br>
<67765C71374B974FBFD2AD05AF438EFF0BD58729@048-CH1MPN3-362.048d.mgd.msft.net><br>
<br>Content-Type: text/plain; charset="utf-8"<br><br>Hi,<br><br>So , in which scenario, does the distributed volumes have files on both
the bricks ?<br><br><br>-----Original Message-----<br>From: Kaushal M [</font></tt><a href=mailto:kshlmster@gmail.com><tt><font size=2>mailto:kshlmster@gmail.com</font></tt></a><tt><font size=2>]
<br>Sent: Tuesday, June 03, 2014 1:19 PM<br>To: Gnan Kumar, Yalla<br>Cc: Franco Broi; gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br><br>You have only 1 file on the gluster volume, the 1GB disk image/volume that
you created. This disk image is attached to the VM as a file system, not
the gluster volume. So whatever you do in the VM's file system, affects
just the 1 disk image. The files, directories etc. you created, are inside
the disk image. So you still have just one file on the gluster volume,
not many as you are assuming.<br><br><br><br>On Tue, Jun 3, 2014 at 1:09 PM, <yalla.gnan.kumar@accenture.com>
wrote:<br>> I have created distributed volume, created a 1 GB volume on
it, and <br>> attached it to the VM and created a filesystem on it. How to
verify that the files in the vm are distributed across both the bricks
on two servers ?<br>><br>><br>><br>> -----Original Message-----<br>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> Sent: Tuesday, June 03, 2014 1:04 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>><br>><br>> Ok, what you have is a single large file (must be filesystem image??).<br>> Gluster will not stripe files, it writes different whole files to
different bricks.<br>><br>> On Tue, 2014-06-03 at 07:29 +0000, yalla.gnan.kumar@accenture.com<br>> wrote:<br>>> root@secondary:/export/sdd1/brick# gluster volume info<br>>><br>>> Volume Name: dst<br>>> Type: Distribute<br>>> Status: Started<br>>> Number of Bricks: 2<br>>> Transport-type: tcp<br>>> Bricks:<br>>> Brick1: primary:/export/sdd1/brick<br>>> Brick2: secondary:/export/sdd1/brick<br>>><br>>><br>>><br>>> -----Original Message-----<br>>> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>>> Sent: Tuesday, June 03, 2014 12:56 PM<br>>> To: Gnan Kumar, Yalla<br>>> Cc: gluster-users@gluster.org<br>>> Subject: Re: [Gluster-users] Distributed volumes<br>>><br>>><br>>> What do gluster vol info and gluster vol status give you?<br>>><br>>> On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>>> wrote:<br>>> > Hi,<br>>> ><br>>> > I have created a distributed volume on my gluster
node. I have attached this volume to a VM on openstack. The size is 1 GB.
I have written files close to 1 GB onto the<br>>> > Volume. But when I do a ls inside the brick directory
, the volume is present only on one gluster server brick. But it is empty
on another server brick. Files are meant to be<br>>> > spread across both the bricks according to distributed volume
definition.<br>>> ><br>>> > On the VM:<br>>> > --------------<br>>> ><br>>> > # ls -al<br>>> > total 1013417<br>>> > drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>>> > drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>>> > -rw------- 1 root root
31478251520 Jun 1 21:52 file<br>>> > -rw------- 1 root root
157391257600 Jun 1 21:54 file1<br>>> > -rw------- 1 root root
629565030400 Jun 1 21:55 file2<br>>> > -rw------- 1 root root
708260659200 Jun 1 21:59 file3<br>>> > -rw------- 1 root root
6295650304 Jun 1 22:01 file4<br>>> > -rw------- 1 root root
39333801984 Jun 1 22:01 file5<br>>> > -rw------- 1 root root
78643200000 Jun 1 22:04 file6<br>>> > drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>>> > ----------<br>>> > # du -sch *<br>>> > 20.0M file<br>>> > 100.0M file1<br>>> > 400.0M file2<br>>> > 454.0M file3<br>>> > 4.0M file4<br>>> > 11.6M file5<br>>> > 0 file6<br>>> > 16.0K lost+found<br>>> > 989.7M total<br>>> > ------------------------<br>>> ><br>>> ><br>>> > On the gluster server nodes:<br>>> > -----------------------<br>>> > root@primary:/export/sdd1/brick# ll total 12 drwxr-xr-x 2
root root <br>>> > 4096 Jun 2 04:08 ./ drwxr-xr-x 4 root root<br>>> > 4096 May 27 08:42 ../ root@primary:/export/sdd1/brick#<br>>> > --------------------------<br>>> ><br>>> > root@secondary:/export/sdd1/brick# ll total 1046536<br>>> > drwxr-xr-x 2 root root 4096 Jun 2
08:51 ./<br>>> > drwxr-xr-x 4 root root 4096 May 27 08:43
../<br>>> > -rw-rw-rw- 1 108 115 1073741824 Jun 2 09:35<br>>> > volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>>> > root@secondary:/export/sdd1/brick#<br>>> > ---------------------------------<br>>> ><br>>> ><br>>> > Thanks<br>>> > Kumar<br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> ><br>>> > -----Original Message-----<br>>> > From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>>> > Sent: Monday, June 02, 2014 6:35 PM<br>>> > To: Gnan Kumar, Yalla<br>>> > Cc: gluster-users@gluster.org<br>>> > Subject: Re: [Gluster-users] Distributed volumes<br>>> ><br>>> > Just do an ls on the bricks, the paths are the same as the
mounted filesystem.<br>>> ><br>>> > On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>>> > wrote:<br>>> > > Hi All,<br>>> > ><br>>> > ><br>>> > ><br>>> > > I have created a distributed volume of 1 GB , using
two bricks <br>>> > > from two different servers.<br>>> > ><br>>> > > I have written 7 files whose sizes are a total of 1
GB.<br>>> > ><br>>> > > How can I check that files are distributed on both the
bricks ?<br>>> > ><br>>> > ><br>>> > ><br>>> > ><br>>> > ><br>>> > > Thanks<br>>> > ><br>>> > > Kumar<br>>> > ><br>>> > ><br>>> > ><br>>> > ><br>>> > > _________________________________________________________________<br>>> > > ___<br>>> > > __<br>>> > ><br>>> > ><br>>> > > This message is for the designated recipient only and
may contain <br>>> > > privileged, proprietary, or otherwise confidential information.
<br>>> > > If you have received it in error, please notify the
sender <br>>> > > immediately and delete the original. Any other use of
the e-mail by you is prohibited.<br>>> > > Where allowed by local law, electronic communications
with <br>>> > > Accenture and its affiliates, including e-mail and instant
<br>>> > > messaging (including content), may be scanned by our
systems for <br>>> > > the purposes of information security and assessment
of internal <br>>> > > compliance with Accenture policy.<br>>> > > _________________________________________________________________<br>>> > > ___<br>>> > > __<br>>> > > ________________<br>>> > ><br>>> > > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>>> > > _______________________________________________<br>>> > > Gluster-users mailing list<br>>> > > Gluster-users@gluster.org<br>>> > > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>>> ><br>>> ><br>>> ><br>>> ><br>>> > ________________________________<br>>> ><br>>> > This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise confidential information.
If you have received it in error, please notify the sender immediately
and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and
its affiliates, including e-mail and instant messaging (including content),
may be scanned by our systems for the purposes of information security
and assessment of internal compliance with Accenture policy.<br>>> > ___________________________________________________________________<br>>> > ___<br>>> > ________________<br>>> ><br>>> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>>><br>>><br>>><br>><br>><br>><br>> _______________________________________________<br>> Gluster-users mailing list<br>> Gluster-users@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br><br><br>------------------------------<br><br>Message: 20<br>Date: Tue, 03 Jun 2014 16:22:16 +0800<br>From: Franco Broi <franco.broi@iongeo.com><br>To: yalla.gnan.kumar@accenture.com<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID: <1401783736.2236.333.camel@tc1><br>Content-Type: text/plain; charset="UTF-8"<br><br>On Tue, 2014-06-03 at 08:20 +0000, yalla.gnan.kumar@accenture.com<br>wrote: <br>> Hi,<br>> <br>> So , in which scenario, does the distributed volumes have files on
both the bricks ?<br><br>If you make more than 1 file.<br><br>> <br>> <br>> -----Original Message-----<br>> From: Kaushal M [</font></tt><a href=mailto:kshlmster@gmail.com><tt><font size=2>mailto:kshlmster@gmail.com</font></tt></a><tt><font size=2>]
<br>> Sent: Tuesday, June 03, 2014 1:19 PM<br>> To: Gnan Kumar, Yalla<br>> Cc: Franco Broi; gluster-users@gluster.org<br>> Subject: Re: [Gluster-users] Distributed volumes<br>> <br>> You have only 1 file on the gluster volume, the 1GB disk image/volume
that you created. This disk image is attached to the VM as a file system,
not the gluster volume. So whatever you do in the VM's file system, affects
just the 1 disk image. The files, directories etc. you created, are inside
the disk image. So you still have just one file on the gluster volume,
not many as you are assuming.<br>> <br>> <br>> <br>> On Tue, Jun 3, 2014 at 1:09 PM, <yalla.gnan.kumar@accenture.com>
wrote:<br>> > I have created distributed volume, created a 1 GB volume
on it, and <br>> > attached it to the VM and created a filesystem on it. How
to verify that the files in the vm are distributed across both the bricks
on two servers ?<br>> ><br>> ><br>> ><br>> > -----Original Message-----<br>> > From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> > Sent: Tuesday, June 03, 2014 1:04 PM<br>> > To: Gnan Kumar, Yalla<br>> > Cc: gluster-users@gluster.org<br>> > Subject: Re: [Gluster-users] Distributed volumes<br>> ><br>> ><br>> > Ok, what you have is a single large file (must be filesystem
image??).<br>> > Gluster will not stripe files, it writes different whole files
to different bricks.<br>> ><br>> > On Tue, 2014-06-03 at 07:29 +0000, yalla.gnan.kumar@accenture.com<br>> > wrote:<br>> >> root@secondary:/export/sdd1/brick# gluster volume info<br>> >><br>> >> Volume Name: dst<br>> >> Type: Distribute<br>> >> Status: Started<br>> >> Number of Bricks: 2<br>> >> Transport-type: tcp<br>> >> Bricks:<br>> >> Brick1: primary:/export/sdd1/brick<br>> >> Brick2: secondary:/export/sdd1/brick<br>> >><br>> >><br>> >><br>> >> -----Original Message-----<br>> >> From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> >> Sent: Tuesday, June 03, 2014 12:56 PM<br>> >> To: Gnan Kumar, Yalla<br>> >> Cc: gluster-users@gluster.org<br>> >> Subject: Re: [Gluster-users] Distributed volumes<br>> >><br>> >><br>> >> What do gluster vol info and gluster vol status give you?<br>> >><br>> >> On Tue, 2014-06-03 at 07:21 +0000, yalla.gnan.kumar@accenture.com<br>> >> wrote:<br>> >> > Hi,<br>> >> ><br>> >> > I have created a distributed volume on my gluster
node. I have attached this volume to a VM on openstack. The size is 1 GB.
I have written files close to 1 GB onto the<br>> >> > Volume. But when I do a ls inside the brick directory
, the volume is present only on one gluster server brick. But it is empty
on another server brick. Files are meant to be<br>> >> > spread across both the bricks according to distributed
volume definition.<br>> >> ><br>> >> > On the VM:<br>> >> > --------------<br>> >> ><br>> >> > # ls -al<br>> >> > total 1013417<br>> >> > drwxr-xr-x 3 root root
4096 Jun 1 22:03 .<br>> >> > drwxrwxr-x 3 root root
1024 Jun 1 21:24 ..<br>> >> > -rw------- 1 root root
31478251520 Jun 1 21:52 file<br>> >> > -rw------- 1 root root
157391257600 Jun 1 21:54 file1<br>> >> > -rw------- 1 root root
629565030400 Jun 1 21:55 file2<br>> >> > -rw------- 1 root root
708260659200 Jun 1 21:59 file3<br>> >> > -rw------- 1 root root
6295650304 Jun 1 22:01 file4<br>> >> > -rw------- 1 root root
39333801984 Jun 1 22:01 file5<br>> >> > -rw------- 1 root root
78643200000 Jun 1 22:04 file6<br>> >> > drwx------ 2 root root
16384 Jun 1 21:24 lost+found<br>> >> > ----------<br>> >> > # du -sch *<br>> >> > 20.0M file<br>> >> > 100.0M file1<br>> >> > 400.0M file2<br>> >> > 454.0M file3<br>> >> > 4.0M file4<br>> >> > 11.6M file5<br>> >> > 0 file6<br>> >> > 16.0K lost+found<br>> >> > 989.7M total<br>> >> > ------------------------<br>> >> ><br>> >> ><br>> >> > On the gluster server nodes:<br>> >> > -----------------------<br>> >> > root@primary:/export/sdd1/brick# ll total 12 drwxr-xr-x
2 root root <br>> >> > 4096 Jun 2 04:08 ./ drwxr-xr-x 4 root root<br>> >> > 4096 May 27 08:42 ../ root@primary:/export/sdd1/brick#<br>> >> > --------------------------<br>> >> ><br>> >> > root@secondary:/export/sdd1/brick# ll total 1046536<br>> >> > drwxr-xr-x 2 root root 4096 Jun
2 08:51 ./<br>> >> > drwxr-xr-x 4 root root 4096 May
27 08:43 ../<br>> >> > -rw-rw-rw- 1 108 115 1073741824 Jun 2
09:35<br>> >> > volume-0ec560be-997f-46da-9ec8-e9d6627f2de1<br>> >> > root@secondary:/export/sdd1/brick#<br>> >> > ---------------------------------<br>> >> ><br>> >> ></font></tt><br><tt><font size=2>> >> > Thanks<br>> >> > Kumar<br>> >> ><br>> >> ><br>> >> ><br>> >> ><br>> >> ><br>> >> ><br>> >> ><br>> >> ><br>> >> > -----Original Message-----<br>> >> > From: Franco Broi [</font></tt><a href=mailto:franco.broi@iongeo.com><tt><font size=2>mailto:franco.broi@iongeo.com</font></tt></a><tt><font size=2>]<br>> >> > Sent: Monday, June 02, 2014 6:35 PM<br>> >> > To: Gnan Kumar, Yalla<br>> >> > Cc: gluster-users@gluster.org<br>> >> > Subject: Re: [Gluster-users] Distributed volumes<br>> >> ><br>> >> > Just do an ls on the bricks, the paths are the same
as the mounted filesystem.<br>> >> ><br>> >> > On Mon, 2014-06-02 at 12:26 +0000, yalla.gnan.kumar@accenture.com<br>> >> > wrote:<br>> >> > > Hi All,<br>> >> > ><br>> >> > ><br>> >> > ><br>> >> > > I have created a distributed volume of 1 GB , using
two bricks <br>> >> > > from two different servers.<br>> >> > ><br>> >> > > I have written 7 files whose sizes are a total
of 1 GB.<br>> >> > ><br>> >> > > How can I check that files are distributed on both
the bricks ?<br>> >> > ><br>> >> > ><br>> >> > ><br>> >> > ><br>> >> > ><br>> >> > > Thanks<br>> >> > ><br>> >> > > Kumar<br>> >> > ><br>> >> > ><br>> >> > ><br>> >> > ><br>> >> > > _________________________________________________________________<br>> >> > > ___<br>> >> > > __<br>> >> > ><br>> >> > ><br>> >> > > This message is for the designated recipient only
and may contain <br>> >> > > privileged, proprietary, or otherwise confidential
information. <br>> >> > > If you have received it in error, please notify
the sender <br>> >> > > immediately and delete the original. Any other
use of the e-mail by you is prohibited.<br>> >> > > Where allowed by local law, electronic communications
with <br>> >> > > Accenture and its affiliates, including e-mail
and instant <br>> >> > > messaging (including content), may be scanned by
our systems for <br>> >> > > the purposes of information security and assessment
of internal <br>> >> > > compliance with Accenture policy.<br>> >> > > _________________________________________________________________<br>> >> > > ___<br>> >> > > __<br>> >> > > ________________<br>> >> > ><br>> >> > > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> >> > > _______________________________________________<br>> >> > > Gluster-users mailing list<br>> >> > > Gluster-users@gluster.org<br>> >> > > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> >> ><br>> >> ><br>> >> ><br>> >> ><br>> >> > ________________________________<br>> >> ><br>> >> > This message is for the designated recipient only and
may contain privileged, proprietary, or otherwise confidential information.
If you have received it in error, please notify the sender immediately
and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and
its affiliates, including e-mail and instant messaging (including content),
may be scanned by our systems for the purposes of information security
and assessment of internal compliance with Accenture policy.<br>> >> > ___________________________________________________________________<br>> >> > ___<br>> >> > ________________<br>> >> ><br>> >> > </font></tt><a href=www.accenture.com><tt><font size=2>www.accenture.com</font></tt></a><tt><font size=2><br>> >><br>> >><br>> >><br>> ><br>> ><br>> ><br>> > _______________________________________________<br>> > Gluster-users mailing list<br>> > Gluster-users@gluster.org<br>> > </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br>> <br><br><br><br><br>------------------------------<br><br>Message: 21<br>Date: Tue, 03 Jun 2014 14:21:39 +0530<br>From: Vijay Bellur <vbellur@redhat.com><br>To: yalla.gnan.kumar@accenture.com, kshlmster@gmail.com<br>Cc: gluster-users@gluster.org<br>Subject: Re: [Gluster-users] Distributed volumes<br>Message-ID: <538D8C9B.3010709@redhat.com><br>Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br><br>On 06/03/2014 01:50 PM, yalla.gnan.kumar@accenture.com wrote:<br>> Hi,<br>><br>> So , in which scenario, does the distributed volumes have files on
both the bricks ?<br>><br>><br><br>Reading the documentation for various volume types [1] can be useful to
<br>obtain answers for questions of this nature.<br><br>-Vijay<br><br>[1] <br></font></tt><a href="https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md"><tt><font size=2>https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md</font></tt></a><tt><font size=2><br><br><br><br>------------------------------<br><br>Message: 22<br>Date: Tue, 3 Jun 2014 17:10:57 +0530<br>From: Indivar Nair <indivar.nair@techterra.in><br>To: Gluster Users <gluster-users@gluster.org><br>Subject: [Gluster-users] NFS ACL Support in Gluster 3.4<br>Message-ID:<br>
<CALuPYL0CykF9Q41SKsWtOSRXnAqor02mfR6W9FAth18XwK=cXQ@mail.gmail.com><br>Content-Type: text/plain; charset="utf-8"<br><br>Hi All,<br><br>I recently upgraded a Gluster 3.3.1 installation to Gluster 3.4.<br>It was a straight forward upgrade using Yum.<br>The OS is CentOS 6.3.<br><br>The main purpose of the upgrade was to get ACL Support on NFS exports.<br>But it doesn't seem to be working.<br><br>I mounted the gluster volume using the following options -<br><br>mount -t nfs -o vers=3,mountproto=tcp,acl <gluster_server>:/volume
/mnt<br><br>The getfacl or setfacl commands does not work on any dir/files on this<br>mount.<br><br>The plan is to re-export the NFS Mounts using Samba+CTDB.<br>NFS mounts seem to give better performance than Gluster Mounts.<br><br>Am I missing something?<br><br>Regards,<br><br><br>Indivar Nair<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140603/e543c5e8/attachment-0001.html"><tt><font size=2>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140603/e543c5e8/attachment-0001.html</font></tt></a><tt><font size=2>><br><br>------------------------------<br><br>Message: 23<br>Date: Tue, 03 Jun 2014 17:26:56 +0530<br>From: Santosh Pradhan <spradhan@redhat.com><br>To: Indivar Nair <indivar.nair@techterra.in>,
Gluster Users<br>
<gluster-users@gluster.org><br>Subject: Re: [Gluster-users] NFS ACL Support in Gluster 3.4<br>Message-ID: <538DB808.5050802@redhat.com><br>Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"<br><br>I guess Gluster 3.5 has fixed the NFS-ACL issues and getfacl/setfacl <br>works there.<br><br>Regards,<br>Santosh<br><br>On 06/03/2014 05:10 PM, Indivar Nair wrote:<br>> Hi All,<br>><br>> I recently upgraded a Gluster 3.3.1 installation to Gluster 3.4.<br>> It was a straight forward upgrade using Yum.<br>> The OS is CentOS 6.3.<br>><br>> The main purpose of the upgrade was to get ACL Support on NFS exports.<br>> But it doesn't seem to be working.<br>><br>> I mounted the gluster volume using the following options -<br>><br>> mount -t nfs -o vers=3,mountproto=tcp,acl <gluster_server>:/volume
/mnt<br>><br>> The getfacl or setfacl commands does not work on any dir/files on
this <br>> mount.<br>><br>> The plan is to re-export the NFS Mounts using Samba+CTDB.<br>> NFS mounts seem to give better performance than Gluster Mounts.<br>><br>> Am I missing something?<br>><br>> Regards,<br>><br>><br>> Indivar Nair<br>><br>><br>><br>><br>> _______________________________________________<br>> Gluster-users mailing list<br>> Gluster-users@gluster.org<br>> </font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <</font></tt><a href="http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140603/04fb291d/attachment-0001.html"><tt><font size=2>http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140603/04fb291d/attachment-0001.html</font></tt></a><tt><font size=2>><br><br>------------------------------<br><br>_______________________________________________<br>Gluster-users mailing list<br>Gluster-users@gluster.org<br></font></tt><a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"><tt><font size=2>http://supercolony.gluster.org/mailman/listinfo/gluster-users</font></tt></a><tt><font size=2><br><br>End of Gluster-users Digest, Vol 74, Issue 3<br>********************************************<br></font></tt><br><BR>
**<BR>
<BR>
This email and any attachments may contain information that is confidential and/or privileged for the sole use of the intended recipient. Any use, review, disclosure, copying, distribution or reliance by others, and any forwarding of this email or its contents, without the express permission of the sender is strictly prohibited by law. If you are not the intended recipient, please contact the sender immediately, delete the e-mail and destroy all copies.<BR>
**<BR>