<div dir="ltr">Hi Mohammed,<div><br></div><div>You are right that mounting it this way will do the appropriate replication. However, there are problems with that for my use case:</div><div><br></div><div>1. I want the mount /etc/fstab to be able to fail over to any one of the three servers that I have. so if one server is down, the client can still mount from servers 2 and 3.</div><div>2. i have configured SSL on the I/O path and I need to be able to configure the client to use TLS when it connects to the brick. I was only able to get that to work with: transport.socket.ssl-enabled off in the configuration file.</div><div><br></div><div>In other words, I was only able to get HA during a mount and TLS to work by using the volume config file and setting that in the /etc/fstab.</div><div><br></div><div><a href="https://www.jamescoyle.net/how-to/439-mount-a-glusterfs-volume">https://www.jamescoyle.net/how-to/439-mount-a-glusterfs-volume</a><br><div><br></div><div>Is there a better way to handle this?</div><div><br></div><div>Thanks,</div><div>Joe</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 24, 2017 at 6:24 AM, Mohammed Rafi K C <span dir="ltr"><<a href="mailto:rkavunga@redhat.com" target="_blank">rkavunga@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hi Joseph,</p>
<p>I think there is gap in understanding your problem. Let me try to
give more clear picture on this,</p>
<p>First , couple of clarification points here</p>
<p>1) client graph is an internally generated configuration file
based on your volume, that said you don't need to create or edit
your own. If you want a 3-way replicated volume you have to
mention that when you create the volume.</p>
<p>2) When you mount a gluster volume, you don't need to provide any
client graph, you just need to give server hostname and volname,
it will automatically fetches the graph and start working on it
(so it does the replication based on the graph generated by
gluster management daemon)</p>
<p><br>
</p>
<p>Now let me briefly describe the procedure for creating a 3-way
replicated volume</p>
<p>1) gluster volume create <volname> replica 3
<hostname>:/<brick_path1>
<hostname>:/<brick_path2>
<hostname>:/<brick_path3></p>
<p> Note : if you give 3 more bricks , then it will create 2-way
distributed 3 way replicated volume (you can increase the
distribution by adding multiple if 3)</p>
<p> this step will automatically create the configuration file
in
/var/lib/glusterd/vols/<<wbr>volname>/trusted-<volname>.<wbr>tcp-fuse.vol<br>
</p>
<p>2) Now start the volume using gluster volume start
<volname></p>
<p>3) Fuse mount the volume in client machine using the command
mount -t glusterfs <server_hostname>:/<volname>
/<mnt></p>
<p> this will automatically fetches the configuration file and
will do the replication. You don't need to do anything</p><span class="">
<p><br>
</p>
<p>Let me know if this helps.</p>
<p><br>
</p>
<p>Regards</p>
<p>Rafi KC</p>
<p><br>
</p>
</span><div><div class="h5"><div class="m_-6947553777168776273moz-cite-prefix">On 02/24/2017 05:13 PM, Joseph
Lorenzini wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">HI Mohammed,
<div><br>
</div>
<div>Its not a bug per se, its a configuration and documentation
issue. I searched the gluster documentation pretty thoroughly
and I did not find anything that discussed the 1) client's
call graph and 2) how to specifically configure a native
glusterfs client to properly specify that call graph so that
replication will happen across multiple bricks. If its there,
then there's a pretty severe organization issue in the
documentation (I am pretty sure I ended up reading almost
every page actually).</div>
<div><br>
</div>
<div>As a result, because I was a new to gluster, my initial set
up really confused me. I would follow the instructions as
documented in official gluster docs (execute the mount
command), write data on the mount...and then only see it
replicated to a single brick. It was only after much furious
googling did I manage to figure out that that 1) i needed a
client configuration file which should be specified in
/etc/fstab and 2) that configuration block mentioned above was
the key.</div>
<div><br>
</div>
<div>I am actually planning on submitting a PR to the
documentation to cover all this. To be clear, I am sure this
is obvious to a seasoned gluster user -- but it is not at all
obvious to someone who is new to gluster such as myself.</div>
<div><br>
</div>
<div>So I am an operations engineer. I like reproducible
deployments and I like monitoring to alert me when something
is wrong. Due to human error or a bug in our deployment code,
its possible that something like not setting the client call
graph properly could happen. I wanted a way to detect this
problem so that if it does happen, it can be remediated
immediately.</div>
<div><br>
</div>
<div>Your suggestion sounds promising. I shall definitely look
into that. Though that might be a useful information to
surface up in a CLI command in a future gluster release IMHO.</div>
<div><br>
</div>
<div>Joe</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Feb 23, 2017 at 11:51 PM,
Mohammed Rafi K C <span dir="ltr"><<a href="mailto:rkavunga@redhat.com" target="_blank"></a><a class="m_-6947553777168776273moz-txt-link-abbreviated" href="mailto:rkavunga@redhat.com" target="_blank">rkavunga@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span>
<p><br>
</p>
<br>
<div class="m_-6947553777168776273m_685628515331254346moz-cite-prefix">On
02/23/2017 11:12 PM, Joseph Lorenzini wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>I have a simple replicated volume with a
replica count of 3. To ensure any file changes
(create/delete/modify) are replicated to all
bricks, I have this setting in my client
configuration.</div>
<div>
<p class="m_-6947553777168776273m_685628515331254346gmail-p1"><span class="m_-6947553777168776273m_685628515331254346gmail-s1"> volume
gv0-replicate-0<br>
</span> type cluster/replicate <br>
subvolumes gv0-client-0 gv0-client-1
gv0-client-2<br>
end-volume</p>
<p class="m_-6947553777168776273m_685628515331254346gmail-p1">And that
works as expected. My question is how one could
detect if this was not happening which could
poise a severe problem with data consistency and
replication. For example, those settings could
be omitted from the client config and then the
client will only write data to one brick and all
kinds of terrible things will start happening. I
have not found a way the gluster volume cli to
detect when that kind of problem is occurring.
For example gluster volume heal <volname>
info does not detect this problem. </p>
<p class="m_-6947553777168776273m_685628515331254346gmail-p1">Is there
any programmatic way to detect when this problem
is occurring?<br>
</p>
</div>
</div>
</blockquote>
<br>
</span> I couldn't understand how you will end up in this
situation. There is only one possibility (assuming there
is no bug :) ), ie you changed the client graph in a way
that there is only one subvolume to replica server.<br>
<br>
To check that the simply way is, there is a xlator called
meta, which provides meta data information through mount
point, similiar to linux proc file system. So you can
check the active graph through meta and see the number of
subvolumes for replica xlator<br>
<br>
for example : the directory <mount
point>/.meta/graphs/active/<vo<wbr>lname>-replicate-0/subvolumes
will have entries for each replica clients , so in your
case you should see 3 directories.<br>
<br>
<br>
Let me know if this helps.<br>
<br>
Regards<br>
Rafi KC<br>
<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<p class="m_-6947553777168776273m_685628515331254346gmail-p1">Thanks,<br>
Joe</p>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<fieldset class="m_-6947553777168776273m_685628515331254346mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Gluster-users mailing list
<a class="m_-6947553777168776273m_685628515331254346moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a class="m_-6947553777168776273m_685628515331254346moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a></pre>
</blockquote>
</div>
</blockquote></div>
</div>
</blockquote>
</div></div></div></blockquote></div><br></div>