[Gluster-users] [Solved] Transport endpoint

[Robin]

Thanks all for commenting.

It turned out to be an easier problem.

This is an interactive login node with a lot of users logging in.
So, I put in lots of limits to prevent abuse of cpu time, number of 
processes, open file limits, etc.

Account provisioning process mistakenly puts root into a group that has 
limitations.

Unfortunately, the limits enforced by /etc/security/limits.conf is not 
logged anywhere (so, the debugging process got a lot longer).
I found out the limit when I decided to probe into /proc/<PID>/limits

To ensure the proper limits, I put the following in mount.glusterfs:

ulimit -S -s 10240
ulimit -n 65536
ulimit -t unlimited

Thanks all for commenting.

Robin



On 12/17/12 10:59 AM, Robin, Robin wrote:
> Yup. I rsync over to a Gluster mounted (and not the actual brick) via
> client over the network.
>
> Robin
>
>
> From: "Washer, Bryan" <bwasher at netsuite.com <mailto:bwasher at netsuite.com>>
> Date: Mon, 17 Dec 2012 10:47:38 -0500
> To: Joe Julian <joe at julianfamily.org <mailto:joe at julianfamily.org>>,
> "gluster-users at gluster.org <mailto:gluster-users at gluster.org>"
> <gluster-users at gluster.org <mailto:gluster-users at gluster.org>>
> Subject: Re: [Gluster-users] Transport endpoint
>
>
>
> Just to make sure we don't miss the obviousŠwhen you say sync'd over to
> the mount pointŠyou mean where you did a glusterfs mount and not eh actual
> location of the brick on one of the mirrors in your replica.
>
> Once you setup the volume and start it.you should NEVER write or delete
> directly on the backend brick unless you really know what you are doing.
>
> Bryan
>
> -----Original Message-----
> From: Joe Julian <joe at julianfamily.org <mailto:joe at julianfamily.org>>
> Date: Monday, December 17, 2012 9:29 AM
> To: "gluster-users at gluster.org <mailto:gluster-users at gluster.org>"
> <gluster-users at gluster.org <mailto:gluster-users at gluster.org>>
> Subject: Re: [Gluster-users] Transport endpoint
>
>>On 12/17/2012 06:56 AM, Robin, Robin wrote:
>>> Hi,
>>>
>>> I've got Gluster error: Transport endpoint not connected.
>>>
>>> It came up twice after trying to rsync 2 TB filesystem over; it
>>> reached about 1.8 TB and got the error.
>>>
>>> Logs on the server side (on reverse time order):
>>> [2012-12-15 00:53:24.747934] I
>>> [server-helpers.c:629:server_connection_destroy]
>>> 0-RedhawkShared-server: destroyed connection of
>>>
>>>mualhpcp01.hpc.muohio.edu-17684-2012/12/13-17:25:16:994209-RedhawkShared-
>>>client-0-0
>>> [2012-12-15 00:53:24.743459] I [server-helpers.c:474:do_fd_cleanup]
>>> 0-RedhawkShared-server: fd cleanup on
>>>
>>>/mkennedy/tramelot_nwfs/rpr3/rpr3/rpr3_sparky/matrix/.4d_ccnoesy.ucsf.QTQ
>>>swL
>>> [2012-12-15 00:53:24.743430] I
>>> [server-helpers.c:330:do_lock_table_cleanup] 0-RedhawkShared-server:
>>> finodelk released on
>>>
>>>/mkennedy/tramelot_nwfs/rpr3/rpr3/rpr3_sparky/matrix/.4d_ccnoesy.ucsf.QTQ
>>>swL
>>> [2012-12-15 00:53:24.743400] I
>>> [server-helpers.c:741:server_connection_put] 0-RedhawkShared-server:
>>> Shutting down connection
>>>
>>>mualhpcp01.hpc.muohio.edu-17684-2012/12/13-17:25:16:994209-RedhawkShared-
>>>client-0-0
>>> [2012-12-15 00:53:24.743368] I [server.c:685:server_rpc_notify]
>>> 0-RedhawkShared-server: disconnecting connectionfrom
>>>
>>>mualhpcp01.hpc.muohio.edu-17684-2012/12/13-17:25:16:994209-RedhawkShared-
>>>client-0-0
>>> [2012-12-15 00:53:24.740055] W [socket.c:195:__socket_rwv]
>>> 0-tcp.RedhawkShared-server: readv failed (Connection reset by peer)
>>>
>>> I can't find relevant logs on the client side.
>>>
>>> From the logs, can we judge for sure that this is a network reset
>>> problem ?
>>>
>>When you say, "I can't find relevant logs on the client side," do you
>>mean that you can't find the log, or that there's nothing in there from
>>around the same timestamp? The client log will be in /var/log/glusterfs
>>and will be named based on the mountpoint.
>>_______________________________________________
>>Gluster-users mailing list
>>Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>>http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>
>
>
> NOTICE: This email and any attachments may contain confidential and
> proprietary information of NetSuite Inc. and is for the sole use of the
> intended recipient for the stated purpose. Any improper use or
> distribution is prohibited. If you are not the intended recipient,
> please notify the sender; do not review, copy or distribute; and
> promptly delete or destroy all transmitted information. Please note that
> all communications and information transmitted through this email system
> may be monitored by NetSuite or its agents and that all incoming email
> is automatically scanned by a third party spam and filtering service.