[Gluster-users] [ovirt-users] vdsm IOProcessClient WARNING Timeout waiting for communication thread for client

[Nir Soffer]

On Fri, Jan 13, 2017 at 7:41 PM, Bill James <bill.james at j2.com> wrote:
> resending without logs, except vdsm.log since list limit is too small.
>
>
>
> On 1/13/17 8:50 AM, Bill James wrote:
>
> We have an ovirt system with 3 clusters, all running centos7.
> ovirt engine is running on separate host,
> ovirt-engine-3.6.4.1-1.el7.centos.noarch
> 2 of the clusters are running newer version of ovirt, 3 nodes each,
> ovirt-engine-4.0.3-1.el7.centos.noarch, glusterfs-3.7.16-1.el7.x86_64,
> vdsm-4.18.11-1.el7.centos.x86_64.
> 1 cluster is still running the older version,
> ovirt-engine-3.6.4.1-1.el7.centos.noarch.

Which ioprocess version?

>
> Yes we are in the process of upgrading the whole system to ovirt4.0, but
> takes time....
>
> One of the 2 clusters running ovirt4 is complaining of timeouts, vdsm
> talking to gluster. No warnings on the 2 other clusters.
>
>
>
> Thread-720062::DEBUG::2017-01-13
> 07:29:46,814::outOfProcess::87::Storage.oop::(getProcessPool) Creating
> ioprocess /rhev/data-center/mnt/glusterSD/ovirt1-gl.dmz.p
> rod.j2noc.com:_gv1
> Thread-720062::INFO::2017-01-13
> 07:29:46,814::__init__::325::IOProcessClient::(__init__) Starting client
> ioprocess-5874
> Thread-720062::DEBUG::2017-01-13
> 07:29:46,814::__init__::334::IOProcessClient::(_run) Starting ioprocess for
> client ioprocess-5874
> Thread-720062::DEBUG::2017-01-13
> 07:29:46,832::__init__::386::IOProcessClient::(_startCommunication) Starting
> communication thread for client ioprocess-5874
> Thread-720062::WARNING::2017-01-13
> 07:29:46,847::__init__::401::IOProcessClient::(_startCommunication) Timeout
> waiting for communication thread for client ioprocess-5874

This warning is harmless, it means that ioprocess thread did not start
in 1 second.

This probably means that the host is overloaded, typically new threads start
instantly.

Anyway I think we are using too short timeout. Can you open an ioprocess
bug for this?

>
>
> [2017-01-12 07:27:58.685680] I [MSGID: 106488]
> [glusterd-handler.c:1533:__glusterd_handle_cli_get_volume] 0-glusterd:
> Received get vol req
> The message "I [MSGID: 106488]
> [glusterd-handler.c:1533:__glusterd_handle_cli_get_volume] 0-glusterd:
> Received get vol req" repeated 31 times between [2017-01-12 07:27:58.685680]
> and [2017-01-12 07:29:46.971939]
>
>
> attached logs: engine.log supervdsm.log vdsm.log
> etc-glusterfs-glusterd.vol.log cli.log
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>