Comment 3 for bug 1460946

Revision history for this message
Raj Reddy (rajreddy) wrote : Re: [Bug 1460946] Re: On network disconnect-reconnect, few tor-agents not responding to their http ports

'timeout' means the daemon is busy with other task and not able to respond to introspect request
in the default 2s time..

You just need to increase the timeout in the command using for e.g.
contrail-status -t 5

-
Raj

On Jun 2, 2015, at 10:21 AM, Ashish Ranjan <email address hidden>
 wrote:

> ** Also affects: juniperopenstack/trunk
> Importance: High
> Assignee: Raj Reddy (rajreddy)
> Status: New
>
> ** Changed in: juniperopenstack/trunk
> Milestone: None => r2.30-fcs
>
> --
> You received this bug notification because you are a member of Contrail
> Systems engineering, which is subscribed to Juniper Openstack.
> https://bugs.launchpad.net/bugs/1460946
>
> Title:
> On network disconnect-reconnect, few tor-agents not responding to
> their http ports
>
> Status in Juniper Openstack distribution:
> New
> Status in Juniper Openstack r2.20 series:
> New
> Status in Juniper Openstack trunk series:
> New
>
> Bug description:
> R2.20 Build 30 Ubuntu 14.04 Juno multi-node
>
> In the tor-scale setup, there are 4 tor-agent/tsn nodes (i.e two pairs
> for active/standby) , each of them with 64 tor-agents
>
> nodei38 and nodei28 is one such pair for TORs ovs-vm1 to ovs-vm64
>
> I disconnected the control/data link on nodei38
> contrail-tor-agent-1 on nodei28 became active for ovs-vm1
>
> After about 20-30 mins, i reconnected the link on nodei38
>
> It was then seen that 3 tor-agents (like contrail-tor-agent-49) were not responding on their http ports.
> contrail-status would show them with 'timeout'
>
> root@nodei38:/var/crashes# ps aux |grep "agent-49"
> root 8672 0.0 0.0 10460 936 pts/0 S+ 12:55 0:00 grep --color=auto agent-49
> root 25622 2.7 0.0 2413912 139988 ? Sl 00:54 19:52 /usr/bin/contrail-tor-agent --config_file /etc/contrail/contrail-tor-agent-49.conf
> root@nodei38:/var/crashes#
>
> root@nodei38:/var/crashes# contrail-status |grep timeout
> contrail-tor-agent-16 timeout
> contrail-tor-agent-4 timeout
> contrail-tor-agent-49 timeout
> root@nodei38:/var/crashes#
>
> root@nodei38:/var/crashes# curl http://127.0.0.1:9058
>
> ^C
> root@nodei38:/var/crashes# lsof -p 25622 |grep 9058
> contrail- 25622 root 10u IPv4 28219150 0t0 TCP *:9058 (LISTEN)
> root@nodei38:/var/crashes#
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juniperopenstack/+bug/1460946/+subscriptions