redis driver raise exception when node removed

Bug #1642526 reported by wang jian
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
DragonFlow
New
Undecided
Unassigned

Bug Description

set enable_port_status_notifier=True for neutron-server, then remove a redis node
2016-11-17 17:37:19.847 115759 WARNING dragonflow.db.pubsub_drivers.redis_db_pubsub_driver [req-bdb4503b-73e5-4362-98ae-b56b66de9dd3 - -] subscriber listening task lost connection Error 111 connecting to 10.248.92.227:4002. ECONNREFUSED.
2016-11-17 17:37:19.848 115759 INFO dragonflow.db.drivers.redis_mgt [req-bdb4503b-73e5-4362-98ae-b56b66de9dd3 - -] remove node 10.248.92.227:4002 from redis master list
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl
    func(*args, **kwargs)
  File "/opt/stack/dragonflow/dragonflow/db/pubsub_drivers/redis_db_pubsub_driver.py", line 239, in run
    self.register_hamsg_for_db()
  File "/opt/stack/dragonflow/dragonflow/db/pubsub_drivers/redis_db_pubsub_driver.py", line 179, in register_hamsg_for_db
    self.redis_mgt.register_ha_topic()
  File "/opt/stack/dragonflow/dragonflow/db/drivers/redis_mgt.py", line 334, in register_ha_topic
    if self.subscriber is not None:
AttributeError: 'RedisMgt' object has no attribute 'subscriber'
2016-11-17 17:37:22.636 115759 INFO dragonflow.db.pubsub_drivers.redis_db_pubsub_driver [-] publish connection old masterlist [{'ip_port': '10.248.92.227:4001', 'slot': [[0, 5460]]}, {'ip_port': '10.248.92.227:4003', 'slot': [[10923, 16383]]}]
2016-11-17 17:37:22.637 115761 WARNING dragonflow.db.drivers.redis_mgt [-] redis cluster nodes less than local, maybe there is a partition in db cluster, nodes:{'10.248.92.227:4001': {'status': 'connected', 'last_pong_rcvd': '0', 'epoch': '1', 'node_id': '023d96311c63155a039cc405a3a6d039d17ac7fc', 'role': 'master', 'last_ping_sent': '0', 'slots': [['0', '5460']], 'master_id': '-'}, '10.248.92.227:4003': {'status': 'connected', 'last_pong_rcvd': '1479375437618', 'epoch': '3', 'node_id': 'c05794146cf2beffe6ca3f51ed673829e4711ec1', 'role': 'master', 'last_ping_sent': '0', 'slots': [['10923', '16383']], 'master_id': '-'}, '10.248.92.227:4002': {'status': 'disconnected', 'last_pong_rcvd': '1479375439649', 'epoch': '2', 'node_id': '1db300860fe8bffa9019e1a5185fc29821e6c44f', 'role': 'master', 'last_ping_sent': '1479375439926', 'slots': [['5461', '10922']], 'master_id': '-'}, '10.248.92.227:4004': {'status': 'connected', 'last_pong_rcvd': '1479375441633', 'epoch': '4', 'node_id': '4f982c8410505830ea0fe5b99bddeeadd111a17d', 'role': 'slave', 'last_ping_sent': '0', 'slots': [], 'master_id': '023d96311c63155a039cc405a3a6d039d17ac7fc'}, '10.248.92.227:4006': {'status': 'connected', 'last_pong_rcvd': '1479375441131', 'epoch': '6', 'node_id': '704857fbe4665e099d5fc7e8d3b48e8f6e64309c', 'role': 'slave', 'last_ping_sent': '0', 'slots': [], 'master_id': 'c05794146cf2beffe6ca3f51ed673829e4711ec1'}}, local nodes:{'10.248.92.227:4001': {'status': 'connected', 'last_pong_rcvd': '0', 'epoch': '1', 'node_id': '023d96311c63155a039cc405a3a6d039d17ac7fc', 'role': 'master', 'last_ping_sent': '0', 'slots': [['0', '5460']], 'master_id': '-'}, '10.248.92.227:4003': {'status': 'connected', 'last_pong_rcvd': '1479375234189', 'epoch': '3', 'node_id': 'c05794146cf2beffe6ca3f51ed673829e4711ec1', 'role': 'master', 'last_ping_sent': '0', 'slots': [['10923', '16383']], 'master_id': '-'}, '10.248.92.227:4002': {'status': 'connected', 'last_pong_rcvd': '1479375232183', 'epoch': '2', 'node_id': '1db300860fe8bffa9019e1a5185fc29821e6c44f', 'role': 'master', 'last_ping_sent': '0', 'slots': [['5461', '10922']], 'master_id': '-'}, '10.248.92.227:4005': {'status': 'connected', 'last_pong_rcvd': '1479375235191', 'epoch': '5', 'node_id': '2e4d50cc44daf56324fa21595cdfe18c2bd1e4c5', 'role': 'slave', 'last_ping_sent': '0', 'slots': [], 'master_id': '1db300860fe8bffa9019e1a5185fc29821e6c44f'}, '10.248.92.227:4004': {'status': 'connected', 'last_pong_rcvd': '1479375231180', 'epoch': '4', 'node_id': '4f982c8410505830ea0fe5b99bddeeadd111a17d', 'role': 'slave', 'last_ping_sent': '0', 'slots': [], 'master_id': '023d96311c63155a039cc405a3a6d039d17ac7fc'}, '10.248.92.227:4006': {'status': 'connected', 'last_pong_rcvd': '1479375233186', 'epoch': '6', 'node_id': '704857fbe4665e099d5fc7e8d3b48e8f6e64309c', 'role': 'slave', 'last_ping_sent': '0', 'slots': [], 'master_id': 'c05794146cf2beffe6ca3f51ed673829e4711ec1'}}
2016-11-17 17:37:22.637 115759 INFO dragonflow.db.pubsub_drivers.redis_db_pubsub_driver [-] publish connection new masterlist [{'ip_port': '10.248.92.227:4001', 'slot': [[0, 5460]]}, {'ip_port': '10.248.92.227:4003', 'slot': [[10923, 16383]]}]

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.