Comment 4 for bug 855030

Revision history for this message
David Kranz (david-kranz) wrote :

I tried again on freshly reinstalled OS and the behavior is repeatable. I also saw one case where there was a slightly different AMPQ error that happened during instance termination:

headline: AMQPConnectionException: (503, u"COMMAND_INVALID - second 'channel.open' seen", (20, 10), 'Channel.open')

2011-09-21 11:33:05,739 DEBUG nova.utils [-] Attempting to grab semaphore "get_dhcp" for method "_get_dhcp_ip"... from (pid=1094) inner /usr/lib/pymodules/python2.7/nova/utils.py:672
2011-09-21 11:33:05,746 ERROR nova.rpc [-] Exception during message handling
(nova.rpc): TRACE: Traceback (most recent call last):
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 626, in _process_data
(nova.rpc): TRACE: ctxt.reply(rval, None)
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 673, in reply
(nova.rpc): TRACE: msg_reply(self.msg_id, *args, **kwargs)
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 781, in msg_reply
(nova.rpc): TRACE: conn.direct_send(msg_id, msg)
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 562, in __exit__
(nova.rpc): TRACE: self._done()
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 547, in _done
(nova.rpc): TRACE: self.connection.reset()
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 383, in reset
(nova.rpc): TRACE: self.channel = self.connection.channel()
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 99, in channel
(nova.rpc): TRACE: return self.transport.create_channel(self.connection)
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 197, in create_channel
(nova.rpc): TRACE: return connection.channel()
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 130, in channel
(nova.rpc): TRACE: return Channel(self, channel_id)
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 82, in __init__
(nova.rpc): TRACE: self._x_open()
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 471, in _x_open
(nova.rpc): TRACE: (20, 11), # Channel.open_ok
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", line 95, in wait
(nova.rpc): TRACE: self.channel_id, allowed_methods)
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line 231, in _wait_method
(nova.rpc): TRACE: self.wait()
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", line 97, in wait
(nova.rpc): TRACE: return self.dispatch_method(method_sig, args, content)
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", line 115, in dispatch_method
(nova.rpc): TRACE: return amqp_method(self, args)
(nova.rpc): TRACE: File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line 380, in _close
(nova.rpc): TRACE: raise AMQPConnectionException(reply_code, reply_text, (class_id, method_id))
(nova.rpc): TRACE: AMQPConnectionException: (503, u"COMMAND_INVALID - second 'channel.open' seen", (20, 10), 'Channel.open')
(nova.rpc): TRACE:
2011-09-21 11:33:05,780 ERROR nova.rpc [-] Returning exception (503, u"COMMAND_INVALID - second 'channel.open' seen", (20, 10), 'Channel.open') to caller
2011-09-21 11:33:05,780 ERROR nova.rpc [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 626, in _process_data\n ctxt.reply(rval, None)\n', ' File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 673, in reply\n msg_reply(self.msg_id, *args, **kwargs)\n', ' File "/usr/lib/pymodules/python2.7/nova/rpc/impl_\
kombu.py", line 781, in msg_reply\n conn.direct_send(msg_id, msg)\n', ' File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 562, in __exit__\n self._done()\n', ' File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 547, in _done\n self.connection.reset()\n', ' File "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 383, in reset\n self.ch\
annel = self.connection.channel()\n', ' File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 99, in channel\n return self.transport.create_channel(self.connection)\n', ' File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 197, in create_channel\n return connection.channel()\n', ' File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqpli\
b.py", line 130, in channel\n return Channel(self, channel_id)\n', ' File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 82, in __init__\n self._x_open()\n', ' File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 471, in _x_open\n (20, 11), # Channel.open_ok\n', ' File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abst\
ract_channel.py", line 95, in wait\n self.channel_id, allowed_methods)\n', ' File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line 231, in _wait_method\n self.wait()\n', ' File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", line 97, in wait\n return self.dispatch_method(method_sig, args, content)\n', ' File "/usr/lib/pytho\
n2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", line 115, in dispatch_method\n return amqp_method(self, args)\n', ' File "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line 380, in _close\n raise AMQPConnectionException(reply_code, reply_text, (class_id, method_id))\n', 'AMQPConnectionException: (503, u"COMMAND_INVALID - second \'channel.open\' s\
een", (20, 10), \'Channel.open\')\n']