Activity log for bug #1431386

Date Who What changed Old value New value Message
2015-03-12 14:27:26 Leontii Istomin bug added bug
2015-03-12 14:55:47 Leontii Istomin attachment added fuel-snapshot-2015-03-12_14-26-11.tgz https://bugs.launchpad.net/fuel/+bug/1431386/+attachment/4342907/+files/fuel-snapshot-2015-03-12_14-26-11.tgz
2015-03-12 14:59:25 Leontii Istomin description [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions: 2014.2-6.1: VERSION: api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510}, {running_applications, [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"}, {rabbitmq_management,"RabbitMQ Management Console","3.3.5"}, {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"}, {rabbit,"RabbitMQ","3.3.5"}, {os_mon,"CPO CXC 138 46","2.2.7"}, {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"}, {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"}, {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"}, {amqp_client,"RabbitMQ AMQP Client","3.3.5"}, {xmerl,"XML parser","1.2.10"}, {inets,"INETS CXC 138 49","5.7.1"}, {mnesia,"MNESIA CXC 138 12","4.5"}, {sasl,"SASL CXC 138 11","2.1.10"}, {stdlib,"ERTS CXC 138 10","1.17.5"}, {kernel,"ERTS CXC 138 10","2.14.5"}]}, {os,{unix,linux}}, {erlang_version, "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"}, {memory, [{total,208516560}, {connection_procs,8229392}, {queue_procs,34821296}, {plugins,10775712}, {other_proc,17252640}, {mnesia,5254824}, {mgmt_db,15881128}, {msg_index,2102624}, {other_ets,3191376}, {binary,20071880}, {code,17784544}, {atom,1625361}, {other_system,71525783}]}, {alarms,[]}, {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,3355982233}, {disk_free_limit,50000000}, {disk_free,9421512704}, {file_descriptors, [{total_limit,102300}, {total_used,217}, {sockets_limit,92068}, {sockets_used,214}]}, {processes,[{limit,1048576},{used,6397}]}, {run_queue,0}, {uptime,64870}] ...done. DG is generating [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions:   2014.2-6.1:     VERSION:       api: '1.0'       astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938       build_id: 2015-03-02_14-00-04       build_number: '154'       feature_groups:       - mirantis       fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343       fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718       nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605       ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46       production: docker       python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694       release: '6.1' Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 Nodes even haven't been provisoned. They are still in bootstrap. I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510},  {running_applications,      [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"},       {rabbitmq_management,"RabbitMQ Management Console","3.3.5"},       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"},       {rabbit,"RabbitMQ","3.3.5"},       {os_mon,"CPO CXC 138 46","2.2.7"},       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"},       {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"},       {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"},       {amqp_client,"RabbitMQ AMQP Client","3.3.5"},       {xmerl,"XML parser","1.2.10"},       {inets,"INETS CXC 138 49","5.7.1"},       {mnesia,"MNESIA CXC 138 12","4.5"},       {sasl,"SASL CXC 138 11","2.1.10"},       {stdlib,"ERTS CXC 138 10","1.17.5"},       {kernel,"ERTS CXC 138 10","2.14.5"}]},  {os,{unix,linux}},  {erlang_version,      "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"},  {memory,      [{total,208516560},       {connection_procs,8229392},       {queue_procs,34821296},       {plugins,10775712},       {other_proc,17252640},       {mnesia,5254824},       {mgmt_db,15881128},       {msg_index,2102624},       {other_ets,3191376},       {binary,20071880},       {code,17784544},       {atom,1625361},       {other_system,71525783}]},  {alarms,[]},  {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},  {vm_memory_high_watermark,0.4},  {vm_memory_limit,3355982233},  {disk_free_limit,50000000},  {disk_free,9421512704},  {file_descriptors,      [{total_limit,102300},       {total_used,217},       {sockets_limit,92068},       {sockets_used,214}]},  {processes,[{limit,1048576},{used,6397}]},  {run_queue,0},  {uptime,64870}] ...done. DG is generating
2015-03-12 15:00:13 Leontii Istomin description [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions:   2014.2-6.1:     VERSION:       api: '1.0'       astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938       build_id: 2015-03-02_14-00-04       build_number: '154'       feature_groups:       - mirantis       fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343       fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718       nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605       ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46       production: docker       python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694       release: '6.1' Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 Nodes even haven't been provisoned. They are still in bootstrap. I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510},  {running_applications,      [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"},       {rabbitmq_management,"RabbitMQ Management Console","3.3.5"},       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"},       {rabbit,"RabbitMQ","3.3.5"},       {os_mon,"CPO CXC 138 46","2.2.7"},       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"},       {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"},       {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"},       {amqp_client,"RabbitMQ AMQP Client","3.3.5"},       {xmerl,"XML parser","1.2.10"},       {inets,"INETS CXC 138 49","5.7.1"},       {mnesia,"MNESIA CXC 138 12","4.5"},       {sasl,"SASL CXC 138 11","2.1.10"},       {stdlib,"ERTS CXC 138 10","1.17.5"},       {kernel,"ERTS CXC 138 10","2.14.5"}]},  {os,{unix,linux}},  {erlang_version,      "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"},  {memory,      [{total,208516560},       {connection_procs,8229392},       {queue_procs,34821296},       {plugins,10775712},       {other_proc,17252640},       {mnesia,5254824},       {mgmt_db,15881128},       {msg_index,2102624},       {other_ets,3191376},       {binary,20071880},       {code,17784544},       {atom,1625361},       {other_system,71525783}]},  {alarms,[]},  {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},  {vm_memory_high_watermark,0.4},  {vm_memory_limit,3355982233},  {disk_free_limit,50000000},  {disk_free,9421512704},  {file_descriptors,      [{total_limit,102300},       {total_used,217},       {sockets_limit,92068},       {sockets_used,214}]},  {processes,[{limit,1048576},{used,6397}]},  {run_queue,0},  {uptime,64870}] ...done. DG is generating [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions:   2014.2-6.1:     VERSION:       api: '1.0'       astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938       build_id: 2015-03-02_14-00-04       build_number: '154'       feature_groups:       - mirantis       fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343       fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718       nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605       ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46       production: docker       python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694       release: '6.1' Baremetal, Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 Nodes even haven't been provisoned. They are still in bootstrap. I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510},  {running_applications,      [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"},       {rabbitmq_management,"RabbitMQ Management Console","3.3.5"},       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"},       {rabbit,"RabbitMQ","3.3.5"},       {os_mon,"CPO CXC 138 46","2.2.7"},       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"},       {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"},       {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"},       {amqp_client,"RabbitMQ AMQP Client","3.3.5"},       {xmerl,"XML parser","1.2.10"},       {inets,"INETS CXC 138 49","5.7.1"},       {mnesia,"MNESIA CXC 138 12","4.5"},       {sasl,"SASL CXC 138 11","2.1.10"},       {stdlib,"ERTS CXC 138 10","1.17.5"},       {kernel,"ERTS CXC 138 10","2.14.5"}]},  {os,{unix,linux}},  {erlang_version,      "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"},  {memory,      [{total,208516560},       {connection_procs,8229392},       {queue_procs,34821296},       {plugins,10775712},       {other_proc,17252640},       {mnesia,5254824},       {mgmt_db,15881128},       {msg_index,2102624},       {other_ets,3191376},       {binary,20071880},       {code,17784544},       {atom,1625361},       {other_system,71525783}]},  {alarms,[]},  {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},  {vm_memory_high_watermark,0.4},  {vm_memory_limit,3355982233},  {disk_free_limit,50000000},  {disk_free,9421512704},  {file_descriptors,      [{total_limit,102300},       {total_used,217},       {sockets_limit,92068},       {sockets_used,214}]},  {processes,[{limit,1048576},{used,6397}]},  {run_queue,0},  {uptime,64870}] ...done. DG is generating
2015-03-12 15:05:25 Leontii Istomin description [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions:   2014.2-6.1:     VERSION:       api: '1.0'       astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938       build_id: 2015-03-02_14-00-04       build_number: '154'       feature_groups:       - mirantis       fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343       fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718       nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605       ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46       production: docker       python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694       release: '6.1' Baremetal, Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 Nodes even haven't been provisoned. They are still in bootstrap. I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510},  {running_applications,      [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"},       {rabbitmq_management,"RabbitMQ Management Console","3.3.5"},       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"},       {rabbit,"RabbitMQ","3.3.5"},       {os_mon,"CPO CXC 138 46","2.2.7"},       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"},       {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"},       {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"},       {amqp_client,"RabbitMQ AMQP Client","3.3.5"},       {xmerl,"XML parser","1.2.10"},       {inets,"INETS CXC 138 49","5.7.1"},       {mnesia,"MNESIA CXC 138 12","4.5"},       {sasl,"SASL CXC 138 11","2.1.10"},       {stdlib,"ERTS CXC 138 10","1.17.5"},       {kernel,"ERTS CXC 138 10","2.14.5"}]},  {os,{unix,linux}},  {erlang_version,      "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"},  {memory,      [{total,208516560},       {connection_procs,8229392},       {queue_procs,34821296},       {plugins,10775712},       {other_proc,17252640},       {mnesia,5254824},       {mgmt_db,15881128},       {msg_index,2102624},       {other_ets,3191376},       {binary,20071880},       {code,17784544},       {atom,1625361},       {other_system,71525783}]},  {alarms,[]},  {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},  {vm_memory_high_watermark,0.4},  {vm_memory_limit,3355982233},  {disk_free_limit,50000000},  {disk_free,9421512704},  {file_descriptors,      [{total_limit,102300},       {total_used,217},       {sockets_limit,92068},       {sockets_used,214}]},  {processes,[{limit,1048576},{used,6397}]},  {run_queue,0},  {uptime,64870}] ...done. DG is generating [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions:   2014.2-6.1:     VERSION:       api: '1.0'       astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938       build_id: 2015-03-02_14-00-04       build_number: '154'       feature_groups:       - mirantis       fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343       fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718       nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605       ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46       production: docker       python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694       release: '6.1' Baremetal, Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 Nodes even haven't been provisoned. They are still in bootstrap. I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510},  {running_applications,      [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"},       {rabbitmq_management,"RabbitMQ Management Console","3.3.5"},       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"},       {rabbit,"RabbitMQ","3.3.5"},       {os_mon,"CPO CXC 138 46","2.2.7"},       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"},       {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"},       {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"},       {amqp_client,"RabbitMQ AMQP Client","3.3.5"},       {xmerl,"XML parser","1.2.10"},       {inets,"INETS CXC 138 49","5.7.1"},       {mnesia,"MNESIA CXC 138 12","4.5"},       {sasl,"SASL CXC 138 11","2.1.10"},       {stdlib,"ERTS CXC 138 10","1.17.5"},       {kernel,"ERTS CXC 138 10","2.14.5"}]},  {os,{unix,linux}},  {erlang_version,      "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"},  {memory,      [{total,208516560},       {connection_procs,8229392},       {queue_procs,34821296},       {plugins,10775712},       {other_proc,17252640},       {mnesia,5254824},       {mgmt_db,15881128},       {msg_index,2102624},       {other_ets,3191376},       {binary,20071880},       {code,17784544},       {atom,1625361},       {other_system,71525783}]},  {alarms,[]},  {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},  {vm_memory_high_watermark,0.4},  {vm_memory_limit,3355982233},  {disk_free_limit,50000000},  {disk_free,9421512704},  {file_descriptors,      [{total_limit,102300},       {total_used,217},       {sockets_limit,92068},       {sockets_used,214}]},  {processes,[{limit,1048576},{used,6397}]},  {run_queue,0},  {uptime,64870}] ...done. [root@ee1f879b6011 ~]# rabbitmqctl list_queues Listing queues ... amq.gen-2wXBIlw1FIzdmdxzF-sG3Q 0 amq.gen-9eKeOiM17ib56w8WpHxvZw 0 amq.gen-G_HXOSSZYDDM4NSBAsBPUg 0 amq.gen-TIXr-6FXTbBHboSv9e4frw 0 amq.gen-pONboF2R3fXeOOTTfkYsBw 0 amq.gen-yKvDYTBIbt0XbrRrtObLsQ 0 amq.gen-ym_nLWc_QnljJYUqqsAJ0A 0 nailgun 0 naily 0 ...done. DG is generating
2015-03-12 16:39:49 Nastya Urlapova fuel: milestone 6.1
2015-03-12 16:39:55 Nastya Urlapova fuel: assignee Vladimir Sharshov (vsharshov)
2015-03-12 16:39:58 Nastya Urlapova fuel: importance Undecided High
2015-03-12 17:37:21 Leontii Istomin description [root@fuel ~]# fuel --fuel-version DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient. api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' release_versions:   2014.2-6.1:     VERSION:       api: '1.0'       astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938       build_id: 2015-03-02_14-00-04       build_number: '154'       feature_groups:       - mirantis       fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343       fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718       nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605       ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46       production: docker       python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694       release: '6.1' Baremetal, Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 Nodes even haven't been provisoned. They are still in bootstrap. I got the following error during deployment cluster Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details From mcollective log of controller node with uid=1: 2015-03-11 20:30:42 ERR 20:30:42.710538 #8391] ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer [root@fuel ~]# dockerctl shell rabbitmq [root@ee1f879b6011 ~]# rabbitmqctl status Status of node rabbit@ee1f879b6011 ... [{pid,1510},  {running_applications,      [{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.3.5"},       {rabbitmq_management,"RabbitMQ Management Console","3.3.5"},       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.3.5"},       {rabbit,"RabbitMQ","3.3.5"},       {os_mon,"CPO CXC 138 46","2.2.7"},       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.3.5"},       {webmachine,"webmachine","1.10.3-rmq3.3.5-gite9359c7"},       {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.3.5-git680dba8"},       {amqp_client,"RabbitMQ AMQP Client","3.3.5"},       {xmerl,"XML parser","1.2.10"},       {inets,"INETS CXC 138 49","5.7.1"},       {mnesia,"MNESIA CXC 138 12","4.5"},       {sasl,"SASL CXC 138 11","2.1.10"},       {stdlib,"ERTS CXC 138 10","1.17.5"},       {kernel,"ERTS CXC 138 10","2.14.5"}]},  {os,{unix,linux}},  {erlang_version,      "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:6:6] [rq:6] [async-threads:30] [kernel-poll:true]\n"},  {memory,      [{total,208516560},       {connection_procs,8229392},       {queue_procs,34821296},       {plugins,10775712},       {other_proc,17252640},       {mnesia,5254824},       {mgmt_db,15881128},       {msg_index,2102624},       {other_ets,3191376},       {binary,20071880},       {code,17784544},       {atom,1625361},       {other_system,71525783}]},  {alarms,[]},  {listeners,[{clustering,41055,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},  {vm_memory_high_watermark,0.4},  {vm_memory_limit,3355982233},  {disk_free_limit,50000000},  {disk_free,9421512704},  {file_descriptors,      [{total_limit,102300},       {total_used,217},       {sockets_limit,92068},       {sockets_used,214}]},  {processes,[{limit,1048576},{used,6397}]},  {run_queue,0},  {uptime,64870}] ...done. [root@ee1f879b6011 ~]# rabbitmqctl list_queues Listing queues ... amq.gen-2wXBIlw1FIzdmdxzF-sG3Q 0 amq.gen-9eKeOiM17ib56w8WpHxvZw 0 amq.gen-G_HXOSSZYDDM4NSBAsBPUg 0 amq.gen-TIXr-6FXTbBHboSv9e4frw 0 amq.gen-pONboF2R3fXeOOTTfkYsBw 0 amq.gen-yKvDYTBIbt0XbrRrtObLsQ 0 amq.gen-ym_nLWc_QnljJYUqqsAJ0A 0 nailgun 0 naily 0 ...done. DG is generating api: '1.0' astute_sha: 1be5b9b827f512d740fe907c7ff72486d4030938 auth_required: true build_id: 2015-03-02_14-00-04 build_number: '154' feature_groups: - mirantis fuellib_sha: b17e3810dbca407fca2a231c26f553a46e166343 fuelmain_sha: baf24424a4e056c6753913de5f8c94851903f718 nailgun_sha: f034fbb4b68be963e4dc5b5d680061b54efbf605 ostf_sha: 103d6cf6badd57b791cfaf4310ec8bd81c7a8a46 production: docker python-fuelclient_sha: 3ebfa9c14a192d0298ff787526bf990055a23694 release: '6.1' I tried to deploy the following configuration with 200 baremetal nodes: Baremetal, Ubuntu, HA, Neutron-gre,Ceph-all, Debug, 6.1_154 Controllers:3 Computes:197 I got the following error during deployment cluster: Mcollective problem with nodes [{"uid"=>"1", "error"=>"Node not answered by RPC."}], please check log for details Nodes even haven't been provisoned. They are still in bootstrap. From mcollective log of controller node with uid=1: ERROR -- : rabbitmq.rb:30:in `on_miscerr' Unexpected error on connection stomp://mcollective@10.20.0.2:61613: es_oldrecv: receive failed: Connection reset by peer
2015-03-12 17:56:24 Łukasz Oleś fuel: status New Incomplete
2015-03-13 07:34:25 Sergii Golovatiuk fuel: status Incomplete Confirmed
2015-03-13 12:10:12 Vladimir Sharshov fuel: status Confirmed Invalid
2015-07-08 04:45:47 Sergii Golovatiuk fuel: status Invalid Confirmed
2015-07-08 04:46:09 Sergii Golovatiuk nominated for series fuel/7.0.x
2015-07-08 04:46:09 Sergii Golovatiuk bug task added fuel/7.0.x
2015-07-08 05:19:15 Sergii Golovatiuk nominated for series fuel/6.1.x
2015-07-08 05:19:15 Sergii Golovatiuk bug task added fuel/6.1.x
2015-07-08 05:19:35 Sergii Golovatiuk fuel/6.1.x: milestone 6.1-mu-1
2015-07-08 05:19:57 Sergii Golovatiuk fuel/6.1.x: assignee Fuel Python Team (fuel-python)
2015-07-08 05:20:01 Sergii Golovatiuk fuel/6.1.x: status New Confirmed
2015-07-08 05:20:03 Sergii Golovatiuk fuel/6.1.x: importance Undecided High
2015-07-08 05:20:09 Sergii Golovatiuk fuel/7.0.x: assignee Vladimir Sharshov (vsharshov) Fuel Python Team (fuel-python)
2015-07-08 09:47:51 Vitaly Sedelnik fuel/6.1.x: milestone 6.1-mu-1 6.1-updates
2015-07-08 09:48:07 Vitaly Sedelnik fuel/7.0.x: milestone 6.1 7.0
2015-07-14 12:13:59 Vladimir Kuklin fuel/6.1.x: status Confirmed Won't Fix
2015-07-21 10:02:01 Aleksandr Didenko tags scale scale tricky
2015-08-04 16:22:17 Dmitry Mescheryakov fuel/7.0.x: assignee Fuel Python Team (fuel-python) Dmitry Mescheryakov (dmitrymex)
2015-08-06 14:00:56 Igor Marnat tags scale tricky fuel-to-mos scale tricky
2015-08-21 10:37:40 OpenStack Infra fuel: status Confirmed In Progress
2015-08-27 00:23:58 Andrey Maximov tags fuel-to-mos scale tricky feature fuel-to-mos scale tricky
2015-08-27 00:24:14 Andrey Maximov nominated for series fuel/8.0.x
2015-08-27 00:24:14 Andrey Maximov bug task added fuel/8.0.x
2015-08-27 00:24:22 Andrey Maximov fuel/8.0.x: status New Confirmed
2015-08-27 00:24:28 Andrey Maximov fuel/8.0.x: importance Undecided High
2015-08-27 00:24:39 Andrey Maximov fuel/8.0.x: assignee Fuel Python Team (fuel-python)
2015-08-27 00:24:52 Andrey Maximov fuel/8.0.x: milestone 8.0
2015-09-01 12:37:56 Andrey Maximov fuel/7.0.x: status In Progress Won't Fix
2015-09-23 16:47:50 Matthew Mosesohn fuel/7.0.x: assignee Dmitry Mescheryakov (dmitrymex) Fuel Python Team (fuel-python)
2015-09-24 15:04:32 Alexey Shtokolov fuel/8.0.x: status Confirmed Triaged
2015-09-24 15:04:39 Alexey Shtokolov fuel/8.0.x: importance High Medium
2015-10-12 17:19:52 Dmitry Pyzhov fuel: milestone 7.0 8.0
2015-10-12 17:19:52 Dmitry Pyzhov fuel: assignee Dmitry Mescheryakov (dmitrymex) Fuel Python Team (fuel-python)
2015-10-12 17:19:57 Dmitry Pyzhov bug task deleted fuel/8.0.x
2015-10-22 03:05:20 Dmitry Pyzhov tags feature fuel-to-mos scale tricky area-python feature fuel-to-mos scale tricky
2016-01-15 12:08:09 Alexander Kislitsky fuel: milestone 8.0 9.0
2016-04-10 21:56:04 Fuel Devops McRobotson fuel/mitaka: importance Undecided Medium
2016-04-10 21:56:04 Fuel Devops McRobotson fuel/mitaka: status New Triaged
2016-04-10 21:56:04 Fuel Devops McRobotson fuel/mitaka: milestone 9.0
2016-04-10 21:56:04 Fuel Devops McRobotson fuel/mitaka: assignee Fuel Python Team (fuel-python)
2016-04-10 21:56:07 Fuel Devops McRobotson fuel/newton: milestone 9.0 10.0
2016-04-10 21:56:09 Fuel Devops McRobotson fuel/mitaka: status Triaged Won't Fix
2016-06-22 14:35:30 Dmitry Pyzhov fuel/newton: assignee Fuel Python (Deprecated) (fuel-python) Fuel Sustaining (fuel-sustaining-team)